[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:15:29.810751  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:46707
I20260709 13:15:29.812650  8925 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:15:29.813822  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:29.832376  8931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.833077  8932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:29.834988  8934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:29.834517  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:31.098161  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:31.098498  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:31.098680  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602931098660 us; error 0 us; skew 500 ppm
I20260709 13:15:31.099664  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:31.106992  8925 webserver.cc:533] Webserver started at http://127.8.183.126:45525/ using document root <none> and password file <none>
I20260709 13:15:31.108254  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:31.108520  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:31.109086  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:31.116595  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "ba146d0315bf4b2bbab46eeed0927c3c"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.126672  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:15:31.133577  8940 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.135608  8925 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:15:31.136128  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "ba146d0315bf4b2bbab46eeed0927c3c"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.136583  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:31.205829  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:31.207811  8925 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:15:31.208397  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:31.296351  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:46707
I20260709 13:15:31.296471  8991 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:46707 every 8 connection(s)
I20260709 13:15:31.306118  8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:31.338603  8992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: Bootstrap starting.
I20260709 13:15:31.345767  8992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:31.348096  8992 log.cc:826] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: Log is configured to *not* fsync() on all Append() calls
I20260709 13:15:31.353523  8992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: No bootstrap required, opened a new log
I20260709 13:15:31.377056  8992 raft_consensus.cc:359] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER }
I20260709 13:15:31.377722  8992 raft_consensus.cc:385] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:31.378027  8992 raft_consensus.cc:740] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba146d0315bf4b2bbab46eeed0927c3c, State: Initialized, Role: FOLLOWER
I20260709 13:15:31.379087  8992 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER }
I20260709 13:15:31.379798  8992 raft_consensus.cc:399] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:31.380110  8992 raft_consensus.cc:493] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:31.380545  8992 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:31.385511  8992 raft_consensus.cc:515] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER }
I20260709 13:15:31.386691  8992 leader_election.cc:304] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba146d0315bf4b2bbab46eeed0927c3c; no voters: 
I20260709 13:15:31.389146  8992 leader_election.cc:290] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:31.389690  8995 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:31.392879  8995 raft_consensus.cc:697] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 1 LEADER]: Becoming Leader. State: Replica: ba146d0315bf4b2bbab46eeed0927c3c, State: Running, Role: LEADER
I20260709 13:15:31.394040  8995 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER }
I20260709 13:15:31.395723  8992 sys_catalog.cc:565] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:31.407526  8996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER } }
I20260709 13:15:31.407894  8997 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba146d0315bf4b2bbab46eeed0927c3c. Latest consensus state: current_term: 1 leader_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba146d0315bf4b2bbab46eeed0927c3c" member_type: VOTER } }
I20260709 13:15:31.408581  8996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:31.408924  8997 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:31.412846  9005 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:31.427328  9005 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:31.430495  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:31.448393  9005 catalog_manager.cc:1383] Generated new cluster ID: 5335913b55fc4bb5b471e279cc23d752
I20260709 13:15:31.448771  9005 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:31.475812  9005 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:31.477908  9005 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:31.498562  9005 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: Generated new TSK 0
I20260709 13:15:31.500142  9005 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:31.562620  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:31.573362  9014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:31.574837  9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:31.579952  9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:31.580672  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:31.581749  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:31.582072  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:31.582304  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602931582287 us; error 0 us; skew 500 ppm
I20260709 13:15:31.583246  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:31.588114  8925 webserver.cc:533] Webserver started at http://127.8.183.65:38011/ using document root <none> and password file <none>
I20260709 13:15:31.588971  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:31.589284  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:31.589670  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:31.591363  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "7616ac7f73484817ab55d8589988ad44"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.599416  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:31.605572  9022 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.606673  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:15:31.607052  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "7616ac7f73484817ab55d8589988ad44"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.607525  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:31.631390  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:31.633006  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:31.635428  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:31.641477  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:31.641804  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.642117  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:31.642325  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.762518  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:43527
I20260709 13:15:31.762676  9084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:43527 every 8 connection(s)
I20260709 13:15:31.771292  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:31.786309  9089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:31.792668  8925 server_base.cc:1061] running on GCE node
W20260709 13:15:31.792609  9090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:31.794160  9092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:31.795480  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:31.795749  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:31.795995  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602931795974 us; error 0 us; skew 500 ppm
I20260709 13:15:31.796870  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:31.800423  8925 webserver.cc:533] Webserver started at http://127.8.183.66:40385/ using document root <none> and password file <none>
I20260709 13:15:31.801038  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:31.801290  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:31.801618  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:31.803182  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.810678  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:31.811654  9085 heartbeater.cc:344] Connected to a master server at 127.8.183.126:46707
I20260709 13:15:31.812251  9085 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:31.813676  9085 heartbeater.cc:507] Master 127.8.183.126:46707 requested a full tablet report, sending...
I20260709 13:15:31.816771  9097 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.817981  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:15:31.818146  8957 ts_manager.cc:194] Registered new tserver with Master: 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527)
I20260709 13:15:31.818409  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
format_stamp: "Formatted at 2026-07-09 13:15:31 on dist-test-slave-ch3x"
I20260709 13:15:31.818866  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:31.821004  8957 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59930
I20260709 13:15:31.857292  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:31.859475  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:31.861565  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:31.865943  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:31.866364  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.866739  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:31.866954  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:31.973974  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:46553
I20260709 13:15:31.974072  9159 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:46553 every 8 connection(s)
I20260709 13:15:31.984488  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:31.994438  9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:31.996475  9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:32.002342  9160 heartbeater.cc:344] Connected to a master server at 127.8.183.126:46707
I20260709 13:15:32.002996  9160 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:32.004244  9160 heartbeater.cc:507] Master 127.8.183.126:46707 requested a full tablet report, sending...
I20260709 13:15:32.007611  8957 ts_manager.cc:194] Registered new tserver with Master: b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:32.008530  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:32.009902  8957 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59942
W20260709 13:15:32.009770  9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:32.011360  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:32.011668  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:32.011881  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602932011864 us; error 0 us; skew 500 ppm
I20260709 13:15:32.012688  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:32.015959  8925 webserver.cc:533] Webserver started at http://127.8.183.67:33707/ using document root <none> and password file <none>
I20260709 13:15:32.016671  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:32.016937  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:32.017284  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:32.018833  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
format_stamp: "Formatted at 2026-07-09 13:15:32 on dist-test-slave-ch3x"
I20260709 13:15:32.025739  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:15:32.030972  9171 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:32.032125  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:15:32.032552  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
format_stamp: "Formatted at 2026-07-09 13:15:32 on dist-test-slave-ch3x"
I20260709 13:15:32.032981  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:32.085736  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:32.087327  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:32.089753  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:32.093346  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:32.093627  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:32.093907  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:32.094105  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:32.213178  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:42907
I20260709 13:15:32.213404  9233 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:42907 every 8 connection(s)
I20260709 13:15:32.235198  9234 heartbeater.cc:344] Connected to a master server at 127.8.183.126:46707
I20260709 13:15:32.235728  9234 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:32.236821  9234 heartbeater.cc:507] Master 127.8.183.126:46707 requested a full tablet report, sending...
I20260709 13:15:32.240163  8957 ts_manager.cc:194] Registered new tserver with Master: e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:32.240913  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020983626s
I20260709 13:15:32.241947  8957 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59954
I20260709 13:15:32.305027  8957 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59970:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:32.310232  8957 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:15:32.416532  9004 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:32.458500  9122 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4de5ebc32d4a979c35122f55a6619d (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:32.459923  9124 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb34a53e97e4ad1b3cd31bc21526114 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:32.460306  9123 tablet_service.cc:1511] Processing CreateTablet for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:32.457849  9125 tablet_service.cc:1511] Processing CreateTablet for tablet e54e833462b44129b948504f0eb55714 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:32.478019  9198 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb34a53e97e4ad1b3cd31bc21526114 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:32.471066  9199 tablet_service.cc:1511] Processing CreateTablet for tablet e54e833462b44129b948504f0eb55714 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:32.480432  9122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4de5ebc32d4a979c35122f55a6619d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.471151  9197 tablet_service.cc:1511] Processing CreateTablet for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:32.481617  9196 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4de5ebc32d4a979c35122f55a6619d (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:32.483901  9198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb34a53e97e4ad1b3cd31bc21526114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.485814  9196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4de5ebc32d4a979c35122f55a6619d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.488348  9125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54e833462b44129b948504f0eb55714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.489439  9199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54e833462b44129b948504f0eb55714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.491735  9121 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad6407878c3407aa2ca069e71226e45 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:32.495100  9197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.501284  9123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.502259  9195 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad6407878c3407aa2ca069e71226e45 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:32.507947  9195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad6407878c3407aa2ca069e71226e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.508927  9120 tablet_service.cc:1511] Processing CreateTablet for tablet 290b9eb922674f538e458e07dcae691f (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:32.511919  9121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad6407878c3407aa2ca069e71226e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.514175  9120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290b9eb922674f538e458e07dcae691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.516716  9194 tablet_service.cc:1511] Processing CreateTablet for tablet 290b9eb922674f538e458e07dcae691f (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:32.520288  9194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290b9eb922674f538e458e07dcae691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.528226  9124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb34a53e97e4ad1b3cd31bc21526114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.543253  9119 tablet_service.cc:1511] Processing CreateTablet for tablet 5e00185e4919476a922eb432f336b644 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:32.551400  9193 tablet_service.cc:1511] Processing CreateTablet for tablet 5e00185e4919476a922eb432f336b644 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:32.556953  9119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e00185e4919476a922eb432f336b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.564793  9193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e00185e4919476a922eb432f336b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.564891  9050 tablet_service.cc:1511] Processing CreateTablet for tablet e54e833462b44129b948504f0eb55714 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:15:32.594728  9050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e54e833462b44129b948504f0eb55714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.575239  9048 tablet_service.cc:1511] Processing CreateTablet for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:15:32.594692  9047 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4de5ebc32d4a979c35122f55a6619d (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:15:32.595022  9049 tablet_service.cc:1511] Processing CreateTablet for tablet 2fb34a53e97e4ad1b3cd31bc21526114 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:15:32.599382  9043 tablet_service.cc:1511] Processing CreateTablet for tablet f577a8bd1f0e42c89c44580265ee5127 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:32.594937  9118 tablet_service.cc:1511] Processing CreateTablet for tablet f577a8bd1f0e42c89c44580265ee5127 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:32.605538  9046 tablet_service.cc:1511] Processing CreateTablet for tablet 2ad6407878c3407aa2ca069e71226e45 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:15:32.602147  9045 tablet_service.cc:1511] Processing CreateTablet for tablet 290b9eb922674f538e458e07dcae691f (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:15:32.594835  9044 tablet_service.cc:1511] Processing CreateTablet for tablet 5e00185e4919476a922eb432f336b644 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:15:32.610061  9047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4de5ebc32d4a979c35122f55a6619d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.609354  9118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f577a8bd1f0e42c89c44580265ee5127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.623137  9043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f577a8bd1f0e42c89c44580265ee5127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.627668  9049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fb34a53e97e4ad1b3cd31bc21526114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.631922  9046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ad6407878c3407aa2ca069e71226e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.634519  9045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 290b9eb922674f538e458e07dcae691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.638620  9192 tablet_service.cc:1511] Processing CreateTablet for tablet f577a8bd1f0e42c89c44580265ee5127 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:15:32.649082  9044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e00185e4919476a922eb432f336b644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.654227  9042 tablet_service.cc:1511] Processing CreateTablet for tablet b0ed33b99f2b44b5a2fd4925b3afd804 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:32.668237  9192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f577a8bd1f0e42c89c44580265ee5127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.683056  9048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76d8fa9ba9c3459cb5760fe37d04f4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.700917  9252 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:32.710794  9042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ed33b99f2b44b5a2fd4925b3afd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.716646  9117 tablet_service.cc:1511] Processing CreateTablet for tablet b0ed33b99f2b44b5a2fd4925b3afd804 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:32.724817  9117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ed33b99f2b44b5a2fd4925b3afd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.743201  9195 tablet_service.cc:1511] Processing CreateTablet for tablet b0ed33b99f2b44b5a2fd4925b3afd804 (DEFAULT_TABLE table=test-workload [id=d0e5b4f5a65a4fbab32a98d1addaa1b8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:15:32.747391  9195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0ed33b99f2b44b5a2fd4925b3afd804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:32.792587  9253 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:32.824252  9254 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:32.824805  9085 heartbeater.cc:499] Master 127.8.183.126:46707 was elected leader, sending a full tablet report...
I20260709 13:15:32.830679  9253 tablet_bootstrap.cc:654] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.847016  9254 tablet_bootstrap.cc:654] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.858110  9252 tablet_bootstrap.cc:654] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.870749  9253 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:32.873021  9253 ts_tablet_manager.cc:1403] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.082s	user 0.008s	sys 0.015s
I20260709 13:15:32.877897  9254 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:32.878486  9254 ts_tablet_manager.cc:1403] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.010s
I20260709 13:15:32.894205  9252 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:32.894857  9252 ts_tablet_manager.cc:1403] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.215s	user 0.027s	sys 0.000s
I20260709 13:15:32.894328  9253 raft_consensus.cc:359] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.894363  9254 raft_consensus.cc:359] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.895439  9253 raft_consensus.cc:385] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.895613  9254 raft_consensus.cc:385] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.895831  9253 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.896044  9254 raft_consensus.cc:740] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.897364  9253 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.897364  9254 consensus_queue.cc:260] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.898545  9252 raft_consensus.cc:359] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.899376  9252 raft_consensus.cc:385] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.899700  9252 raft_consensus.cc:740] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.900655  9252 consensus_queue.cc:260] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.906769  9160 heartbeater.cc:499] Master 127.8.183.126:46707 was elected leader, sending a full tablet report...
I20260709 13:15:32.907318  9254 ts_tablet_manager.cc:1434] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.006s
I20260709 13:15:32.907673  9253 ts_tablet_manager.cc:1434] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.010s
I20260709 13:15:32.908797  9254 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:32.908795  9253 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:32.917315  9253 tablet_bootstrap.cc:654] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.917317  9254 tablet_bootstrap.cc:654] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.920849  9234 heartbeater.cc:499] Master 127.8.183.126:46707 was elected leader, sending a full tablet report...
I20260709 13:15:32.922103  9252 ts_tablet_manager.cc:1434] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.006s
I20260709 13:15:32.923352  9252 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:32.930615  9252 tablet_bootstrap.cc:654] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.940847  9253 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:32.943353  9253 ts_tablet_manager.cc:1403] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.007s
I20260709 13:15:32.945797  9254 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:32.946782  9254 ts_tablet_manager.cc:1403] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.008s
I20260709 13:15:32.946666  9253 raft_consensus.cc:359] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.951687  9253 raft_consensus.cc:385] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.952214  9253 raft_consensus.cc:740] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.953145  9253 consensus_queue.cc:260] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.952986  9254 raft_consensus.cc:359] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.959501  9253 ts_tablet_manager.cc:1434] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:32.960729  9253 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:32.955785  9254 raft_consensus.cc:385] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.965237  9254 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.966835  9254 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
W20260709 13:15:32.973192  9235 tablet.cc:2406] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:15:32.976898  9254 ts_tablet_manager.cc:1434] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 13:15:32.978622  9254 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:32.975744  9253 tablet_bootstrap.cc:654] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.983546  9252 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:32.984167  9252 ts_tablet_manager.cc:1403] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.061s	user 0.030s	sys 0.021s
I20260709 13:15:32.987587  9252 raft_consensus.cc:359] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.988533  9252 raft_consensus.cc:385] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:32.988894  9252 raft_consensus.cc:740] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:32.989871  9252 consensus_queue.cc:260] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:32.991688  9254 tablet_bootstrap.cc:654] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:32.992378  9252 ts_tablet_manager.cc:1434] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.006s
I20260709 13:15:32.993644  9252 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:32.995427  9259 raft_consensus.cc:493] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.995944  9259 raft_consensus.cc:515] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.003888  9259 leader_election.cc:290] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.006953  9252 tablet_bootstrap.cc:654] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.022827  9253 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.023432  9253 ts_tablet_manager.cc:1403] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.007s
I20260709 13:15:33.034088  9254 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.034848  9254 ts_tablet_manager.cc:1403] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.057s	user 0.029s	sys 0.014s
I20260709 13:15:33.038957  9254 raft_consensus.cc:359] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.040016  9254 raft_consensus.cc:385] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.040328  9254 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.040985  9254 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.043407  9254 ts_tablet_manager.cc:1434] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:33.044730  9254 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.042328  9252 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.046134  9252 ts_tablet_manager.cc:1403] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.012s
I20260709 13:15:33.051872  9254 tablet_bootstrap.cc:654] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.026785  9253 raft_consensus.cc:359] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.053398  9253 raft_consensus.cc:385] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.053782  9253 raft_consensus.cc:740] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.054702  9253 consensus_queue.cc:260] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.060645  9252 raft_consensus.cc:359] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.061487  9252 raft_consensus.cc:385] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.061795  9252 raft_consensus.cc:740] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.062659  9252 consensus_queue.cc:260] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.064707  9254 tablet_bootstrap.cc:492] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.065269  9254 ts_tablet_manager.cc:1403] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:15:33.065868  9252 ts_tablet_manager.cc:1434] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 13:15:33.066979  9252 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:33.069808  9254 raft_consensus.cc:359] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.070508  9254 raft_consensus.cc:385] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.070797  9254 raft_consensus.cc:740] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.071689  9254 consensus_queue.cc:260] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.074138  9254 ts_tablet_manager.cc:1434] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:15:33.074882  9252 tablet_bootstrap.cc:654] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.076715  9253 ts_tablet_manager.cc:1434] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:15:33.077951  9253 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.089485  9254 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.090969  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.091837  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.092782  9135 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
W20260709 13:15:33.094322  9026 leader_election.cc:343] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.095793  9024 leader_election.cc:304] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:33.097231  9259 raft_consensus.cc:2804] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.097672  9259 raft_consensus.cc:493] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.098079  9259 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.100426  9254 tablet_bootstrap.cc:654] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.107983  9253 tablet_bootstrap.cc:654] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.110288  9252 tablet_bootstrap.cc:492] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.111006  9252 ts_tablet_manager.cc:1403] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.016s
I20260709 13:15:33.114379  9252 raft_consensus.cc:359] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.115394  9252 raft_consensus.cc:385] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.115780  9252 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.116843  9252 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.121088  9252 ts_tablet_manager.cc:1434] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:15:33.122310  9252 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:33.122534  9254 tablet_bootstrap.cc:492] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.123175  9254 ts_tablet_manager.cc:1403] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.012s
I20260709 13:15:33.127075  9254 raft_consensus.cc:359] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.116788  9259 raft_consensus.cc:515] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.130146  9252 tablet_bootstrap.cc:654] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.131995  9259 leader_election.cc:290] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.132652  9254 raft_consensus.cc:385] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.132877  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.133065  9254 raft_consensus.cc:740] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.133375  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:33.133587  9135 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.134176  9209 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.134011  9254 consensus_queue.cc:260] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.137110  9254 ts_tablet_manager.cc:1434] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 13:15:33.138440  9254 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.144451  9252 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.145210  9252 ts_tablet_manager.cc:1403] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.013s
I20260709 13:15:33.146266  9254 tablet_bootstrap.cc:654] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.148140  9253 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.148818  9253 ts_tablet_manager.cc:1403] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.000s
I20260709 13:15:33.149734  9252 raft_consensus.cc:359] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.150655  9252 raft_consensus.cc:385] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.151058  9252 raft_consensus.cc:740] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.152006  9252 consensus_queue.cc:260] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.154812  9252 ts_tablet_manager.cc:1434] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:33.155625  9209 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.156253  9252 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:33.157200  9026 leader_election.cc:304] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:33.158208  9259 raft_consensus.cc:2804] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.164222  9252 tablet_bootstrap.cc:654] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.142520  9135 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.172493  9253 raft_consensus.cc:359] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.173656  9253 raft_consensus.cc:385] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.174037  9253 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.175025  9253 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.175740  9254 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.177105  9254 ts_tablet_manager.cc:1403] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.015s
I20260709 13:15:33.178746  9259 raft_consensus.cc:697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:33.182480  9252 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.183900  9252 ts_tablet_manager.cc:1403] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.005s
I20260709 13:15:33.182282  9254 raft_consensus.cc:359] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.185190  9254 raft_consensus.cc:385] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.185590  9254 raft_consensus.cc:740] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.192034  9253 ts_tablet_manager.cc:1434] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:15:33.187701  9259 consensus_queue.cc:237] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.198007  9253 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.186657  9254 consensus_queue.cc:260] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.195199  9277 raft_consensus.cc:493] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.200764  9277 raft_consensus.cc:515] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.196581  9252 raft_consensus.cc:359] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.207916  9277 leader_election.cc:290] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.209090  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.211956  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.213070  9135 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.214789  9024 leader_election.cc:304] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:33.210117  9260 raft_consensus.cc:493] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.215988  9260 raft_consensus.cc:515] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.209707  9252 raft_consensus.cc:385] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.218164  9209 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.219473  9259 raft_consensus.cc:2804] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.218811  9252 raft_consensus.cc:740] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.229017  9260 leader_election.cc:290] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.229566  9259 raft_consensus.cc:493] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.236516  9259 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.223520  9279 raft_consensus.cc:493] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.227792  8956 catalog_manager.cc:5697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:33.235502  9260 raft_consensus.cc:493] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.251135  9260 raft_consensus.cc:515] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.253206  9252 consensus_queue.cc:260] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.250787  9279 raft_consensus.cc:515] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.261543  9279 leader_election.cc:290] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.262486  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.261343  9252 ts_tablet_manager.cc:1434] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.003s
I20260709 13:15:33.264173  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.264694  9252 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
W20260709 13:15:33.266045  9026 leader_election.cc:343] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.266207  9260 leader_election.cc:290] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.267241  9260 raft_consensus.cc:493] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.267375  9024 leader_election.cc:343] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.267853  9024 leader_election.cc:304] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44; no voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:33.267866  9260 raft_consensus.cc:515] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.269232  9279 raft_consensus.cc:2749] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:33.270380  9260 leader_election.cc:290] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.273334  9254 ts_tablet_manager.cc:1434] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.096s	user 0.018s	sys 0.022s
I20260709 13:15:33.276791  9254 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.279645  9253 tablet_bootstrap.cc:654] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.299391  9259 raft_consensus.cc:515] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.300392  9252 tablet_bootstrap.cc:654] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.303675  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.304625  9135 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.315485  9253 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.318102  9135 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.320140  9024 leader_election.cc:304] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:33.320557  9252 tablet_bootstrap.cc:492] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.321084  9252 ts_tablet_manager.cc:1403] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.006s
I20260709 13:15:33.321348  9281 raft_consensus.cc:2804] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.322014  9281 raft_consensus.cc:697] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:33.328184  9253 ts_tablet_manager.cc:1403] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.130s	user 0.021s	sys 0.004s
I20260709 13:15:33.333321  9259 leader_election.cc:290] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.333590  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:33.334331  9209 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.341744  9209 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.328527  9281 consensus_queue.cc:237] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.346966  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.347555  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.348598  9252 raft_consensus.cc:359] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.350551  9259 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.352407  9135 raft_consensus.cc:2393] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e06ad0761567434f9f3b1dcb2a6c592d in current term 1: Already voted for candidate 7616ac7f73484817ab55d8589988ad44 in this term.
I20260709 13:15:33.352581  9259 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.353662  9254 tablet_bootstrap.cc:654] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.350421  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.352104  9252 raft_consensus.cc:385] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.357008  9252 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.360498  9284 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.360659  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
W20260709 13:15:33.359691  9173 leader_election.cc:343] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.362140  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:33.362207  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:33.361624  9284 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.375176  9259 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.376379  9175 leader_election.cc:304] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e06ad0761567434f9f3b1dcb2a6c592d; no voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.373246  9252 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
W20260709 13:15:33.378468  9175 leader_election.cc:343] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.380671  9282 raft_consensus.cc:2749] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:33.380892  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.372135  9058 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 0.
I20260709 13:15:33.373697  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.386397  9135 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.387156  9284 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.385910  9209 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.387965  9024 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:33.389781  9252 ts_tablet_manager.cc:1434] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.068s	user 0.003s	sys 0.017s
I20260709 13:15:33.390362  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:33.390957  9175 leader_election.cc:304] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.390050  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.392026  9252 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
I20260709 13:15:33.392194  9135 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 0.
I20260709 13:15:33.394495  9260 raft_consensus.cc:2804] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.394973  9260 raft_consensus.cc:493] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.390087  9281 raft_consensus.cc:2804] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.395818  9260 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.396626  9059 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 0.
I20260709 13:15:33.394930  9173 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:33.397799  9281 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.399863  9259 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.401156  9284 raft_consensus.cc:2804] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.401937  9281 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.402623  9284 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.401948  9259 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.404963  9284 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.409252  9254 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.407831  9253 raft_consensus.cc:359] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.411983  9254 ts_tablet_manager.cc:1403] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.011s
I20260709 13:15:33.412626  9253 raft_consensus.cc:385] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.414022  9253 raft_consensus.cc:740] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.417630  9259 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.417101  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.415618  9253 consensus_queue.cc:260] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.419538  9135 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.420840  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.420524  9254 raft_consensus.cc:359] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.423786  9209 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.424186  9024 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:33.427897  9253 ts_tablet_manager.cc:1434] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.096s	user 0.006s	sys 0.006s
I20260709 13:15:33.429435  9282 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.430263  9259 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.430899  9259 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.425967  9254 raft_consensus.cc:385] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.431331  9253 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.430498  9282 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.432283  9259 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.430974  9260 raft_consensus.cc:515] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.432219  9254 raft_consensus.cc:740] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.437009  9260 leader_election.cc:290] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.438664  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:33.435729  9254 consensus_queue.cc:260] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.440610  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.444976  9279 raft_consensus.cc:493] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.440490  9058 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.445688  9279 raft_consensus.cc:515] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.435936  9284 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.440446  9282 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527)
I20260709 13:15:33.447036  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:33.464880  9254 ts_tablet_manager.cc:1434] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
W20260709 13:15:33.447692  9173 leader_election.cc:343] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.473747  9058 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.474967  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
W20260709 13:15:33.478848  9175 leader_election.cc:343] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.479601  9254 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.481889  9175 leader_election.cc:304] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.482820  9282 raft_consensus.cc:2804] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.483564  9282 raft_consensus.cc:697] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
W20260709 13:15:33.483589  9173 leader_election.cc:343] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.484422  9252 tablet_bootstrap.cc:654] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.479686  9259 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.484699  9284 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.484915  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:33.484951  9282 consensus_queue.cc:237] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.485997  9173 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e06ad0761567434f9f3b1dcb2a6c592d; no voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.490722  9284 raft_consensus.cc:2749] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:33.494750  9259 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.496410  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.501726  9133 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.499737  9135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.504349  9135 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.507650  9281 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.510327  9252 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.510962  9252 ts_tablet_manager.cc:1403] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.012s
I20260709 13:15:33.511155  9254 tablet_bootstrap.cc:654] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.513870  9134 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 0.
I20260709 13:15:33.515599  9253 tablet_bootstrap.cc:654] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.516418  9173 leader_election.cc:304] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:33.518249  9133 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.519891  9024 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:33.520922  9259 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.521512  9259 raft_consensus.cc:697] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
W20260709 13:15:33.521977  9004 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:33.522467  9259 consensus_queue.cc:237] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.523164  9282 raft_consensus.cc:2804] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.523598  9282 raft_consensus.cc:493] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.523967  9282 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.524444  9135 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.525936  9173 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:33.526400  9281 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.526829  9284 raft_consensus.cc:2804] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.527534  9284 raft_consensus.cc:697] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:33.527446  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.528544  9133 raft_consensus.cc:2393] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7616ac7f73484817ab55d8589988ad44 in current term 1: Already voted for candidate e06ad0761567434f9f3b1dcb2a6c592d in this term.
I20260709 13:15:33.528622  9284 consensus_queue.cc:237] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.529882  9252 raft_consensus.cc:359] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.530337  9279 leader_election.cc:290] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.530608  9252 raft_consensus.cc:385] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.530965  9252 raft_consensus.cc:740] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.531915  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.534358  9252 consensus_queue.cc:260] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
W20260709 13:15:33.535738  9024 leader_election.cc:343] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.536921  9252 ts_tablet_manager.cc:1434] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:15:33.537945  9252 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: Bootstrap starting.
W20260709 13:15:33.534427  9026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:42907, user_credentials={real_user=slave}} blocked reactor thread for 105137us
I20260709 13:15:33.558264  9252 tablet_bootstrap.cc:654] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.568956  9252 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: No bootstrap required, opened a new log
I20260709 13:15:33.551307  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:33.569774  9252 ts_tablet_manager.cc:1403] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.002s
I20260709 13:15:33.568563  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:33.573055  9254 tablet_bootstrap.cc:492] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.573496  9254 ts_tablet_manager.cc:1403] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.001s
I20260709 13:15:33.571874  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:33.571470  9209 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.580083  9026 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44; no voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:33.581979  9259 raft_consensus.cc:2749] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:33.581763  9254 raft_consensus.cc:359] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.582790  9254 raft_consensus.cc:385] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.583314  9254 raft_consensus.cc:740] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.582779  9252 raft_consensus.cc:359] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.584201  9252 raft_consensus.cc:385] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.584069  9254 consensus_queue.cc:260] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.584483  9252 raft_consensus.cc:740] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.585404  9252 consensus_queue.cc:260] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.587400  9254 ts_tablet_manager.cc:1434] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.003s
I20260709 13:15:33.588089  9252 ts_tablet_manager.cc:1434] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:15:33.587934  9207 raft_consensus.cc:2468] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:33.585172  9282 raft_consensus.cc:515] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.590767  9026 leader_election.cc:304] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.592531  9259 raft_consensus.cc:2804] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.592733  9282 leader_election.cc:290] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.593035  9259 raft_consensus.cc:493] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.593096  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:33.593549  9259 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.593889  9059 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.595770  9254 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: Bootstrap starting.
I20260709 13:15:33.596026  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.596732  9133 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.600965  8957 catalog_manager.cc:5697] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:33.604295  9254 tablet_bootstrap.cc:654] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.603662  9259 raft_consensus.cc:515] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.606818  9209 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.608496  8957 catalog_manager.cc:5697] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:33.612543  9059 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.612438  9279 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:33.616168  9259 leader_election.cc:290] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:33.617069  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.617802  9175 leader_election.cc:304] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:33.618577  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:33.618863  9282 raft_consensus.cc:2804] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.619884  9282 raft_consensus.cc:697] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:33.620311  9133 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.621536  9253 tablet_bootstrap.cc:492] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.621234  9282 consensus_queue.cc:237] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.622452  9253 ts_tablet_manager.cc:1403] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.023s
W20260709 13:15:33.625711  9024 leader_election.cc:343] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.619256  9207 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.640525  8955 catalog_manager.cc:5697] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 0 to 1, leader changed from <none> to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:33.642714  8955 catalog_manager.cc:5697] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 0 to 1, leader changed from <none> to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:33.644671  9253 raft_consensus.cc:359] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.648639  9253 raft_consensus.cc:385] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.649541  9253 raft_consensus.cc:740] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.651155  9253 consensus_queue.cc:260] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.655952  9253 ts_tablet_manager.cc:1434] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 13:15:33.657213  9253 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.675647  9253 tablet_bootstrap.cc:654] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.687294  9282 raft_consensus.cc:493] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.688133  9282 raft_consensus.cc:515] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.690649  9207 raft_consensus.cc:2468] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:33.692276  9026 leader_election.cc:304] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:33.693383  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:33.693735  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:33.694224  9279 raft_consensus.cc:2804] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.694444  9059 raft_consensus.cc:2468] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 0.
I20260709 13:15:33.694801  9279 raft_consensus.cc:697] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:33.695756  9279 consensus_queue.cc:237] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.698342  9175 leader_election.cc:304] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:33.699980  9284 raft_consensus.cc:2804] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:33.700387  9284 raft_consensus.cc:493] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.700742  9284 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:33.702351  9173 leader_election.cc:343] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:33.701786  9282 leader_election.cc:290] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.705454  9254 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: No bootstrap required, opened a new log
I20260709 13:15:33.706015  9254 ts_tablet_manager.cc:1403] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: Time spent bootstrapping tablet: real 0.110s	user 0.013s	sys 0.028s
I20260709 13:15:33.719846  9254 raft_consensus.cc:359] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.721572  9254 raft_consensus.cc:385] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.722692  9254 raft_consensus.cc:740] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.709599  8957 catalog_manager.cc:5697] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 0 to 1, leader changed from <none> to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:33.724568  9254 consensus_queue.cc:260] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.731012  9253 tablet_bootstrap.cc:492] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.731634  9253 ts_tablet_manager.cc:1403] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.075s	user 0.039s	sys 0.024s
I20260709 13:15:33.732717  9254 ts_tablet_manager.cc:1434] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 13:15:33.733172  9284 raft_consensus.cc:515] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.734810  9253 raft_consensus.cc:359] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.735566  9253 raft_consensus.cc:385] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.735863  9253 raft_consensus.cc:740] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.736645  9253 consensus_queue.cc:260] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.738571  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:33.739251  9059 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.739312  9281 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:33.747247  9059 raft_consensus.cc:2468] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.749434  9253 ts_tablet_manager.cc:1434] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:15:33.749814  9175 leader_election.cc:304] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:33.750144  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:33.750732  9282 raft_consensus.cc:2804] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:33.751508  9282 raft_consensus.cc:697] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:33.750844  9284 leader_election.cc:290] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 1 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:33.753017  9282 consensus_queue.cc:237] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.750814  9133 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.763654  9253 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.767061  8956 catalog_manager.cc:5697] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:33.770201  9133 raft_consensus.cc:2468] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 1.
I20260709 13:15:33.773141  9253 tablet_bootstrap.cc:654] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.786404  9253 tablet_bootstrap.cc:492] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.786994  9253 ts_tablet_manager.cc:1403] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.005s
I20260709 13:15:33.794241  9253 raft_consensus.cc:359] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.795236  9253 raft_consensus.cc:385] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.795543  9253 raft_consensus.cc:740] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.796414  9253 consensus_queue.cc:260] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:33.800670  9253 ts_tablet_manager.cc:1434] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:15:33.801863  9253 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: Bootstrap starting.
I20260709 13:15:33.807459  8957 catalog_manager.cc:5697] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 0 to 1, leader changed from <none> to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:33.809618  9253 tablet_bootstrap.cc:654] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:33.823657  9279 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:33.829124  9253 tablet_bootstrap.cc:492] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: No bootstrap required, opened a new log
I20260709 13:15:33.829799  9253 ts_tablet_manager.cc:1403] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.012s
I20260709 13:15:33.833465  9253 raft_consensus.cc:359] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.834337  9253 raft_consensus.cc:385] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:33.834625  9253 raft_consensus.cc:740] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Initialized, Role: FOLLOWER
I20260709 13:15:33.835429  9253 consensus_queue.cc:260] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:33.838128  9253 ts_tablet_manager.cc:1434] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:33.855576  9259 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:33.953783  9133 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:33.961452  9282 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:33.969679  9279 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:33.976251  9284 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:33.994345  9279 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.036608  9284 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.052793  9284 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.065284  9284 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.080972  9317 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.097033  9258 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.097931  9258 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:34.101236  9258 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907), 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527)
I20260709 13:15:34.126992  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:34.127935  9209 raft_consensus.cc:2468] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 0.
I20260709 13:15:34.129150  9277 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.129814  9277 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:34.132983  9277 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:34.134343  9209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:34.136332  9209 raft_consensus.cc:2468] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 0.
I20260709 13:15:34.135444  9101 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:34.138073  9324 raft_consensus.cc:2804] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:34.138574  9324 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.138657  9282 consensus_queue.cc:1048] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.139518  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" is_pre_election: true
I20260709 13:15:34.138089  9026 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:34.141136  9277 raft_consensus.cc:2804] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:34.141544  9277 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.141891  9277 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:34.141999  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:34.138983  9324 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:34.151373  9277 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:34.152789  9058 raft_consensus.cc:2393] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b1d8fa77f77045a9a9154a72ebb5e785 in current term 1: Already voted for candidate 7616ac7f73484817ab55d8589988ad44 in this term.
I20260709 13:15:34.154402  9279 consensus_queue.cc:1048] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.158767  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:34.160904  9277 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Requested vote from peers e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:34.162339  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.162999  9208 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:34.172946  9260 consensus_queue.cc:1048] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.177986  9324 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:34.179378  9134 raft_consensus.cc:2393] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7616ac7f73484817ab55d8589988ad44 in current term 1: Already voted for candidate b1d8fa77f77045a9a9154a72ebb5e785 in this term.
I20260709 13:15:34.183223  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:34.184247  9059 raft_consensus.cc:2393] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b1d8fa77f77045a9a9154a72ebb5e785 in current term 1: Already voted for candidate 7616ac7f73484817ab55d8589988ad44 in this term.
I20260709 13:15:34.185943  9324 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 election: Requested vote from peers e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907), 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527)
I20260709 13:15:34.191579  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.194092  9101 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785; no voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:34.197182  9258 raft_consensus.cc:2749] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:34.197932  9258 raft_consensus.cc:493] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.196542  9208 raft_consensus.cc:2468] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 1.
I20260709 13:15:34.198802  9258 raft_consensus.cc:515] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.201905  9258 leader_election.cc:290] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:34.202340  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7616ac7f73484817ab55d8589988ad44" is_pre_election: true
I20260709 13:15:34.204922  9134 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:34.208329  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" is_pre_election: true
I20260709 13:15:34.208971  9026 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:34.210047  9259 raft_consensus.cc:2804] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:34.210762  9259 raft_consensus.cc:697] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:34.211710  9259 consensus_queue.cc:237] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:34.213907  9101 leader_election.cc:304] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785; no voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:34.229349  9258 raft_consensus.cc:2749] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:34.236963  9259 consensus_queue.cc:1048] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.240566  8956 catalog_manager.cc:5697] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:34.636246  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:34.636953  9207 raft_consensus.cc:606] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Received request to transfer leadership to TS 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:34.651851  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:34.652460  9207 raft_consensus.cc:606] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:34.658629  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "290b9eb922674f538e458e07dcae691f"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:34.659287  9207 raft_consensus.cc:606] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:34.680038  9059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:43358
I20260709 13:15:34.680781  9059 raft_consensus.cc:606] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:34.688243  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:34.688947  9207 raft_consensus.cc:606] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:34.696324  9059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:43358
I20260709 13:15:34.697129  9059 raft_consensus.cc:606] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Received request to transfer leadership to TS e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:34.735021  9281 consensus_queue.cc:1048] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:34.737461  9281 consensus_queue.cc:1048] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:34.782889  9260 raft_consensus.cc:993] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.789896  9317 raft_consensus.cc:1081] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.792531  9134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55966
I20260709 13:15:34.793186  9134 raft_consensus.cc:493] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.793628  9134 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.801815  9134 raft_consensus.cc:515] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.803997  9134 leader_election.cc:290] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:34.805228  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:34.806109  9059 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.807014  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f577a8bd1f0e42c89c44580265ee5127" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.807746  9207 raft_consensus.cc:3055] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.808140  9207 raft_consensus.cc:740] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:34.809100  9207 consensus_queue.cc:260] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.810729  9207 raft_consensus.cc:3060] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.812752  9059 raft_consensus.cc:2468] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.814208  9101 leader_election.cc:304] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:34.815097  9258 raft_consensus.cc:2804] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:34.819096  9207 raft_consensus.cc:2468] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.821045  9281 raft_consensus.cc:993] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.821569  9259 raft_consensus.cc:1081] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.821811  9258 raft_consensus.cc:697] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:34.822784  9258 consensus_queue.cc:237] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.823753  9134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55958
I20260709 13:15:34.824489  9134 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.824941  9134 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.830861  8954 catalog_manager.cc:5697] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:34.832329  9134 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.834761  9134 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:34.836410  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:34.836593  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.837204  9059 raft_consensus.cc:3055] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.837316  9207 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.837718  9059 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:34.838735  9059 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.840423  9059 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.844925  9207 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.846369  9101 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:34.847692  9258 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:34.848450  9258 raft_consensus.cc:697] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:34.849763  9258 consensus_queue.cc:237] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.856480  9059 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.863719  8954 catalog_manager.cc:5697] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:34.880885  9317 raft_consensus.cc:993] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.881516  9260 raft_consensus.cc:1081] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:34.883457  9134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "290b9eb922674f538e458e07dcae691f"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55966
I20260709 13:15:34.884192  9134 raft_consensus.cc:493] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.884658  9134 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.890925  9134 raft_consensus.cc:515] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.893075  9134 leader_election.cc:290] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:34.894724  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.895429  9207 raft_consensus.cc:3055] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.895889  9207 raft_consensus.cc:740] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:34.894853  9059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:34.896896  9059 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.896816  9207 consensus_queue.cc:260] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.898073  9207 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.903252  9317 raft_consensus.cc:993] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:34.903808  9282 raft_consensus.cc:1081] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Signalling peer 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:34.905601  9059 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.906154  9058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:43344
I20260709 13:15:34.906952  9058 raft_consensus.cc:493] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.907547  9058 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.907531  9101 leader_election.cc:304] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:34.909093  9258 raft_consensus.cc:2804] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:34.909901  9258 raft_consensus.cc:697] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:34.911226  9258 consensus_queue.cc:237] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.911926  9207 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:34.918038  9058 raft_consensus.cc:515] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.920846  9058 leader_election.cc:290] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 2 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:34.921571  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:34.921326  8954 catalog_manager.cc:5697] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:34.922536  9207 raft_consensus.cc:3055] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:34.922998  9207 raft_consensus.cc:740] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:34.923962  9207 consensus_queue.cc:260] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.921373  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:34.925225  9134 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.925370  9207 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:34.934144  9207 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 2.
I20260709 13:15:34.935818  9026 leader_election.cc:304] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:34.936933  9259 raft_consensus.cc:2804] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:34.937556  9259 raft_consensus.cc:697] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:34.938629  9134 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 2.
I20260709 13:15:34.938490  9259 consensus_queue.cc:237] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:34.947351  8954 catalog_manager.cc:5697] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 1 to 2, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.118583  9282 raft_consensus.cc:993] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:35.119077  9317 raft_consensus.cc:1081] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:35.120927  9134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55966
I20260709 13:15:35.121486  9134 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.121798  9134 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.126955  9134 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.128774  9134 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:35.130286  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:35.130553  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:35.131013  9058 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.131321  9207 raft_consensus.cc:3055] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:35.131713  9207 raft_consensus.cc:740] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:35.132473  9207 consensus_queue.cc:260] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.133495  9207 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.137791  9058 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:35.139170  9101 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:35.139930  9207 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:35.140059  9258 raft_consensus.cc:2804] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:35.140697  9258 raft_consensus.cc:697] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:35.141562  9258 consensus_queue.cc:237] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.149641  8954 catalog_manager.cc:5697] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:35.220633  9281 raft_consensus.cc:993] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: : Instructing follower e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:35.221179  9281 raft_consensus.cc:1081] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Signalling peer e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:35.222707  9207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:48950
I20260709 13:15:35.222882  9058 raft_consensus.cc:1275] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.223246  9207 raft_consensus.cc:493] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.223635  9207 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.224408  9258 consensus_queue.cc:1048] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.229761  9207 raft_consensus.cc:515] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.233795  9207 leader_election.cc:290] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:35.233897  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:35.235263  9208 raft_consensus.cc:1275] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.235634  9058 raft_consensus.cc:3055] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:35.236110  9058 raft_consensus.cc:740] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:35.237228  9341 consensus_queue.cc:1048] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.237460  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0ed33b99f2b44b5a2fd4925b3afd804" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:35.238207  9134 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.238143  9058 consensus_queue.cc:260] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.239787  9058 raft_consensus.cc:3060] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:35.246502  9134 raft_consensus.cc:2468] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 2.
I20260709 13:15:35.248347  9173 leader_election.cc:304] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:35.249439  9317 raft_consensus.cc:2804] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:35.250008  9317 raft_consensus.cc:697] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:35.250958  9317 consensus_queue.cc:237] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:35.255925  9058 raft_consensus.cc:2468] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 2.
I20260709 13:15:35.266431  8954 catalog_manager.cc:5697] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 1 to 2, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 2 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:35.344053  9208 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.345476  9341 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.356736  9058 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.359179  9341 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.373432  9058 raft_consensus.cc:1275] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.375231  9258 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.388969  9208 raft_consensus.cc:1275] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.390713  9351 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.496657  9134 raft_consensus.cc:1275] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.498442  9259 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.509235  9208 raft_consensus.cc:1275] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.512252  9356 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.597061  9208 raft_consensus.cc:1275] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.599366  9351 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:35.612682  9058 raft_consensus.cc:1275] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.614598  9351 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.822944  9058 raft_consensus.cc:1275] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.824782  9317 consensus_queue.cc:1048] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:35.833909  9134 raft_consensus.cc:1275] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:35.836303  9361 consensus_queue.cc:1048] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:37.737107  8925 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:15:37.738229  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:37.764690  9134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:37.765329  9134 raft_consensus.cc:606] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Received request to transfer leadership to TS e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:37.780026  9058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:43366
I20260709 13:15:37.780794  9058 raft_consensus.cc:606] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Received request to transfer leadership to TS e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:37.782186  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:37.782743  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:37.860091  9351 raft_consensus.cc:993] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:37.860860  9371 raft_consensus.cc:1081] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Signalling peer e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:37.862739  9207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:15:37.863410  9207 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:37.863767  9207 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.869724  9207 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:37.871860  9207 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:37.873217  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:37.873543  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:37.873852  9058 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.874178  9134 raft_consensus.cc:3055] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:37.874526  9134 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:37.875378  9134 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:37.876789  9134 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:37.881083  9058 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:37.882352  9134 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:37.882731  9175 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:37.883653  9366 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:37.884809  9366 raft_consensus.cc:697] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:37.885667  9366 consensus_queue.cc:237] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:37.893227  8954 catalog_manager.cc:5697] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 2 to 3, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 3 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.219620  9367 raft_consensus.cc:993] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: : Instructing follower e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:38.220103  9379 raft_consensus.cc:1081] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Signalling peer e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:38.222137  9207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:48950
I20260709 13:15:38.222657  9207 raft_consensus.cc:493] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.222954  9207 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.228667  9207 raft_consensus.cc:515] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.230273  9207 leader_election.cc:290] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:38.231632  9134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:38.231632  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:38.232270  9134 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.232503  9058 raft_consensus.cc:3055] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:38.232895  9058 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:38.233664  9058 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.234670  9058 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:38.237370  9134 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 2.
I20260709 13:15:38.238595  9173 leader_election.cc:304] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:38.239480  9366 raft_consensus.cc:2804] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:38.239981  9366 raft_consensus.cc:697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:38.240860  9058 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 2.
I20260709 13:15:38.240859  9366 consensus_queue.cc:237] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.249389  8954 catalog_manager.cc:5697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 1 to 2, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 2 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.411585  9058 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.413077  9366 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:38.424827  9134 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:38.426751  9373 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:38.744484  9058 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:38.746179  9373 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:38.758394  9134 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:38.760562  9387 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:38.783730  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:38.795253  9058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e54e833462b44129b948504f0eb55714"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:43358
I20260709 13:15:38.795936  9058 raft_consensus.cc:606] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:38.801982  9058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:43358
I20260709 13:15:38.802615  9058 raft_consensus.cc:606] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:38.808650  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:38.809187  9207 raft_consensus.cc:606] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:38.815317  9058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:43358
I20260709 13:15:38.815887  9058 raft_consensus.cc:606] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:38.827729  9207 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
mode: GRACEFUL
new_leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:48976
I20260709 13:15:38.828392  9207 raft_consensus.cc:606] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Received request to transfer leadership to TS b1d8fa77f77045a9a9154a72ebb5e785
I20260709 13:15:38.954392  9366 raft_consensus.cc:993] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:38.954919  9366 raft_consensus.cc:1081] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:38.956871  9134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55966
I20260709 13:15:38.957206  9389 raft_consensus.cc:993] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:38.957486  9134 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.957865  9379 raft_consensus.cc:1081] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:38.957927  9134 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.960196  9133 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55958
I20260709 13:15:38.960867  9133 raft_consensus.cc:493] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.961264  9133 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.965875  9134 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.968230  9134 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 4 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:38.969018  9133 raft_consensus.cc:515] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.969569  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:38.969362  9058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:38.970264  9058 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.970254  9207 raft_consensus.cc:3055] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:38.971505  9133 leader_election.cc:290] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 3 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:38.971586  9207 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:38.972815  9207 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.973469  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:38.974110  9208 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.974246  9207 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:38.974421  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fb34a53e97e4ad1b3cd31bc21526114" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:38.975003  9060 raft_consensus.cc:3055] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:38.975361  9060 raft_consensus.cc:740] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:38.976521  9060 consensus_queue.cc:260] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.978544  9060 raft_consensus.cc:3060] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:38.984057  9208 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 3.
I20260709 13:15:38.984722  9058 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 4.
I20260709 13:15:38.985564  9207 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 4.
I20260709 13:15:38.985882  9101 leader_election.cc:304] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:38.986896  9371 raft_consensus.cc:2804] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:38.987156  9101 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, b1d8fa77f77045a9a9154a72ebb5e785; no voters: 
I20260709 13:15:38.988322  9371 raft_consensus.cc:697] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:38.988863  9390 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:38.989351  9371 consensus_queue.cc:237] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.989383  9390 raft_consensus.cc:697] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:38.991542  9060 raft_consensus.cc:2468] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 3.
I20260709 13:15:38.991463  9390 consensus_queue.cc:237] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:38.998775  8954 catalog_manager.cc:5697] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 2 to 3, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 3 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.017305  8954 catalog_manager.cc:5697] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 3 to 4, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 4 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.163590  9389 raft_consensus.cc:993] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.164139  9379 raft_consensus.cc:1081] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.165776  9133 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55958
I20260709 13:15:39.166287  9133 raft_consensus.cc:493] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:39.166572  9133 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.173301  9133 raft_consensus.cc:515] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.174985  9133 leader_election.cc:290] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:39.176179  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:39.176462  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:39.176829  9060 raft_consensus.cc:3055] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:39.177201  9060 raft_consensus.cc:740] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:39.177210  9207 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.178138  9060 consensus_queue.cc:260] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.179243  9060 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.182508  9207 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:39.183986  9101 leader_election.cc:304] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:39.184706  9060 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:39.184801  9390 raft_consensus.cc:2804] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:39.185406  9390 raft_consensus.cc:697] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:39.186378  9390 consensus_queue.cc:237] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.194272  8954 catalog_manager.cc:5697] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.207253  9389 raft_consensus.cc:993] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.207847  9379 raft_consensus.cc:1081] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.209728  9133 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e54e833462b44129b948504f0eb55714"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55958
I20260709 13:15:39.210271  9133 raft_consensus.cc:493] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:39.210558  9133 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.216725  9133 raft_consensus.cc:515] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:39.218591  9133 leader_election.cc:290] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Requested vote from peers e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907), 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527)
I20260709 13:15:39.219942  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:39.219843  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e54e833462b44129b948504f0eb55714" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:39.220865  9060 raft_consensus.cc:3055] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:39.221244  9207 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.221307  9060 raft_consensus.cc:740] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:39.222445  9060 consensus_queue.cc:260] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:39.223631  9060 raft_consensus.cc:3060] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:39.226617  9207 raft_consensus.cc:2468] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:39.227912  9101 leader_election.cc:304] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:39.228830  9390 raft_consensus.cc:2804] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:39.229378  9390 raft_consensus.cc:697] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:39.229789  9060 raft_consensus.cc:2468] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 2.
I20260709 13:15:39.230276  9390 consensus_queue.cc:237] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } }
I20260709 13:15:39.239101  8954 catalog_manager.cc:5697] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 1 to 2, leader changed from 7616ac7f73484817ab55d8589988ad44 (127.8.183.65) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.361083  9366 raft_consensus.cc:993] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: : Instructing follower b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.361644  9387 raft_consensus.cc:1081] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Signalling peer b1d8fa77f77045a9a9154a72ebb5e785 to start an election
I20260709 13:15:39.363868  9133 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
 from {username='slave'} at 127.0.0.1:55966
I20260709 13:15:39.364661  9133 raft_consensus.cc:493] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:39.365131  9133 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.371078  9133 raft_consensus.cc:515] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.373278  9133 leader_election.cc:290] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 3 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:39.374341  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:39.374782  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:39.375373  9207 raft_consensus.cc:3055] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:39.375712  9208 raft_consensus.cc:1275] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:39.375874  9060 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.375711  9207 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:39.377179  9207 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.377651  9390 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:39.378286  9207 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:39.384734  9207 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 3.
I20260709 13:15:39.385195  9060 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d8fa77f77045a9a9154a72ebb5e785 in term 3.
I20260709 13:15:39.386116  9101 leader_election.cc:304] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1d8fa77f77045a9a9154a72ebb5e785, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:39.386981  9390 raft_consensus.cc:2804] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:39.387830  9390 raft_consensus.cc:697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Becoming Leader. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:39.388834  9390 consensus_queue.cc:237] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:39.394217  9060 raft_consensus.cc:1275] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:39.395866  9371 consensus_queue.cc:1048] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:39.399382  8954 catalog_manager.cc:5697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 reported cstate change: term changed from 2 to 3, leader changed from e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67) to b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66). New cstate: current_term: 3 leader_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:39.437861  9060 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:39.439723  9371 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:39.451188  9207 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:39.454399  9404 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:39.621001  9060 raft_consensus.cc:1275] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:39.623016  9404 consensus_queue.cc:1048] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:39.623679  9060 raft_consensus.cc:1275] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:39.625244  9404 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:39.634521  9207 raft_consensus.cc:1275] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:39.636162  9390 consensus_queue.cc:1048] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:39.646986  9207 raft_consensus.cc:1275] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:39.651162  9404 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:39.784713  9207 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:39.786178  9404 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:39.796341  9060 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Refusing update from remote peer b1d8fa77f77045a9a9154a72ebb5e785: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:39.798142  9404 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:41.837805  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:41.846311  9133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:41.847285  9133 raft_consensus.cc:606] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Received request to transfer leadership to TS e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:41.851227  9133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "290b9eb922674f538e458e07dcae691f"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:41.851831  9133 raft_consensus.cc:606] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Received request to transfer leadership to TS e06ad0761567434f9f3b1dcb2a6c592d
I20260709 13:15:41.854701  9133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:41.855336  9133 raft_consensus.cc:606] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Received request to transfer leadership to TS 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:41.858258  9133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:41.858939  9133 raft_consensus.cc:606] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 LEADER]: Received request to transfer leadership to TS 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:41.862280  9133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
mode: GRACEFUL
new_leader_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:55978
I20260709 13:15:41.863063  9133 raft_consensus.cc:606] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Received request to transfer leadership to TS 7616ac7f73484817ab55d8589988ad44
I20260709 13:15:41.864668  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:41.865258  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:41.909801  9391 raft_consensus.cc:993] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:41.910634  9414 raft_consensus.cc:1081] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Signalling peer e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:41.912642  9207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "290b9eb922674f538e458e07dcae691f"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:15:41.913329  9207 raft_consensus.cc:493] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.913679  9207 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.919027  9207 raft_consensus.cc:515] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.920979  9207 leader_election.cc:290] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:41.922314  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:41.922240  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "290b9eb922674f538e458e07dcae691f" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:41.923229  9060 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.923319  9133 raft_consensus.cc:3055] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:41.923735  9133 raft_consensus.cc:740] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:41.924686  9133 consensus_queue.cc:260] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.926419  9133 raft_consensus.cc:3060] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:41.929883  9060 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:41.931686  9175 leader_election.cc:304] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:41.933012  9387 raft_consensus.cc:2804] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:41.933955  9133 raft_consensus.cc:2468] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:41.934273  9387 raft_consensus.cc:697] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:41.935585  9387 consensus_queue.cc:237] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.944449  8954 catalog_manager.cc:5697] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 2 to 3, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 3 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.948432  9391 raft_consensus.cc:993] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:41.949059  9391 raft_consensus.cc:1081] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 LEADER]: Signalling peer 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:41.951454  9060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e00185e4919476a922eb432f336b644"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:43346
I20260709 13:15:41.952194  9060 raft_consensus.cc:493] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.952592  9060 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.960968  9060 raft_consensus.cc:515] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.963660  9060 leader_election.cc:290] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 5 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:41.964941  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:41.965199  9207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e00185e4919476a922eb432f336b644" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:41.965798  9133 raft_consensus.cc:3055] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:41.965896  9207 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.966187  9133 raft_consensus.cc:740] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:41.967162  9133 consensus_queue.cc:260] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.968362  9133 raft_consensus.cc:3060] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:41.971971  9207 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 5.
I20260709 13:15:41.973917  9026 leader_election.cc:304] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:41.974332  9133 raft_consensus.cc:2468] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 5.
I20260709 13:15:41.976815  9421 raft_consensus.cc:2804] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:41.978292  9421 raft_consensus.cc:697] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 5 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:41.979964  9421 consensus_queue.cc:237] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:41.988977  8954 catalog_manager.cc:5697] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 4 to 5, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 5 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.993979  9391 raft_consensus.cc:993] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:41.994582  9414 raft_consensus.cc:1081] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Signalling peer 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:41.996493  9060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:43346
I20260709 13:15:41.997246  9060 raft_consensus.cc:493] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.997614  9060 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.003798  9060 raft_consensus.cc:515] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.005932  9060 leader_election.cc:290] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 4 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:42.006546  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:42.007272  9133 raft_consensus.cc:3055] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:42.007397  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76d8fa9ba9c3459cb5760fe37d04f4fa" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:42.007679  9133 raft_consensus.cc:740] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:42.008044  9208 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.008618  9133 consensus_queue.cc:260] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.009752  9133 raft_consensus.cc:3060] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:42.014398  9208 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 4.
I20260709 13:15:42.015697  9133 raft_consensus.cc:2468] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 4.
I20260709 13:15:42.015846  9026 leader_election.cc:304] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:42.017079  9421 raft_consensus.cc:2804] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:42.017632  9421 raft_consensus.cc:697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 4 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:42.018509  9421 consensus_queue.cc:237] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.028048  8954 catalog_manager.cc:5697] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 3 to 4, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 4 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.074996  9391 raft_consensus.cc:993] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:42.075552  9391 raft_consensus.cc:1081] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Signalling peer e06ad0761567434f9f3b1dcb2a6c592d to start an election
I20260709 13:15:42.077306  9208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d"
dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
 from {username='slave'} at 127.0.0.1:48966
I20260709 13:15:42.077864  9208 raft_consensus.cc:493] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:42.078238  9208 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.083412  9208 raft_consensus.cc:515] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.085397  9208 leader_election.cc:290] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Requested vote from peers 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527), b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553)
I20260709 13:15:42.086606  9060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7616ac7f73484817ab55d8589988ad44"
I20260709 13:15:42.086859  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4de5ebc32d4a979c35122f55a6619d" candidate_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:42.087337  9060 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.087522  9133 raft_consensus.cc:3055] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:42.087939  9133 raft_consensus.cc:740] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:42.088639  9133 consensus_queue.cc:260] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.089596  9133 raft_consensus.cc:3060] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.095176  9060 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:42.095817  9133 raft_consensus.cc:2468] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e06ad0761567434f9f3b1dcb2a6c592d in term 3.
I20260709 13:15:42.096812  9175 leader_election.cc:304] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:42.098034  9387 raft_consensus.cc:2804] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:42.098598  9387 raft_consensus.cc:697] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Becoming Leader. State: Replica: e06ad0761567434f9f3b1dcb2a6c592d, State: Running, Role: LEADER
I20260709 13:15:42.099630  9387 consensus_queue.cc:237] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.108176  8954 catalog_manager.cc:5697] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d reported cstate change: term changed from 2 to 3, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67). New cstate: current_term: 3 leader_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:42.158308  9414 raft_consensus.cc:993] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: : Instructing follower 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:42.158779  9391 raft_consensus.cc:1081] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Signalling peer 7616ac7f73484817ab55d8589988ad44 to start an election
I20260709 13:15:42.160609  9060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45"
dest_uuid: "7616ac7f73484817ab55d8589988ad44"
 from {username='slave'} at 127.0.0.1:43346
I20260709 13:15:42.161119  9060 raft_consensus.cc:493] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:42.161417  9060 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.166162  9060 raft_consensus.cc:515] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.167770  9060 leader_election.cc:290] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 3 election: Requested vote from peers b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553), e06ad0761567434f9f3b1dcb2a6c592d (127.8.183.67:42907)
I20260709 13:15:42.169138  9133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b1d8fa77f77045a9a9154a72ebb5e785"
I20260709 13:15:42.169261  9208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ad6407878c3407aa2ca069e71226e45" candidate_uuid: "7616ac7f73484817ab55d8589988ad44" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e06ad0761567434f9f3b1dcb2a6c592d"
I20260709 13:15:42.169822  9133 raft_consensus.cc:3055] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:42.169940  9208 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.170220  9133 raft_consensus.cc:740] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b1d8fa77f77045a9a9154a72ebb5e785, State: Running, Role: LEADER
I20260709 13:15:42.171077  9133 consensus_queue.cc:260] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.172159  9133 raft_consensus.cc:3060] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:42.176929  9208 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 3.
I20260709 13:15:42.177513  9133 raft_consensus.cc:2468] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7616ac7f73484817ab55d8589988ad44 in term 3.
I20260709 13:15:42.178257  9026 leader_election.cc:304] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7616ac7f73484817ab55d8589988ad44, e06ad0761567434f9f3b1dcb2a6c592d; no voters: 
I20260709 13:15:42.179287  9421 raft_consensus.cc:2804] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:42.179762  9421 raft_consensus.cc:697] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 3 LEADER]: Becoming Leader. State: Replica: 7616ac7f73484817ab55d8589988ad44, State: Running, Role: LEADER
I20260709 13:15:42.180619  9421 consensus_queue.cc:237] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } }
I20260709 13:15:42.191864  8954 catalog_manager.cc:5697] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 reported cstate change: term changed from 2 to 3, leader changed from b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66) to 7616ac7f73484817ab55d8589988ad44 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "7616ac7f73484817ab55d8589988ad44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:42.377157  9208 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 5 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:42.378813  9421 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:42.394255  9133 raft_consensus.cc:1275] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 5 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:42.396417  9429 consensus_queue.cc:1048] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:42.441473  9060 raft_consensus.cc:1275] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.442978  9387 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.454241  9133 raft_consensus.cc:1275] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.456427  9415 consensus_queue.cc:1048] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:42.497964  9060 raft_consensus.cc:1275] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.499624  9434 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7616ac7f73484817ab55d8589988ad44" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.510180  9133 raft_consensus.cc:1275] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Refusing update from remote peer e06ad0761567434f9f3b1dcb2a6c592d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.511924  9387 consensus_queue.cc:1048] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.531486  9208 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:42.533010  9429 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:42.543766  9133 raft_consensus.cc:1275] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:42.545353  9429 consensus_queue.cc:1048] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:42.572086  9208 raft_consensus.cc:1275] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.574069  9429 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06ad0761567434f9f3b1dcb2a6c592d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.585999  9133 raft_consensus.cc:1275] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Refusing update from remote peer 7616ac7f73484817ab55d8589988ad44: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:42.587630  9421 consensus_queue.cc:1048] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d8fa77f77045a9a9154a72ebb5e785" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:42.866284  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:42.872236  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
I20260709 13:15:42.905995  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:42.906879  8925 tablet_replica.cc:333] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.907720  8925 raft_consensus.cc:2243] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.908396  8925 raft_consensus.cc:2272] T 0f4de5ebc32d4a979c35122f55a6619d P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.911865  8925 tablet_replica.cc:333] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.912530  8925 raft_consensus.cc:2243] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:15:42.913574  8925 raft_consensus.cc:2272] T 5e00185e4919476a922eb432f336b644 P 7616ac7f73484817ab55d8589988ad44 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.916530  8925 tablet_replica.cc:333] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.917057  8925 raft_consensus.cc:2243] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.917472  8925 raft_consensus.cc:2272] T 290b9eb922674f538e458e07dcae691f P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.919866  8925 tablet_replica.cc:333] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.920413  8925 raft_consensus.cc:2243] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.920907  8925 raft_consensus.cc:2272] T b0ed33b99f2b44b5a2fd4925b3afd804 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.922979  8925 tablet_replica.cc:333] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.923579  8925 raft_consensus.cc:2243] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:42.924379  8925 raft_consensus.cc:2272] T 2ad6407878c3407aa2ca069e71226e45 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.926558  8925 tablet_replica.cc:333] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.927186  8925 raft_consensus.cc:2243] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:15:42.928036  8925 raft_consensus.cc:2272] T 76d8fa9ba9c3459cb5760fe37d04f4fa P 7616ac7f73484817ab55d8589988ad44 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.930426  8925 tablet_replica.cc:333] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.930941  8925 raft_consensus.cc:2243] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.931461  8925 raft_consensus.cc:2272] T e54e833462b44129b948504f0eb55714 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.933138  8925 tablet_replica.cc:333] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.933615  8925 raft_consensus.cc:2243] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.934003  8925 raft_consensus.cc:2272] T 2fb34a53e97e4ad1b3cd31bc21526114 P 7616ac7f73484817ab55d8589988ad44 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:42.935812  8925 tablet_replica.cc:333] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44: stopping tablet replica
I20260709 13:15:42.936229  8925 raft_consensus.cc:2243] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:42.936625  8925 raft_consensus.cc:2272] T f577a8bd1f0e42c89c44580265ee5127 P 7616ac7f73484817ab55d8589988ad44 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.054975  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:15:43.063936  9175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43527: connect: Connection refused (error 111)
W20260709 13:15:43.069069  9175 consensus_peers.cc:597] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d -> Peer 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527): Couldn't send request to peer 7616ac7f73484817ab55d8589988ad44. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:43.079541  9175 consensus_peers.cc:597] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d -> Peer 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527): Couldn't send request to peer 7616ac7f73484817ab55d8589988ad44. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:43.085893  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
I20260709 13:15:43.119928  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:43.120744  8925 tablet_replica.cc:333] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.121490  8925 raft_consensus.cc:2243] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.122432  8925 raft_consensus.cc:2272] T e54e833462b44129b948504f0eb55714 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.124424  8925 tablet_replica.cc:333] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.125025  8925 raft_consensus.cc:2243] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.125859  8925 raft_consensus.cc:2272] T f577a8bd1f0e42c89c44580265ee5127 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.127863  8925 tablet_replica.cc:333] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.128468  8925 raft_consensus.cc:2243] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.129303  8925 raft_consensus.cc:2272] T 2fb34a53e97e4ad1b3cd31bc21526114 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.131055  8925 tablet_replica.cc:333] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.131687  8925 raft_consensus.cc:2243] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.132169  8925 raft_consensus.cc:2272] T 5e00185e4919476a922eb432f336b644 P b1d8fa77f77045a9a9154a72ebb5e785 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.135522  8925 tablet_replica.cc:333] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.136323  8925 raft_consensus.cc:2243] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.136938  8925 raft_consensus.cc:2272] T 0f4de5ebc32d4a979c35122f55a6619d P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.140260  8925 tablet_replica.cc:333] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
W20260709 13:15:43.140511  9175 consensus_peers.cc:597] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d -> Peer 7616ac7f73484817ab55d8589988ad44 (127.8.183.65:43527): Couldn't send request to peer 7616ac7f73484817ab55d8589988ad44. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:43.141124  8925 raft_consensus.cc:2243] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.141708  8925 raft_consensus.cc:2272] T b0ed33b99f2b44b5a2fd4925b3afd804 P b1d8fa77f77045a9a9154a72ebb5e785 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.143858  8925 tablet_replica.cc:333] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.144439  8925 raft_consensus.cc:2243] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.144913  8925 raft_consensus.cc:2272] T 290b9eb922674f538e458e07dcae691f P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.147575  8925 tablet_replica.cc:333] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.148135  8925 raft_consensus.cc:2243] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.148613  8925 raft_consensus.cc:2272] T 2ad6407878c3407aa2ca069e71226e45 P b1d8fa77f77045a9a9154a72ebb5e785 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.151258  8925 tablet_replica.cc:333] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785: stopping tablet replica
I20260709 13:15:43.151782  8925 raft_consensus.cc:2243] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.152222  8925 raft_consensus.cc:2272] T 76d8fa9ba9c3459cb5760fe37d04f4fa P b1d8fa77f77045a9a9154a72ebb5e785 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.182394  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
I20260709 13:15:43.204030  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
W20260709 13:15:43.224457  9173 consensus_peers.cc:597] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d -> Peer b1d8fa77f77045a9a9154a72ebb5e785 (127.8.183.66:46553): Couldn't send request to peer b1d8fa77f77045a9a9154a72ebb5e785. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:46553: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:43.241190  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:43.242071  8925 tablet_replica.cc:333] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.242775  8925 raft_consensus.cc:2243] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.243392  8925 raft_consensus.cc:2272] T 5e00185e4919476a922eb432f336b644 P e06ad0761567434f9f3b1dcb2a6c592d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.246030  8925 tablet_replica.cc:333] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.246658  8925 raft_consensus.cc:2243] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.247740  8925 raft_consensus.cc:2272] T b0ed33b99f2b44b5a2fd4925b3afd804 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.249770  8925 tablet_replica.cc:333] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.250355  8925 raft_consensus.cc:2243] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.250866  8925 raft_consensus.cc:2272] T 2ad6407878c3407aa2ca069e71226e45 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.253242  8925 tablet_replica.cc:333] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.253831  8925 raft_consensus.cc:2243] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.254778  8925 raft_consensus.cc:2272] T 290b9eb922674f538e458e07dcae691f P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.257045  8925 tablet_replica.cc:333] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.257630  8925 raft_consensus.cc:2243] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.258129  8925 raft_consensus.cc:2272] T 76d8fa9ba9c3459cb5760fe37d04f4fa P e06ad0761567434f9f3b1dcb2a6c592d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.260452  8925 tablet_replica.cc:333] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.261039  8925 raft_consensus.cc:2243] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.261513  8925 raft_consensus.cc:2272] T f577a8bd1f0e42c89c44580265ee5127 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.263794  8925 tablet_replica.cc:333] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.264362  8925 raft_consensus.cc:2243] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.264828  8925 raft_consensus.cc:2272] T e54e833462b44129b948504f0eb55714 P e06ad0761567434f9f3b1dcb2a6c592d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.266557  8925 tablet_replica.cc:333] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.267096  8925 raft_consensus.cc:2243] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.267654  8925 raft_consensus.cc:2272] T 2fb34a53e97e4ad1b3cd31bc21526114 P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.269531  8925 tablet_replica.cc:333] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d: stopping tablet replica
I20260709 13:15:43.270067  8925 raft_consensus.cc:2243] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.270898  8925 raft_consensus.cc:2272] T 0f4de5ebc32d4a979c35122f55a6619d P e06ad0761567434f9f3b1dcb2a6c592d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.299963  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:15:43.324112  8925 master.cc:562] Master@127.8.183.126:46707 shutting down...
I20260709 13:15:43.552021  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:43.552668  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.553040  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P ba146d0315bf4b2bbab46eeed0927c3c: stopping tablet replica
I20260709 13:15:43.574705  8925 master.cc:584] Master@127.8.183.126:46707 shutdown complete.
7616ac7f73484817ab55d8589988ad44  4
b1d8fa77f77045a9a9154a72ebb5e785  4
e06ad0761567434f9f3b1dcb2a6c592d  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13815 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:15:43.622406  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:42731
I20260709 13:15:43.623759  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:43.631283  9447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.631268  9448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.632069  9450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:43.632750  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:43.633688  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:43.633944  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:43.634126  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602943634107 us; error 0 us; skew 500 ppm
I20260709 13:15:43.634711  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:43.637676  8925 webserver.cc:533] Webserver started at http://127.8.183.126:37141/ using document root <none> and password file <none>
I20260709 13:15:43.638299  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:43.638543  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:43.638877  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:43.640168  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "3a208c10e923497893da9946ddfe2b5e"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.645385  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:15:43.649515  9455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:43.650661  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:43.650980  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "3a208c10e923497893da9946ddfe2b5e"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.651367  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:43.675566  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:43.676863  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:43.721992  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:42731
I20260709 13:15:43.722139  9506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:42731 every 8 connection(s)
I20260709 13:15:43.728016  9507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:43.739651  9507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e: Bootstrap starting.
I20260709 13:15:43.745518  9507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:43.751103  9507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e: No bootstrap required, opened a new log
I20260709 13:15:43.753968  9507 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER }
I20260709 13:15:43.754451  9507 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:43.754730  9507 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a208c10e923497893da9946ddfe2b5e, State: Initialized, Role: FOLLOWER
I20260709 13:15:43.755559  9507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER }
I20260709 13:15:43.756104  9507 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:43.756372  9507 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:43.756690  9507 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:43.761204  9507 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER }
I20260709 13:15:43.761874  9507 leader_election.cc:304] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a208c10e923497893da9946ddfe2b5e; no voters: 
I20260709 13:15:43.763216  9507 leader_election.cc:290] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:43.763648  9510 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:43.765158  9510 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 1 LEADER]: Becoming Leader. State: Replica: 3a208c10e923497893da9946ddfe2b5e, State: Running, Role: LEADER
I20260709 13:15:43.765959  9510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER }
I20260709 13:15:43.766439  9507 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:43.771257  9511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a208c10e923497893da9946ddfe2b5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER } }
I20260709 13:15:43.772308  9511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:43.772670  9512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a208c10e923497893da9946ddfe2b5e. Latest consensus state: current_term: 1 leader_uuid: "3a208c10e923497893da9946ddfe2b5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a208c10e923497893da9946ddfe2b5e" member_type: VOTER } }
I20260709 13:15:43.774683  9512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:43.778813  9517 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:43.784618  9517 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:43.787621  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:43.795193  9517 catalog_manager.cc:1383] Generated new cluster ID: 7063592862ba49d8a24b4c3b23e3a62c
I20260709 13:15:43.795571  9517 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:43.819736  9517 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:43.821115  9517 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:43.835805  9517 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e: Generated new TSK 0
I20260709 13:15:43.836640  9517 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:43.854825  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:43.862498  9528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.863919  9529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.865157  9531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:43.866096  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:43.867100  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:43.867427  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:43.867621  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602943867602 us; error 0 us; skew 500 ppm
I20260709 13:15:43.868240  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:43.871069  8925 webserver.cc:533] Webserver started at http://127.8.183.65:34443/ using document root <none> and password file <none>
I20260709 13:15:43.871726  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:43.871982  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:43.872293  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:43.873531  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "5e6a8136a1f848c5aceed687ee884228"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.878875  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:15:43.883368  9536 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:43.884507  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:15:43.884941  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "5e6a8136a1f848c5aceed687ee884228"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.885283  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:43.899892  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:43.901316  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:43.903227  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:43.905956  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:43.906236  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:43.906562  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:43.906771  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:43.954501  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:44977
I20260709 13:15:43.954592  9598 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:44977 every 8 connection(s)
I20260709 13:15:43.960913  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:43.969940  9603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.972162  9604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:43.974018  9606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:43.974249  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:43.975679  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:43.975920  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:43.976058  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602943976046 us; error 0 us; skew 500 ppm
I20260709 13:15:43.976531  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:43.979736  8925 webserver.cc:533] Webserver started at http://127.8.183.66:36557/ using document root <none> and password file <none>
I20260709 13:15:43.980381  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:43.980651  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:43.981012  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:43.982725  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.984009  9599 heartbeater.cc:344] Connected to a master server at 127.8.183.126:42731
I20260709 13:15:43.984592  9599 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:43.985469  9599 heartbeater.cc:507] Master 127.8.183.126:42731 requested a full tablet report, sending...
I20260709 13:15:43.988129  9472 ts_manager.cc:194] Registered new tserver with Master: 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977)
I20260709 13:15:43.988766  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:15:43.990167  9472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46276
I20260709 13:15:43.993039  9611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:43.994042  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:15:43.994385  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
format_stamp: "Formatted at 2026-07-09 13:15:43 on dist-test-slave-ch3x"
I20260709 13:15:43.994692  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:44.005995  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:44.007694  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:44.009555  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:44.012387  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:44.012606  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:44.012943  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:44.013103  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:44.064205  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:46145
I20260709 13:15:44.064347  9673 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:46145 every 8 connection(s)
I20260709 13:15:44.069728  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:44.078034  9677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:44.080364  9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:44.083218  9680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:44.083470  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:44.084859  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:44.085178  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:44.085424  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602944085400 us; error 0 us; skew 500 ppm
I20260709 13:15:44.086015  9674 heartbeater.cc:344] Connected to a master server at 127.8.183.126:42731
I20260709 13:15:44.086167  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:44.086684  9674 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:44.088102  9674 heartbeater.cc:507] Master 127.8.183.126:42731 requested a full tablet report, sending...
I20260709 13:15:44.090159  8925 webserver.cc:533] Webserver started at http://127.8.183.67:35357/ using document root <none> and password file <none>
I20260709 13:15:44.090857  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:44.091164  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:44.091398  9472 ts_manager.cc:194] Registered new tserver with Master: 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:44.091532  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:44.093421  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "792521f85edc44319aa7bf189556921e"
format_stamp: "Formatted at 2026-07-09 13:15:44 on dist-test-slave-ch3x"
I20260709 13:15:44.093528  9472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46286
I20260709 13:15:44.099977  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:15:44.104226  9685 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:44.105370  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:15:44.105894  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "792521f85edc44319aa7bf189556921e"
format_stamp: "Formatted at 2026-07-09 13:15:44 on dist-test-slave-ch3x"
I20260709 13:15:44.106261  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:44.132731  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:44.134135  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:44.135790  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:44.138536  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:44.138844  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:44.139240  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:44.139451  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:44.191045  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:34753
I20260709 13:15:44.191282  9747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:34753 every 8 connection(s)
I20260709 13:15:44.208771  9748 heartbeater.cc:344] Connected to a master server at 127.8.183.126:42731
I20260709 13:15:44.209255  9748 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:44.210095  9748 heartbeater.cc:507] Master 127.8.183.126:42731 requested a full tablet report, sending...
I20260709 13:15:44.212610  9472 ts_manager.cc:194] Registered new tserver with Master: 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.212761  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017846416s
I20260709 13:15:44.214588  9472 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46296
I20260709 13:15:44.244868  9472 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46298:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:44.248386  9472 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:44.336565  9564 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9f7c1827047c489b4990aa24cbbb4 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:44.338421  9564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9f7c1827047c489b4990aa24cbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.338373  9563 tablet_service.cc:1511] Processing CreateTablet for tablet 22bc2a2983fc4a01b8c7d361a9c820be (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:44.340118  9563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bc2a2983fc4a01b8c7d361a9c820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.346366  9562 tablet_service.cc:1511] Processing CreateTablet for tablet d21e8f4a2af74ed4a4c460ce24457721 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:44.348070  9562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21e8f4a2af74ed4a4c460ce24457721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.349802  9639 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9f7c1827047c489b4990aa24cbbb4 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:44.352444  9638 tablet_service.cc:1511] Processing CreateTablet for tablet 22bc2a2983fc4a01b8c7d361a9c820be (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:44.356557  9635 tablet_service.cc:1511] Processing CreateTablet for tablet 2b0db9e1138e4ea6b777713504155a37 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:44.358713  9633 tablet_service.cc:1511] Processing CreateTablet for tablet f6910d75b0be4ff8b6a12f32f2cff6a1 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:44.359462  9634 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1939727ba9472ba9c1325959a0062d (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:44.360831  9637 tablet_service.cc:1511] Processing CreateTablet for tablet d21e8f4a2af74ed4a4c460ce24457721 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:44.361541  9636 tablet_service.cc:1511] Processing CreateTablet for tablet 31d1925dbd7a4b018a9b098740dbd71f (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:44.363242  9639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9f7c1827047c489b4990aa24cbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.364224  9561 tablet_service.cc:1511] Processing CreateTablet for tablet 31d1925dbd7a4b018a9b098740dbd71f (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:44.367419  9763 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.367630  9636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d1925dbd7a4b018a9b098740dbd71f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.369534  9633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6910d75b0be4ff8b6a12f32f2cff6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.369742  9559 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1939727ba9472ba9c1325959a0062d (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:44.370664  9634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1939727ba9472ba9c1325959a0062d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.367520  9560 tablet_service.cc:1511] Processing CreateTablet for tablet 2b0db9e1138e4ea6b777713504155a37 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:44.371516  9638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bc2a2983fc4a01b8c7d361a9c820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.375667  9637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21e8f4a2af74ed4a4c460ce24457721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.379657  9635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0db9e1138e4ea6b777713504155a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.387606  9561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d1925dbd7a4b018a9b098740dbd71f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.390954  9559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1939727ba9472ba9c1325959a0062d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.394627  9764 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.395695  9560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0db9e1138e4ea6b777713504155a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.397742  9564 tablet_service.cc:1511] Processing CreateTablet for tablet f6910d75b0be4ff8b6a12f32f2cff6a1 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:44.398219  9558 tablet_service.cc:1511] Processing CreateTablet for tablet 7d57583a0d6b4393bd5150e4c28b21e0 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:44.399281  9713 tablet_service.cc:1511] Processing CreateTablet for tablet 6ef9f7c1827047c489b4990aa24cbbb4 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:15:44.399765  9558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d57583a0d6b4393bd5150e4c28b21e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.400763  9713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ef9f7c1827047c489b4990aa24cbbb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.401335  9564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6910d75b0be4ff8b6a12f32f2cff6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.404544  9763 tablet_bootstrap.cc:654] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.405190  9712 tablet_service.cc:1511] Processing CreateTablet for tablet 22bc2a2983fc4a01b8c7d361a9c820be (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:15:44.406880  9712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bc2a2983fc4a01b8c7d361a9c820be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.408561  9711 tablet_service.cc:1511] Processing CreateTablet for tablet d21e8f4a2af74ed4a4c460ce24457721 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:15:44.411516  9711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d21e8f4a2af74ed4a4c460ce24457721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.421110  9763 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.421733  9763 ts_tablet_manager.cc:1403] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.000s
I20260709 13:15:44.425690  9763 raft_consensus.cc:359] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.426652  9763 raft_consensus.cc:385] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.426999  9763 raft_consensus.cc:740] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.428290  9763 consensus_queue.cc:260] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.435201  9710 tablet_service.cc:1511] Processing CreateTablet for tablet 31d1925dbd7a4b018a9b098740dbd71f (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:15:44.436966  9710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31d1925dbd7a4b018a9b098740dbd71f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.440320  9709 tablet_service.cc:1511] Processing CreateTablet for tablet 2b0db9e1138e4ea6b777713504155a37 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:15:44.442045  9709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0db9e1138e4ea6b777713504155a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.446874  9557 tablet_service.cc:1511] Processing CreateTablet for tablet f5b0dca477714bdcba7d1a1966d41bbd (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:44.448557  9557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b0dca477714bdcba7d1a1966d41bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.449046  9637 tablet_service.cc:1511] Processing CreateTablet for tablet 7d57583a0d6b4393bd5150e4c28b21e0 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:44.450564  9637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d57583a0d6b4393bd5150e4c28b21e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.452351  9708 tablet_service.cc:1511] Processing CreateTablet for tablet 3d1939727ba9472ba9c1325959a0062d (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:15:44.452839  9707 tablet_service.cc:1511] Processing CreateTablet for tablet f6910d75b0be4ff8b6a12f32f2cff6a1 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:15:44.454202  9708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d1939727ba9472ba9c1325959a0062d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.455029  9707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6910d75b0be4ff8b6a12f32f2cff6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.458360  9639 tablet_service.cc:1511] Processing CreateTablet for tablet f5b0dca477714bdcba7d1a1966d41bbd (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:44.459955  9639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b0dca477714bdcba7d1a1966d41bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.473332  9706 tablet_service.cc:1511] Processing CreateTablet for tablet 7d57583a0d6b4393bd5150e4c28b21e0 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:15:44.474978  9706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d57583a0d6b4393bd5150e4c28b21e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.475025  9705 tablet_service.cc:1511] Processing CreateTablet for tablet f5b0dca477714bdcba7d1a1966d41bbd (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:15:44.477255  9705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5b0dca477714bdcba7d1a1966d41bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.477859  9561 tablet_service.cc:1511] Processing CreateTablet for tablet c8c9ac1b66b34d879376b4b86cc87442 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:44.486965  9634 tablet_service.cc:1511] Processing CreateTablet for tablet c8c9ac1b66b34d879376b4b86cc87442 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:44.488538  9634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c9ac1b66b34d879376b4b86cc87442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.493034  9561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c9ac1b66b34d879376b4b86cc87442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.493654  9599 heartbeater.cc:499] Master 127.8.183.126:42731 was elected leader, sending a full tablet report...
I20260709 13:15:44.497701  9704 tablet_service.cc:1511] Processing CreateTablet for tablet c8c9ac1b66b34d879376b4b86cc87442 (DEFAULT_TABLE table=test-workload [id=0de4fd50f9224d6e9b0b310c770aeb1d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:15:44.499462  9704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c9ac1b66b34d879376b4b86cc87442. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:44.500218  9763 ts_tablet_manager.cc:1434] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.078s	user 0.011s	sys 0.004s
I20260709 13:15:44.501384  9763 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.519737  9764 tablet_bootstrap.cc:654] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.576903  9764 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.577471  9764 ts_tablet_manager.cc:1403] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.183s	user 0.010s	sys 0.005s
I20260709 13:15:44.594434  9767 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.600054  9764 raft_consensus.cc:359] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.600986  9764 raft_consensus.cc:385] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.601332  9764 raft_consensus.cc:740] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.602077  9763 tablet_bootstrap.cc:654] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.602569  9764 consensus_queue.cc:260] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.615550  9674 heartbeater.cc:499] Master 127.8.183.126:42731 was elected leader, sending a full tablet report...
I20260709 13:15:44.615968  9764 ts_tablet_manager.cc:1434] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.010s
I20260709 13:15:44.617115  9764 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.618557  9763 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.619241  9767 tablet_bootstrap.cc:654] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.619241  9763 ts_tablet_manager.cc:1403] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.009s
I20260709 13:15:44.623407  9763 raft_consensus.cc:359] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.624472  9764 tablet_bootstrap.cc:654] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.624739  9763 raft_consensus.cc:385] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.625406  9763 raft_consensus.cc:740] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.626984  9763 consensus_queue.cc:260] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.638667  9763 ts_tablet_manager.cc:1434] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 13:15:44.640753  9764 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.640987  9763 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.641407  9764 ts_tablet_manager.cc:1403] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.011s
I20260709 13:15:44.644654  9764 raft_consensus.cc:359] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.645217  9766 raft_consensus.cc:493] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.645524  9764 raft_consensus.cc:385] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.645917  9764 raft_consensus.cc:740] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.646041  9766 raft_consensus.cc:515] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.647079  9764 consensus_queue.cc:260] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.649989  9764 ts_tablet_manager.cc:1434] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 13:15:44.651085  9764 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.654218  9769 raft_consensus.cc:493] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.654361  9766 leader_election.cc:290] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.654884  9769 raft_consensus.cc:515] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.658105  9769 leader_election.cc:290] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.663509  9763 tablet_bootstrap.cc:654] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.665261  9767 tablet_bootstrap.cc:492] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:44.669497  9767 ts_tablet_manager.cc:1403] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.022s
I20260709 13:15:44.674468  9767 raft_consensus.cc:359] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.675700  9767 raft_consensus.cc:385] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.676100  9767 raft_consensus.cc:740] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.679891  9767 consensus_queue.cc:260] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.685248  9748 heartbeater.cc:499] Master 127.8.183.126:42731 was elected leader, sending a full tablet report...
I20260709 13:15:44.685987  9767 ts_tablet_manager.cc:1434] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.002s
I20260709 13:15:44.689603  9766 raft_consensus.cc:493] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.690475  9766 raft_consensus.cc:515] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.691792  9763 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.692430  9763 ts_tablet_manager.cc:1403] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.000s
I20260709 13:15:44.692921  9764 tablet_bootstrap.cc:654] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.693612  9766 leader_election.cc:290] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.695984  9763 raft_consensus.cc:359] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.700217  9763 raft_consensus.cc:385] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.700608  9763 raft_consensus.cc:740] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.696790  9767 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.706311  9763 consensus_queue.cc:260] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.713469  9763 ts_tablet_manager.cc:1434] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.003s
I20260709 13:15:44.716532  9648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:44.717114  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:44.717859  9649 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
W20260709 13:15:44.718446  9539 leader_election.cc:343] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.709756  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:44.720477  9539 leader_election.cc:304] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:44.721555  9766 raft_consensus.cc:2804] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.721930  9763 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.721952  9766 raft_consensus.cc:493] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.722477  9766 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.723384  9764 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.723924  9764 ts_tablet_manager.cc:1403] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.007s
I20260709 13:15:44.725103  9767 tablet_bootstrap.cc:654] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.727358  9764 raft_consensus.cc:359] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.728164  9764 raft_consensus.cc:385] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.728570  9764 raft_consensus.cc:740] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.729480  9763 tablet_bootstrap.cc:654] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.730077  9764 consensus_queue.cc:260] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
W20260709 13:15:44.736707  9614 leader_election.cc:343] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792521f85edc44319aa7bf189556921e (127.8.183.67:34753): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.741295  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:44.742412  9723 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:44.745489  9767 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:44.746207  9767 ts_tablet_manager.cc:1403] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.006s
I20260709 13:15:44.745223  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:44.750500  9764 ts_tablet_manager.cc:1434] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:15:44.750344  9767 raft_consensus.cc:359] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.751536  9767 raft_consensus.cc:385] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.751935  9767 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.753578  9767 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.753810  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:44.756521  9767 ts_tablet_manager.cc:1434] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
W20260709 13:15:44.756506  9614 leader_election.cc:343] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.757398  9614 leader_election.cc:304] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0; no voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e
I20260709 13:15:44.758327  9767 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.759083  9769 raft_consensus.cc:2749] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.763010  9764 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
W20260709 13:15:44.762712  9539 leader_election.cc:343] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792521f85edc44319aa7bf189556921e (127.8.183.67:34753): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.763672  9539 leader_election.cc:304] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228; no voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e
I20260709 13:15:44.730773  9766 raft_consensus.cc:515] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.766610  9767 tablet_bootstrap.cc:654] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.768154  9766 leader_election.cc:290] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.770855  9764 tablet_bootstrap.cc:654] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.776058  9766 raft_consensus.cc:2749] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:44.774576  9763 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.778959  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:44.779767  9649 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:44.781356  9520 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:44.782035  9764 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.782548  9764 ts_tablet_manager.cc:1403] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.004s
I20260709 13:15:44.782779  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:44.782053  9767 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:44.783450  9723 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.783829  9767 ts_tablet_manager.cc:1403] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.005s
I20260709 13:15:44.785857  9764 raft_consensus.cc:359] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.787184  9764 raft_consensus.cc:385] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.787580  9764 raft_consensus.cc:740] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.787736  9767 raft_consensus.cc:359] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.788605  9767 raft_consensus.cc:385] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.788496  9764 consensus_queue.cc:260] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.788988  9767 raft_consensus.cc:740] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.789222  9649 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:44.790202  9767 consensus_queue.cc:260] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.790607  9539 leader_election.cc:304] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:44.792579  9723 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:44.792806  9767 ts_tablet_manager.cc:1434] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:15:44.794168  9767 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.794930  9766 raft_consensus.cc:2804] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.795611  9766 raft_consensus.cc:697] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:44.796528  9766 consensus_queue.cc:237] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.779850  9763 ts_tablet_manager.cc:1403] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.024s
I20260709 13:15:44.800352  9764 ts_tablet_manager.cc:1434] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:15:44.802109  9767 tablet_bootstrap.cc:654] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.802970  9763 raft_consensus.cc:359] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.804679  9764 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.803892  9763 raft_consensus.cc:385] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.812875  9779 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.813437  9779 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.816512  9779 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:44.816570  9764 tablet_bootstrap.cc:654] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.812927  9763 raft_consensus.cc:740] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.819533  9763 consensus_queue.cc:260] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.822125  9763 ts_tablet_manager.cc:1434] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:15:44.823432  9763 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.822266  9470 catalog_manager.cc:5697] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.828193  9767 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:44.830086  9767 ts_tablet_manager.cc:1403] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.004s
I20260709 13:15:44.834017  9763 tablet_bootstrap.cc:654] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.837271  9767 raft_consensus.cc:359] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.842854  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:44.843756  9764 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.844276  9764 ts_tablet_manager.cc:1403] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.000s
W20260709 13:15:44.845026  9688 leader_election.cc:343] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.845777  9767 raft_consensus.cc:385] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.846673  9767 raft_consensus.cc:740] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.847625  9764 raft_consensus.cc:359] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.850379  9767 consensus_queue.cc:260] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.851380  9764 raft_consensus.cc:385] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.851742  9764 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.852692  9764 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.856073  9767 ts_tablet_manager.cc:1434] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 13:15:44.857697  9767 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.866756  9764 ts_tablet_manager.cc:1434] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.003s
I20260709 13:15:44.871047  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:44.871963  9649 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 0.
I20260709 13:15:44.873819  9688 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:44.875182  9779 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.875590  9779 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.875942  9779 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.878430  9764 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.880119  9767 tablet_bootstrap.cc:654] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.885407  9779 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.890384  9763 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.891525  9763 ts_tablet_manager.cc:1403] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.068s	user 0.038s	sys 0.026s
I20260709 13:15:44.895610  9779 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:44.895964  9763 raft_consensus.cc:359] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.897111  9763 raft_consensus.cc:385] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.897902  9763 raft_consensus.cc:740] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.898164  9766 raft_consensus.cc:493] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.898813  9766 raft_consensus.cc:515] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.898969  9763 consensus_queue.cc:260] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.899483  9767 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:44.900731  9767 ts_tablet_manager.cc:1403] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.001s
I20260709 13:15:44.901160  9766 leader_election.cc:290] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.901479  9763 ts_tablet_manager.cc:1434] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:44.902513  9763 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.896347  9779 raft_consensus.cc:493] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.903039  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:44.904330  9779 raft_consensus.cc:515] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.905573  9767 raft_consensus.cc:359] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.906389  9767 raft_consensus.cc:385] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.906617  9779 leader_election.cc:290] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:44.906704  9767 raft_consensus.cc:740] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.886257  9764 tablet_bootstrap.cc:654] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:44.912834  9539 leader_election.cc:343] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.903254  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:44.913540  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:44.913890  9723 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:44.915932  9539 leader_election.cc:304] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:44.916837  9763 tablet_bootstrap.cc:654] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.917236  9766 raft_consensus.cc:2804] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:44.917728  9766 raft_consensus.cc:493] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.918148  9766 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.918670  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:44.919512  9649 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.924695  9766 raft_consensus.cc:515] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.917228  9767 consensus_queue.cc:260] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.926995  9766 leader_election.cc:290] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:44.928347  9648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:44.932224  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:44.932998  9723 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:44.933856  9649 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 1.
I20260709 13:15:44.934280  9764 tablet_bootstrap.cc:492] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:44.935056  9767 ts_tablet_manager.cc:1434] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 13:15:44.933036  9763 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:44.934803  9764 ts_tablet_manager.cc:1403] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.009s
I20260709 13:15:44.935869  9763 ts_tablet_manager.cc:1403] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.003s
I20260709 13:15:44.936317  9767 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:44.938987  9764 raft_consensus.cc:359] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.939321  9763 raft_consensus.cc:359] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.939859  9764 raft_consensus.cc:385] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.940336  9763 raft_consensus.cc:385] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:44.940393  9764 raft_consensus.cc:740] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.940794  9763 raft_consensus.cc:740] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:44.941464  9764 consensus_queue.cc:260] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.941896  9763 consensus_queue.cc:260] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.944119  9764 ts_tablet_manager.cc:1434] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:44.945185  9764 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:44.945492  9767 tablet_bootstrap.cc:654] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.947829  9763 ts_tablet_manager.cc:1434] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:15:44.949139  9763 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:44.954214  9764 tablet_bootstrap.cc:654] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:44.957053  9763 tablet_bootstrap.cc:654] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:44.969321  9539 leader_election.cc:343] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:44.981040  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:44.984864  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:44.988065  9688 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:44.989462  9794 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.990118  9723 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:44.990147  9794 raft_consensus.cc:697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:44.991556  9539 leader_election.cc:304] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:44.991459  9794 consensus_queue.cc:237] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:44.992479  9766 raft_consensus.cc:2804] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:44.992990  9766 raft_consensus.cc:697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:44.993860  9766 consensus_queue.cc:237] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
W20260709 13:15:44.989509  9688 leader_election.cc:343] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:45.001533  9688 leader_election.cc:343] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:45.002735  9688 leader_election.cc:343] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.003314  9688 leader_election.cc:304] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 792521f85edc44319aa7bf189556921e; no voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:45.004354  9779 raft_consensus.cc:2749] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.019253  9767 tablet_bootstrap.cc:492] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:45.016746  9471 catalog_manager.cc:5697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.019851  9767 ts_tablet_manager.cc:1403] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.062s
I20260709 13:15:45.019904  9469 catalog_manager.cc:5697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 0 to 1, leader changed from <none> to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 1 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.021056  9763 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:45.021660  9763 ts_tablet_manager.cc:1403] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.020s
I20260709 13:15:45.023424  9767 raft_consensus.cc:359] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.024088  9767 raft_consensus.cc:385] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.024390  9767 raft_consensus.cc:740] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.024852  9763 raft_consensus.cc:359] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.025710  9763 raft_consensus.cc:385] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.026051  9763 raft_consensus.cc:740] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.027037  9763 consensus_queue.cc:260] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.026938  9767 consensus_queue.cc:260] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.030218  9763 ts_tablet_manager.cc:1434] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:15:45.031011  9764 tablet_bootstrap.cc:492] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:45.031507  9763 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:45.031594  9764 ts_tablet_manager.cc:1403] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.087s	user 0.026s	sys 0.038s
I20260709 13:15:45.032218  9767 ts_tablet_manager.cc:1434] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:15:45.033469  9767 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:45.034870  9764 raft_consensus.cc:359] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.035827  9764 raft_consensus.cc:385] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.036159  9764 raft_consensus.cc:740] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.037139  9764 consensus_queue.cc:260] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.039961  9766 raft_consensus.cc:493] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.040484  9766 raft_consensus.cc:515] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.042698  9767 tablet_bootstrap.cc:654] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.042764  9766 leader_election.cc:290] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.045123  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:45.045828  9649 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:45.046325  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.046949  9723 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:45.047204  9539 leader_election.cc:304] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.048666  9766 raft_consensus.cc:2804] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.049041  9766 raft_consensus.cc:493] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.049393  9766 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.050648  9764 ts_tablet_manager.cc:1434] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:15:45.054630  9764 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:45.057605  9766 raft_consensus.cc:515] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.039355  9763 tablet_bootstrap.cc:654] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.064950  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.065500  9764 tablet_bootstrap.cc:654] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.065719  9649 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.069650  9783 raft_consensus.cc:493] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.070219  9783 raft_consensus.cc:515] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.073503  9783 leader_election.cc:290] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.076587  9648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:45.077699  9648 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:45.079789  9539 leader_election.cc:304] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.082201  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.083870  9783 raft_consensus.cc:2804] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:15:45.083976  9539 leader_election.cc:343] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 792521f85edc44319aa7bf189556921e (127.8.183.67:34753): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:45.084450  9767 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:45.085007  9767 ts_tablet_manager.cc:1403] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.033s
I20260709 13:15:45.085386  9769 raft_consensus.cc:493] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.085901  9769 raft_consensus.cc:515] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.088449  9769 leader_election.cc:290] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.089216  9769 raft_consensus.cc:493] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.089605  9649 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.084365  9783 raft_consensus.cc:493] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.090957  9783 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.093870  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.089854  9769 raft_consensus.cc:515] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.094805  9723 raft_consensus.cc:2468] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.095332  9767 raft_consensus.cc:359] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.096200  9767 raft_consensus.cc:385] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.096350  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:45.096582  9767 raft_consensus.cc:740] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.097121  9574 raft_consensus.cc:2468] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.097513  9767 consensus_queue.cc:260] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.098574  9769 leader_election.cc:290] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.100270  9539 leader_election.cc:304] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.103936  9766 leader_election.cc:290] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.104290  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:45.104849  9766 raft_consensus.cc:2804] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.105506  9766 raft_consensus.cc:697] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.108228  9767 ts_tablet_manager.cc:1434] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.004s
I20260709 13:15:45.109656  9767 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:45.116170  9614 leader_election.cc:304] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.120565  9763 tablet_bootstrap.cc:492] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:45.121304  9763 ts_tablet_manager.cc:1403] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.090s	user 0.002s	sys 0.014s
I20260709 13:15:45.123682  9807 raft_consensus.cc:2804] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.124590  9807 raft_consensus.cc:493] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.125227  9807 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.118710  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.125824  9783 raft_consensus.cc:515] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.126580  9723 raft_consensus.cc:2393] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in current term 1: Already voted for candidate 5e6a8136a1f848c5aceed687ee884228 in this term.
I20260709 13:15:45.106729  9766 consensus_queue.cc:237] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.125077  9763 raft_consensus.cc:359] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.128448  9614 leader_election.cc:304] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0; no voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e
I20260709 13:15:45.129006  9763 raft_consensus.cc:385] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.129410  9763 raft_consensus.cc:740] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.118743  9767 tablet_bootstrap.cc:654] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.127766  9783 leader_election.cc:290] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.129452  9769 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.130416  9763 consensus_queue.cc:260] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.133533  9764 tablet_bootstrap.cc:492] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:45.134121  9764 ts_tablet_manager.cc:1403] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.080s	user 0.028s	sys 0.023s
I20260709 13:15:45.137188  9764 raft_consensus.cc:359] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.138607  9764 raft_consensus.cc:385] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.138679  9769 raft_consensus.cc:2749] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.139014  9764 raft_consensus.cc:740] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.139806  9764 consensus_queue.cc:260] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.142167  9764 ts_tablet_manager.cc:1434] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:45.143231  9764 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
W20260709 13:15:45.144827  9539 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:46145, user_credentials={real_user=slave}} blocked reactor thread for 44752.8us
I20260709 13:15:45.148362  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.148653  9763 ts_tablet_manager.cc:1434] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:15:45.149087  9723 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.150077  9763 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:45.152319  9807 raft_consensus.cc:515] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.157678  9807 leader_election.cc:290] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.159801  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.160712  9764 tablet_bootstrap.cc:654] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.160815  9722 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.159291  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.163550  9574 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.164776  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.165490  9649 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.157979  9763 tablet_bootstrap.cc:654] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.159929  9769 raft_consensus.cc:493] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.167357  9769 raft_consensus.cc:515] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.170997  9767 tablet_bootstrap.cc:492] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:45.171633  9767 ts_tablet_manager.cc:1403] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.004s
I20260709 13:15:45.172438  9723 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.175181  9767 raft_consensus.cc:359] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.178138  9649 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.181587  9573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:45.187388  9573 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.188333  9722 raft_consensus.cc:2468] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.184609  9721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.186990  9767 raft_consensus.cc:385] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.188822  9574 raft_consensus.cc:2468] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.189587  9767 raft_consensus.cc:740] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.185141  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.185339  9769 leader_election.cc:290] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.190376  9614 leader_election.cc:304] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.189419  9721 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.190585  9767 consensus_queue.cc:260] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.191671  9723 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.182817  9470 catalog_manager.cc:5697] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.193305  9767 ts_tablet_manager.cc:1434] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:15:45.194631  9767 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:45.193318  9614 leader_election.cc:304] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.198050  9807 raft_consensus.cc:2804] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.200810  9723 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.201164  9767 tablet_bootstrap.cc:654] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.203207  9769 raft_consensus.cc:2804] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.203877  9769 raft_consensus.cc:493] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.204344  9769 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.206560  9763 tablet_bootstrap.cc:492] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:45.207175  9763 ts_tablet_manager.cc:1403] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.004s
I20260709 13:15:45.220396  9763 raft_consensus.cc:359] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.221534  9763 raft_consensus.cc:385] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.221899  9763 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.230693  9763 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.231372  9807 raft_consensus.cc:697] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.235549  9539 leader_election.cc:304] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.236730  9808 raft_consensus.cc:2804] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.237329  9808 raft_consensus.cc:697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.237560  9767 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:45.238106  9767 ts_tablet_manager.cc:1403] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.015s
I20260709 13:15:45.238260  9808 consensus_queue.cc:237] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.241216  9767 raft_consensus.cc:359] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.242017  9767 raft_consensus.cc:385] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.242316  9767 raft_consensus.cc:740] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.243062  9767 consensus_queue.cc:260] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.246373  9763 ts_tablet_manager.cc:1434] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 13:15:45.246237  9767 ts_tablet_manager.cc:1434] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:15:45.247470  9763 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: Bootstrap starting.
I20260709 13:15:45.247762  9767 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: Bootstrap starting.
I20260709 13:15:45.248395  9766 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.243083  9769 raft_consensus.cc:515] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.249374  9766 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.249902  9764 tablet_bootstrap.cc:492] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:45.250412  9764 ts_tablet_manager.cc:1403] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.107s	user 0.029s	sys 0.023s
I20260709 13:15:45.245916  9807 consensus_queue.cc:237] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.253544  9766 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.254058  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.254805  9574 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.256307  9764 raft_consensus.cc:359] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.257373  9764 raft_consensus.cc:385] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.257730  9764 raft_consensus.cc:740] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.259500  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.260871  9769 leader_election.cc:290] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.261999  9764 consensus_queue.cc:260] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.262557  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:45.264331  9649 raft_consensus.cc:2393] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e6a8136a1f848c5aceed687ee884228 in current term 1: Already voted for candidate 792521f85edc44319aa7bf189556921e in this term.
I20260709 13:15:45.265733  9764 ts_tablet_manager.cc:1434] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:15:45.267302  9764 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: Bootstrap starting.
I20260709 13:15:45.270988  9539 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228; no voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e
I20260709 13:15:45.271616  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.276513  9723 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.282752  9809 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.274047  9767 tablet_bootstrap.cc:654] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.275816  9766 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.286628  9764 tablet_bootstrap.cc:654] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.291986  9766 raft_consensus.cc:2749] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:45.295584  9723 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.297250  9614 leader_election.cc:304] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.298194  9763 tablet_bootstrap.cc:654] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:45.298281  9769 raft_consensus.cc:2804] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.299177  9769 raft_consensus.cc:697] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.300366  9769 consensus_queue.cc:237] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.305465  9766 raft_consensus.cc:493] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.301246  9574 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.306464  9766 raft_consensus.cc:515] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.310216  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" is_pre_election: true
I20260709 13:15:45.311472  9649 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:45.313270  9539 leader_election.cc:304] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.319072  9809 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:15:45.327409  9470 catalog_manager.cc:5697] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.314498  9808 raft_consensus.cc:2804] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.332749  9767 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: No bootstrap required, opened a new log
I20260709 13:15:45.335877  9808 raft_consensus.cc:493] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.336336  9767 ts_tablet_manager.cc:1403] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: Time spent bootstrapping tablet: real 0.089s	user 0.013s	sys 0.008s
I20260709 13:15:45.336578  9808 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.325426  9472 catalog_manager.cc:5697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.340122  9769 raft_consensus.cc:493] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.340652  9769 raft_consensus.cc:515] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.343617  9769 leader_election.cc:290] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.344669  9766 leader_election.cc:290] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.345180  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:45.345256  9767 raft_consensus.cc:359] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.346019  9574 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.347960  9614 leader_election.cc:304] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.348752  9721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.349550  9721 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 0.
I20260709 13:15:45.348783  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.350983  9722 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.346095  9767 raft_consensus.cc:385] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.351598  9764 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: No bootstrap required, opened a new log
I20260709 13:15:45.351521  9767 raft_consensus.cc:740] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.352262  9764 ts_tablet_manager.cc:1403] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.023s
I20260709 13:15:45.352573  9767 consensus_queue.cc:260] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.354938  9808 raft_consensus.cc:515] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.355278  9764 raft_consensus.cc:359] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.356071  9764 raft_consensus.cc:385] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.355688  9767 ts_tablet_manager.cc:1434] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 13:15:45.356426  9764 raft_consensus.cc:740] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.358234  9648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.358880  9648 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.359226  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.359813  9722 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.360422  9763 tablet_bootstrap.cc:492] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: No bootstrap required, opened a new log
I20260709 13:15:45.360924  9763 ts_tablet_manager.cc:1403] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: Time spent bootstrapping tablet: real 0.114s	user 0.012s	sys 0.000s
I20260709 13:15:45.363710  9763 raft_consensus.cc:359] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.364434  9763 raft_consensus.cc:385] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:45.364723  9763 raft_consensus.cc:740] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Initialized, Role: FOLLOWER
I20260709 13:15:45.365542  9763 consensus_queue.cc:260] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.357456  9764 consensus_queue.cc:260] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.368549  9808 leader_election.cc:290] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.368009  9722 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.368453  9648 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 1.
I20260709 13:15:45.370798  9764 ts_tablet_manager.cc:1434] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:15:45.370913  9539 leader_election.cc:304] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.369632  9763 ts_tablet_manager.cc:1434] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:15:45.372192  9808 raft_consensus.cc:2804] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.373124  9808 raft_consensus.cc:697] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.374687  9808 consensus_queue.cc:237] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.349568  9769 raft_consensus.cc:2804] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.376693  9769 raft_consensus.cc:493] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.377322  9769 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.391916  9769 raft_consensus.cc:515] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.396693  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.397395  9574 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.398397  9769 leader_election.cc:290] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.399286  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.400053  9723 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.397392  9471 catalog_manager.cc:5697] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.406430  9574 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.407725  9723 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.407832  9614 leader_election.cc:304] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.409009  9769 raft_consensus.cc:2804] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.409940  9769 raft_consensus.cc:697] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.410908  9769 consensus_queue.cc:237] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.413875  9472 catalog_manager.cc:5697] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.421496  9470 catalog_manager.cc:5697] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.423137  9808 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.441658  9809 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.470000  9794 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.480516  9807 raft_consensus.cc:493] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.481148  9807 raft_consensus.cc:515] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.483518  9807 leader_election.cc:290] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.484911  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b0dca477714bdcba7d1a1966d41bbd" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e" is_pre_election: true
I20260709 13:15:45.485327  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b0dca477714bdcba7d1a1966d41bbd" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228" is_pre_election: true
I20260709 13:15:45.485677  9723 raft_consensus.cc:2468] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.485965  9574 raft_consensus.cc:2468] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 0.
I20260709 13:15:45.485545  9779 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.487300  9614 leader_election.cc:304] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.488773  9807 raft_consensus.cc:2804] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:45.489873  9807 raft_consensus.cc:493] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.490408  9807 raft_consensus.cc:3060] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.498359  9807 raft_consensus.cc:515] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.500669  9807 leader_election.cc:290] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.501840  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b0dca477714bdcba7d1a1966d41bbd" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.502053  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5b0dca477714bdcba7d1a1966d41bbd" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.502652  9574 raft_consensus.cc:3060] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.502743  9723 raft_consensus.cc:3060] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:45.509521  9574 raft_consensus.cc:2468] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.510143  9723 raft_consensus.cc:2468] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 1.
I20260709 13:15:45.510917  9614 leader_election.cc:304] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:45.511958  9807 raft_consensus.cc:2804] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:45.512517  9807 raft_consensus.cc:697] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.513351  9807 consensus_queue.cc:237] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.522631  9470 catalog_manager.cc:5697] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.554319  9809 consensus_queue.cc:1048] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.568104  9783 consensus_queue.cc:1048] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.678762  9648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:39448
I20260709 13:15:45.679661  9648 raft_consensus.cc:606] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:45.683552  9807 consensus_queue.cc:1048] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.688675  9648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:39448
I20260709 13:15:45.689302  9648 raft_consensus.cc:606] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:45.696913  9769 raft_consensus.cc:993] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.698269  9769 raft_consensus.cc:1081] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.702658  9574 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59402
I20260709 13:15:45.703433  9574 raft_consensus.cc:493] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:45.703917  9574 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.706121  9783 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.709982  9769 consensus_queue.cc:1048] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.713332  9723 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "792521f85edc44319aa7bf189556921e"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:53178
I20260709 13:15:45.712435  9574 raft_consensus.cc:515] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.714023  9723 raft_consensus.cc:606] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:45.722224  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.724972  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ef9f7c1827047c489b4990aa24cbbb4" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.732161  9649 raft_consensus.cc:3055] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.732539  9649 raft_consensus.cc:740] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.715818  9574 leader_election.cc:290] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.733558  9649 consensus_queue.cc:260] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.754019  9649 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.759268  9783 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.759979  9574 raft_consensus.cc:1240] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Rejecting Update request from peer 4c2174747b544c0da2b7e6c1a32bbcb0 for earlier term 1. Current term is 2. Ops: []
I20260709 13:15:45.766193  9649 raft_consensus.cc:2468] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:45.771164  9539 leader_election.cc:304] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 792521f85edc44319aa7bf189556921e
I20260709 13:15:45.772665  9783 raft_consensus.cc:2804] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:45.773454  9783 raft_consensus.cc:697] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.774416  9783 consensus_queue.cc:237] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.780540  9809 consensus_queue.cc:1048] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.786151  9574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:59426
I20260709 13:15:45.786927  9574 raft_consensus.cc:606] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Received request to transfer leadership to TS 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:45.790148  9472 catalog_manager.cc:5697] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 1 to 2, leader changed from 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.795699  9574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:59426
I20260709 13:15:45.796449  9574 raft_consensus.cc:606] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Received request to transfer leadership to TS 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:45.811493  9783 consensus_queue.cc:1048] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.811826  9574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:59426
I20260709 13:15:45.812450  9574 raft_consensus.cc:606] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Received request to transfer leadership to TS 792521f85edc44319aa7bf189556921e
I20260709 13:15:45.814858  9807 consensus_queue.cc:1048] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:45.829281  9766 raft_consensus.cc:993] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:45.829844  9766 raft_consensus.cc:1081] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Signalling peer 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:45.831391  9769 consensus_queue.cc:1048] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:45.832115  9723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442"
dest_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:53166
I20260709 13:15:45.832840  9723 raft_consensus.cc:493] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:45.833228  9723 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.841674  9723 raft_consensus.cc:515] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.844966  9769 raft_consensus.cc:993] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.845507  9769 raft_consensus.cc:1081] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.847847  9573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59402
I20260709 13:15:45.848394  9574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.848631  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c9ac1b66b34d879376b4b86cc87442" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.848542  9573 raft_consensus.cc:493] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:45.849123  9574 raft_consensus.cc:3055] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.849391  9649 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.849576  9574 raft_consensus.cc:740] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.849445  9573 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.850767  9574 consensus_queue.cc:260] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.852632  9574 raft_consensus.cc:3060] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.857615  9723 leader_election.cc:290] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 2 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:45.858057  9649 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 2.
I20260709 13:15:45.857945  9814 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:45.859848  9688 leader_election.cc:304] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.861143  9832 raft_consensus.cc:2804] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:45.861719  9832 raft_consensus.cc:697] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:45.862674  9832 consensus_queue.cc:237] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.863610  9574 raft_consensus.cc:2468] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 2.
I20260709 13:15:45.864068  9573 raft_consensus.cc:515] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.869024  9814 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:45.873154  9472 catalog_manager.cc:5697] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 1 to 2, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 2 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.875164  9779 raft_consensus.cc:993] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.875972  9573 leader_election.cc:290] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.875973  9832 raft_consensus.cc:1081] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:45.880054  9573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59410
I20260709 13:15:45.880764  9573 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:45.881259  9573 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.876798  9723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.884857  9723 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.890417  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.891435  9723 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:45.892598  9539 leader_election.cc:304] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.891183  9649 raft_consensus.cc:3055] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.893603  9783 raft_consensus.cc:2804] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:45.894229  9783 raft_consensus.cc:697] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.893560  9649 raft_consensus.cc:740] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:45.895212  9783 consensus_queue.cc:237] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.895826  9649 consensus_queue.cc:260] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.896745  9573 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.898947  9573 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.901151  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.901871  9722 raft_consensus.cc:3055] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.902377  9722 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:45.903522  9722 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.904924  9722 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.901925  9649 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.905042  9648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:45.911835  9648 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.914048  9649 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:45.914350  9472 catalog_manager.cc:5697] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 1 to 2, leader changed from 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.917856  9722 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:45.920429  9648 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:45.921093  9539 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:45.925295  9783 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:45.926031  9783 raft_consensus.cc:697] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.927340  9783 consensus_queue.cc:237] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.936143  9471 catalog_manager.cc:5697] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 1 to 2, leader changed from 792521f85edc44319aa7bf189556921e (127.8.183.67) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.982563  9809 raft_consensus.cc:993] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:45.983218  9766 raft_consensus.cc:1081] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Signalling peer 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:45.985352  9648 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:39434
I20260709 13:15:45.986007  9648 raft_consensus.cc:493] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:45.986418  9648 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.993222  9648 raft_consensus.cc:515] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.995280  9648 leader_election.cc:290] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:45.996582  9573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:45.996975  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:45.997335  9573 raft_consensus.cc:3055] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:45.997609  9722 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:45.997761  9573 raft_consensus.cc:740] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:45.998701  9573 consensus_queue.cc:260] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:45.999814  9573 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.003979  9722 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:46.005498  9614 leader_election.cc:304] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:46.005558  9573 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:46.006492  9841 raft_consensus.cc:2804] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:46.007042  9841 raft_consensus.cc:697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:46.007929  9841 consensus_queue.cc:237] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:46.015407  9471 catalog_manager.cc:5697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 1 to 2, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.113224  9841 consensus_queue.cc:1048] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:46.126353  9841 consensus_queue.cc:1048] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:46.278425  9648 raft_consensus.cc:1275] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.280287  9808 consensus_queue.cc:1048] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:46.290320  9721 raft_consensus.cc:3060] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.298900  9721 raft_consensus.cc:1275] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.300755  9783 consensus_queue.cc:1048] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.340749  9648 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.342641  9783 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.353518  9721 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.356623  9648 raft_consensus.cc:1275] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.357486  9809 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:46.358424  9832 consensus_queue.cc:1048] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.367959  9573 raft_consensus.cc:1275] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.371295  9779 consensus_queue.cc:1048] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:46.375031  9809 raft_consensus.cc:993] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:46.375608  9809 raft_consensus.cc:1081] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Signalling peer 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:46.377556  9648 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:39434
I20260709 13:15:46.378214  9648 raft_consensus.cc:493] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:46.378643  9648 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.386737  9648 raft_consensus.cc:515] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:46.388916  9648 leader_election.cc:290] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:46.389819  9573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:46.390226  9721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:46.390609  9573 raft_consensus.cc:3055] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:46.390955  9721 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.391095  9573 raft_consensus.cc:740] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:46.392271  9573 consensus_queue.cc:260] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:46.394148  9573 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:46.398250  9721 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:46.400920  9614 leader_election.cc:304] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:46.401440  9573 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:46.402083  9841 raft_consensus.cc:2804] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:46.402704  9841 raft_consensus.cc:697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:46.403530  9841 consensus_queue.cc:237] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:46.411368  9472 catalog_manager.cc:5697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 1 to 2, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.421205  9648 raft_consensus.cc:1275] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.422905  9809 consensus_queue.cc:1048] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:46.432444  9721 raft_consensus.cc:1275] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.434633  9766 consensus_queue.cc:1048] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.466255  9573 raft_consensus.cc:1275] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.468281  9841 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:46.479084  9721 raft_consensus.cc:1275] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.480836  9807 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.831346  9573 raft_consensus.cc:1275] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.833002  9769 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:46.838521  9721 raft_consensus.cc:1275] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:46.840039  9769 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:48.856724  8925 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:15:48.857662  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:48.879014  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:48.879827  9572 raft_consensus.cc:606] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Received request to transfer leadership to TS 792521f85edc44319aa7bf189556921e
I20260709 13:15:48.883179  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:48.883788  9572 raft_consensus.cc:606] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Received request to transfer leadership to TS 792521f85edc44319aa7bf189556921e
I20260709 13:15:48.885154  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:48.885929  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:49.049419  9860 raft_consensus.cc:993] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:49.049968  9808 raft_consensus.cc:1081] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Signalling peer 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:49.052004  9722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:53166
I20260709 13:15:49.052687  9722 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.053145  9722 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.059383  9722 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.061601  9722 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 3 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:49.063668  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:49.065554  9572 raft_consensus.cc:3055] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:49.065907  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:49.066116  9572 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:49.066573  9649 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.067453  9572 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.070967  9572 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:49.074723  9649 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 3.
I20260709 13:15:49.076244  9688 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:49.077239  9832 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:49.078020  9572 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 3.
I20260709 13:15:49.078871  9832 raft_consensus.cc:697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:49.079913  9832 consensus_queue.cc:237] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.088707  9472 catalog_manager.cc:5697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 2 to 3, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 3 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.193336  9860 raft_consensus.cc:993] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:49.193871  9808 raft_consensus.cc:1081] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Signalling peer 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:49.196192  9722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f"
dest_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:53166
I20260709 13:15:49.196983  9722 raft_consensus.cc:493] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:49.197366  9722 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.203059  9722 raft_consensus.cc:515] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.205125  9722 leader_election.cc:290] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 2 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:49.206368  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:49.206802  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:49.207072  9572 raft_consensus.cc:3055] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:49.207540  9649 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.207587  9572 raft_consensus.cc:740] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:49.208673  9572 consensus_queue.cc:260] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.209858  9572 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:49.213951  9649 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 2.
I20260709 13:15:49.215826  9688 leader_election.cc:304] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:49.216178  9572 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 2.
I20260709 13:15:49.216881  9832 raft_consensus.cc:2804] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:49.217406  9832 raft_consensus.cc:697] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:49.218313  9832 consensus_queue.cc:237] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:49.227869  9472 catalog_manager.cc:5697] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 1 to 2, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 2 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:49.471870  9572 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:49.473831  9832 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:49.488929  9649 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:49.490998  9873 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:49.755237  9572 raft_consensus.cc:1275] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:49.756881  9873 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:49.768051  9649 raft_consensus.cc:1275] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:49.770133  9863 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:49.887109  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:49.961781  9722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "792521f85edc44319aa7bf189556921e"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:53178
I20260709 13:15:49.962793  9722 raft_consensus.cc:606] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:49.970582  9722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f"
dest_uuid: "792521f85edc44319aa7bf189556921e"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:53178
I20260709 13:15:49.971283  9722 raft_consensus.cc:606] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:49.982295  9649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:39448
I20260709 13:15:49.982972  9649 raft_consensus.cc:606] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:49.990109  9649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:39448
I20260709 13:15:49.990723  9649 raft_consensus.cc:606] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:49.998730  9879 raft_consensus.cc:993] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:49.998602  9649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
mode: GRACEFUL
new_leader_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:39448
I20260709 13:15:49.999473  9862 raft_consensus.cc:1081] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:49.999490  9649 raft_consensus.cc:606] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Received request to transfer leadership to TS 5e6a8136a1f848c5aceed687ee884228
I20260709 13:15:50.002197  9572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59402
I20260709 13:15:50.003011  9572 raft_consensus.cc:493] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:50.003628  9572 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.012863  9572 raft_consensus.cc:515] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.015039  9572 leader_election.cc:290] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:50.016531  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:50.016857  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6910d75b0be4ff8b6a12f32f2cff6a1" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:50.017387  9649 raft_consensus.cc:3055] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:50.017707  9722 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.017825  9649 raft_consensus.cc:740] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:50.019037  9649 consensus_queue.cc:260] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.020470  9649 raft_consensus.cc:3060] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.024760  9722 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.026429  9539 leader_election.cc:304] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:50.027751  9649 raft_consensus.cc:2468] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.027999  9808 raft_consensus.cc:2804] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:50.029948  9808 raft_consensus.cc:697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:50.031797  9808 consensus_queue.cc:237] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.046641  9472 catalog_manager.cc:5697] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 2 to 3, leader changed from 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.052992  9832 raft_consensus.cc:993] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.053723  9832 raft_consensus.cc:1081] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.056242  9572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59410
I20260709 13:15:50.057076  9572 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:50.057690  9572 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.066233  9572 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.069435  9572 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 4 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:50.071091  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:50.071522  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:50.071942  9722 raft_consensus.cc:3055] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:50.072285  9649 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.072410  9722 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:50.073851  9722 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.075654  9722 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:50.078383  9649 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 4.
I20260709 13:15:50.079690  9539 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:50.080945  9808 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:50.081485  9808 raft_consensus.cc:697] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:50.081931  9722 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 4.
I20260709 13:15:50.083048  9808 consensus_queue.cc:237] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.091444  9472 catalog_manager.cc:5697] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 3 to 4, leader changed from 792521f85edc44319aa7bf189556921e (127.8.183.67) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 4 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.161304  9879 raft_consensus.cc:993] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.161849  9862 raft_consensus.cc:1081] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.163707  9572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59402
I20260709 13:15:50.164384  9572 raft_consensus.cc:493] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:50.164731  9572 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.171499  9572 raft_consensus.cc:515] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.173504  9572 leader_election.cc:290] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:50.174494  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:50.174762  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:50.175209  9649 raft_consensus.cc:3055] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:50.175339  9722 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.175618  9649 raft_consensus.cc:740] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:50.176592  9649 consensus_queue.cc:260] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.177917  9649 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:50.183614  9722 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:50.185201  9649 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 2.
I20260709 13:15:50.185189  9539 leader_election.cc:304] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:50.186462  9808 raft_consensus.cc:2804] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:50.187004  9808 raft_consensus.cc:697] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:50.188023  9808 consensus_queue.cc:237] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.196790  9472 catalog_manager.cc:5697] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 1 to 2, leader changed from 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.251328  9879 raft_consensus.cc:993] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.251940  9862 raft_consensus.cc:1081] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.254101  9572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59402
I20260709 13:15:50.254721  9572 raft_consensus.cc:493] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:50.255131  9572 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.262327  9572 raft_consensus.cc:515] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.264706  9572 leader_election.cc:290] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:50.266718  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:50.266819  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d57583a0d6b4393bd5150e4c28b21e0" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:50.267391  9722 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.267454  9649 raft_consensus.cc:3055] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:50.268106  9649 raft_consensus.cc:740] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:50.269132  9649 consensus_queue.cc:260] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.270458  9649 raft_consensus.cc:3060] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.274425  9722 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.275981  9539 leader_election.cc:304] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6a8136a1f848c5aceed687ee884228, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:50.276597  9649 raft_consensus.cc:2468] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.276880  9808 raft_consensus.cc:2804] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:50.277583  9808 raft_consensus.cc:697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:50.278621  9808 consensus_queue.cc:237] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.286595  9472 catalog_manager.cc:5697] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 2 to 3, leader changed from 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.374238  9863 raft_consensus.cc:993] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: : Instructing follower 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.374892  9873 raft_consensus.cc:1081] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Signalling peer 5e6a8136a1f848c5aceed687ee884228 to start an election
I20260709 13:15:50.376821  9572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
 from {username='slave'} at 127.0.0.1:59410
I20260709 13:15:50.377465  9572 raft_consensus.cc:493] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:50.377766  9572 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.384434  9572 raft_consensus.cc:515] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.386531  9572 leader_election.cc:290] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Requested vote from peers 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:50.388052  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:50.388043  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31d1925dbd7a4b018a9b098740dbd71f" candidate_uuid: "5e6a8136a1f848c5aceed687ee884228" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:50.388919  9649 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.389015  9722 raft_consensus.cc:3055] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:50.389518  9722 raft_consensus.cc:740] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:50.390532  9722 consensus_queue.cc:260] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.391851  9722 raft_consensus.cc:3060] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:50.394861  9649 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.396610  9539 leader_election.cc:304] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 5e6a8136a1f848c5aceed687ee884228; no voters: 
I20260709 13:15:50.397876  9881 raft_consensus.cc:2804] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:50.398494  9881 raft_consensus.cc:697] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Becoming Leader. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:50.399643  9722 raft_consensus.cc:2468] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6a8136a1f848c5aceed687ee884228 in term 3.
I20260709 13:15:50.399643  9881 consensus_queue.cc:237] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:50.407696  9472 catalog_manager.cc:5697] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 reported cstate change: term changed from 2 to 3, leader changed from 792521f85edc44319aa7bf189556921e (127.8.183.67) to 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "5e6a8136a1f848c5aceed687ee884228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:50.448096  9649 raft_consensus.cc:1275] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.449582  9881 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:50.464286  9722 raft_consensus.cc:1275] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.465893  9808 consensus_queue.cc:1048] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:50.479190  9722 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 4 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:50.481002  9808 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:50.494638  9649 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 4 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:15:50.496647  9894 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:50.703938  9722 raft_consensus.cc:1275] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.705438  9894 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:50.715961  9649 raft_consensus.cc:1275] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:50.717742  9808 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:50.875622  9649 raft_consensus.cc:1275] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.877218  9808 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:50.888689  9722 raft_consensus.cc:1275] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.890899  9894 consensus_queue.cc:1048] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:50.959583  9649 raft_consensus.cc:1275] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.961562  9808 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:50.973676  9722 raft_consensus.cc:1275] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Refusing update from remote peer 5e6a8136a1f848c5aceed687ee884228: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:50.975975  9808 consensus_queue.cc:1048] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.011453  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:53.020162  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:53.020762  9572 raft_consensus.cc:606] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Received request to transfer leadership to TS 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:53.024178  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:53.024744  9572 raft_consensus.cc:606] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Received request to transfer leadership to TS 4c2174747b544c0da2b7e6c1a32bbcb0
I20260709 13:15:53.027643  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:53.028230  9572 raft_consensus.cc:606] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Received request to transfer leadership to TS 792521f85edc44319aa7bf189556921e
I20260709 13:15:53.030843  9572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
mode: GRACEFUL
new_leader_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:59438
I20260709 13:15:53.031535  9572 raft_consensus.cc:606] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 LEADER]: Received request to transfer leadership to TS 792521f85edc44319aa7bf189556921e
I20260709 13:15:53.032734  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:15:53.033192  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:53.220590  9902 raft_consensus.cc:993] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:53.221163  9808 raft_consensus.cc:1081] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Signalling peer 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:53.223012  9649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:39434
I20260709 13:15:53.223716  9649 raft_consensus.cc:493] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.224053  9649 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.230026  9649 raft_consensus.cc:515] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.232512  9649 leader_election.cc:290] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:53.233574  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:53.234026  9721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b0db9e1138e4ea6b777713504155a37" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:53.234182  9902 raft_consensus.cc:993] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:53.234434  9572 raft_consensus.cc:3055] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:15:53.234699  9721 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.234825  9808 raft_consensus.cc:1081] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Signalling peer 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:53.234948  9572 raft_consensus.cc:740] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:53.236068  9572 consensus_queue.cc:260] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.236896  9722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d"
dest_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:53166
I20260709 13:15:53.237293  9572 raft_consensus.cc:3060] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:15:53.237495  9722 raft_consensus.cc:493] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.237893  9722 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.241228  9721 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:53.242636  9614 leader_election.cc:304] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:53.243456  9862 raft_consensus.cc:2804] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:15:53.244573  9862 raft_consensus.cc:697] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:53.244845  9572 raft_consensus.cc:2468] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 2.
I20260709 13:15:53.244867  9722 raft_consensus.cc:515] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.245635  9862 consensus_queue.cc:237] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.247260  9722 leader_election.cc:290] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 3 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:53.248901  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:53.249500  9649 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.253048  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d1939727ba9472ba9c1325959a0062d" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:53.253947  9572 raft_consensus.cc:3055] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:53.254339  9572 raft_consensus.cc:740] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:53.255261  9572 consensus_queue.cc:260] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.256116  9649 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 3.
I20260709 13:15:53.255853  9472 catalog_manager.cc:5697] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 1 to 2, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.256693  9572 raft_consensus.cc:3060] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.257629  9688 leader_election.cc:304] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:53.258420  9873 raft_consensus.cc:2804] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:53.259616  9873 raft_consensus.cc:697] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:53.260500  9873 consensus_queue.cc:237] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.265609  9572 raft_consensus.cc:2468] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 3.
I20260709 13:15:53.270198  9471 catalog_manager.cc:5697] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 2 to 3, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 3 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:53.322872  9902 raft_consensus.cc:993] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:53.323552  9808 raft_consensus.cc:1081] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 LEADER]: Signalling peer 792521f85edc44319aa7bf189556921e to start an election
I20260709 13:15:53.325265  9722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721"
dest_uuid: "792521f85edc44319aa7bf189556921e"
 from {username='slave'} at 127.0.0.1:53166
I20260709 13:15:53.325843  9722 raft_consensus.cc:493] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.326174  9722 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.332791  9722 raft_consensus.cc:515] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.334456  9722 leader_election.cc:290] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 5 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66:46145)
I20260709 13:15:53.335961  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:53.336290  9649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d21e8f4a2af74ed4a4c460ce24457721" candidate_uuid: "792521f85edc44319aa7bf189556921e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
I20260709 13:15:53.336894  9572 raft_consensus.cc:3055] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:53.337060  9649 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.337497  9572 raft_consensus.cc:740] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:53.338594  9572 consensus_queue.cc:260] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.339936  9572 raft_consensus.cc:3060] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:53.344266  9649 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 5.
I20260709 13:15:53.345660  9572 raft_consensus.cc:2468] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 792521f85edc44319aa7bf189556921e in term 5.
I20260709 13:15:53.345742  9688 leader_election.cc:304] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:53.346903  9873 raft_consensus.cc:2804] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:53.347550  9873 raft_consensus.cc:697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 5 LEADER]: Becoming Leader. State: Replica: 792521f85edc44319aa7bf189556921e, State: Running, Role: LEADER
I20260709 13:15:53.348438  9873 consensus_queue.cc:237] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.356829  9471 catalog_manager.cc:5697] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e reported cstate change: term changed from 4 to 5, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 792521f85edc44319aa7bf189556921e (127.8.183.67). New cstate: current_term: 5 leader_uuid: "792521f85edc44319aa7bf189556921e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:53.572062  9902 raft_consensus.cc:993] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: : Instructing follower 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:53.572647  9808 raft_consensus.cc:1081] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Signalling peer 4c2174747b544c0da2b7e6c1a32bbcb0 to start an election
I20260709 13:15:53.574522  9649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be"
dest_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0"
 from {username='slave'} at 127.0.0.1:39434
I20260709 13:15:53.575043  9649 raft_consensus.cc:493] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:53.575359  9649 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.580590  9649 raft_consensus.cc:515] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.582222  9649 leader_election.cc:290] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 3 election: Requested vote from peers 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977), 792521f85edc44319aa7bf189556921e (127.8.183.67:34753)
I20260709 13:15:53.583360  9572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e6a8136a1f848c5aceed687ee884228"
I20260709 13:15:53.583562  9722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bc2a2983fc4a01b8c7d361a9c820be" candidate_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "792521f85edc44319aa7bf189556921e"
I20260709 13:15:53.584254  9722 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.584201  9572 raft_consensus.cc:3055] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:15:53.584775  9572 raft_consensus.cc:740] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e6a8136a1f848c5aceed687ee884228, State: Running, Role: LEADER
I20260709 13:15:53.585641  9572 consensus_queue.cc:260] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.586897  9572 raft_consensus.cc:3060] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:15:53.592144  9722 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 3.
I20260709 13:15:53.593680  9614 leader_election.cc:304] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2174747b544c0da2b7e6c1a32bbcb0, 792521f85edc44319aa7bf189556921e; no voters: 
I20260709 13:15:53.594379  9572 raft_consensus.cc:2468] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c2174747b544c0da2b7e6c1a32bbcb0 in term 3.
I20260709 13:15:53.594851  9862 raft_consensus.cc:2804] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:15:53.595439  9862 raft_consensus.cc:697] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 LEADER]: Becoming Leader. State: Replica: 4c2174747b544c0da2b7e6c1a32bbcb0, State: Running, Role: LEADER
I20260709 13:15:53.596388  9862 consensus_queue.cc:237] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } }
I20260709 13:15:53.603732  9471 catalog_manager.cc:5697] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 reported cstate change: term changed from 2 to 3, leader changed from 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65) to 4c2174747b544c0da2b7e6c1a32bbcb0 (127.8.183.66). New cstate: current_term: 3 leader_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:53.674899  9649 raft_consensus.cc:1275] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.676471  9873 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:53.689985  9572 raft_consensus.cc:1275] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:53.695556  9905 consensus_queue.cc:1048] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:53.772279  9572 raft_consensus.cc:1275] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.773962  9722 raft_consensus.cc:1275] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:15:53.773975  9862 consensus_queue.cc:1048] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:53.775544  9903 consensus_queue.cc:1048] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:53.934312  9572 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 5 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:53.936023  9916 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6a8136a1f848c5aceed687ee884228" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 44977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:53.948465  9648 raft_consensus.cc:1275] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 5 FOLLOWER]: Refusing update from remote peer 792521f85edc44319aa7bf189556921e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:15:53.950443  9923 consensus_queue.cc:1048] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2174747b544c0da2b7e6c1a32bbcb0" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:54.034142  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:54.038728  9722 raft_consensus.cc:1275] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Refusing update from remote peer 4c2174747b544c0da2b7e6c1a32bbcb0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:15:54.040596  9921 consensus_queue.cc:1048] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "792521f85edc44319aa7bf189556921e" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 34753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:54.043619  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
W20260709 13:15:54.049939  9614 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:15:54.071013  9614 consensus_peers.cc:597] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 -> Peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Couldn't send request to peer 5e6a8136a1f848c5aceed687ee884228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:54.100942  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:54.101979  8925 tablet_replica.cc:333] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.103037  8925 raft_consensus.cc:2243] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.103706  8925 raft_consensus.cc:2272] T 3d1939727ba9472ba9c1325959a0062d P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.106294  8925 tablet_replica.cc:333] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.106900  8925 raft_consensus.cc:2243] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.107478  8925 raft_consensus.cc:2272] T d21e8f4a2af74ed4a4c460ce24457721 P 5e6a8136a1f848c5aceed687ee884228 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.110976  8925 tablet_replica.cc:333] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.111768  8925 raft_consensus.cc:2243] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.112308  8925 raft_consensus.cc:2272] T 22bc2a2983fc4a01b8c7d361a9c820be P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.114532  8925 tablet_replica.cc:333] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.115329  8925 raft_consensus.cc:2243] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.115892  8925 raft_consensus.cc:2272] T c8c9ac1b66b34d879376b4b86cc87442 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.118139  8925 tablet_replica.cc:333] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.118781  8925 raft_consensus.cc:2243] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.119393  8925 raft_consensus.cc:2272] T f5b0dca477714bdcba7d1a1966d41bbd P 5e6a8136a1f848c5aceed687ee884228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.121289  8925 tablet_replica.cc:333] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.121857  8925 raft_consensus.cc:2243] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.122320  8925 raft_consensus.cc:2272] T 2b0db9e1138e4ea6b777713504155a37 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.124775  8925 tablet_replica.cc:333] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.125408  8925 raft_consensus.cc:2243] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.126248  8925 raft_consensus.cc:2272] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.128259  8925 tablet_replica.cc:333] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.128876  8925 raft_consensus.cc:2243] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.129756  8925 raft_consensus.cc:2272] T 7d57583a0d6b4393bd5150e4c28b21e0 P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.132112  8925 tablet_replica.cc:333] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.132843  8925 raft_consensus.cc:2243] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.133808  8925 raft_consensus.cc:2272] T 31d1925dbd7a4b018a9b098740dbd71f P 5e6a8136a1f848c5aceed687ee884228 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.136240  8925 tablet_replica.cc:333] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228: stopping tablet replica
I20260709 13:15:54.136837  8925 raft_consensus.cc:2243] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.137755  8925 raft_consensus.cc:2272] T 6ef9f7c1827047c489b4990aa24cbbb4 P 5e6a8136a1f848c5aceed687ee884228 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:54.155459  9614 consensus_peers.cc:597] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 -> Peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Couldn't send request to peer 5e6a8136a1f848c5aceed687ee884228. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:54.158993  9688 connection.cc:573] client connection to 127.8.183.65:44977 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:15:54.168025  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:15:54.169523  9688 consensus_peers.cc:597] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e -> Peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Couldn't send request to peer 5e6a8136a1f848c5aceed687ee884228. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:44977: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:54.173309  9688 consensus_peers.cc:597] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e -> Peer 5e6a8136a1f848c5aceed687ee884228 (127.8.183.65:44977): Couldn't send request to peer 5e6a8136a1f848c5aceed687ee884228. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:44977: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:54.192539  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
I20260709 13:15:54.228338  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:54.229342  8925 tablet_replica.cc:333] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.230149  8925 raft_consensus.cc:2243] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.230762  8925 raft_consensus.cc:2272] T 7d57583a0d6b4393bd5150e4c28b21e0 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.233109  8925 tablet_replica.cc:333] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.233815  8925 raft_consensus.cc:2243] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.234812  8925 raft_consensus.cc:2272] T f5b0dca477714bdcba7d1a1966d41bbd P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.237115  8925 tablet_replica.cc:333] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.237785  8925 raft_consensus.cc:2243] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.238730  8925 raft_consensus.cc:2272] T 2b0db9e1138e4ea6b777713504155a37 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.241308  8925 tablet_replica.cc:333] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.241953  8925 raft_consensus.cc:2243] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.242497  8925 raft_consensus.cc:2272] T c8c9ac1b66b34d879376b4b86cc87442 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.244696  8925 tablet_replica.cc:333] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.245292  8925 raft_consensus.cc:2243] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.246189  8925 raft_consensus.cc:2272] T 22bc2a2983fc4a01b8c7d361a9c820be P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.249217  8925 tablet_replica.cc:333] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.249819  8925 raft_consensus.cc:2243] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.250298  8925 raft_consensus.cc:2272] T d21e8f4a2af74ed4a4c460ce24457721 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.253080  8925 tablet_replica.cc:333] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.253826  8925 raft_consensus.cc:2243] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.254387  8925 raft_consensus.cc:2272] T 6ef9f7c1827047c489b4990aa24cbbb4 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.256282  8925 tablet_replica.cc:333] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.256819  8925 raft_consensus.cc:2243] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.257283  8925 raft_consensus.cc:2272] T 3d1939727ba9472ba9c1325959a0062d P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.259804  8925 tablet_replica.cc:333] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.260357  8925 raft_consensus.cc:2243] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.260835  8925 raft_consensus.cc:2272] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.262634  8925 tablet_replica.cc:333] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0: stopping tablet replica
I20260709 13:15:54.263209  8925 raft_consensus.cc:2243] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.263691  8925 raft_consensus.cc:2272] T 31d1925dbd7a4b018a9b098740dbd71f P 4c2174747b544c0da2b7e6c1a32bbcb0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.296875  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
I20260709 13:15:54.318539  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
I20260709 13:15:54.353332  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:54.354125  8925 tablet_replica.cc:333] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.354830  8925 raft_consensus.cc:2243] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.355412  8925 raft_consensus.cc:2272] T f6910d75b0be4ff8b6a12f32f2cff6a1 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.357385  8925 tablet_replica.cc:333] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.358008  8925 raft_consensus.cc:2243] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.358880  8925 raft_consensus.cc:2272] T 3d1939727ba9472ba9c1325959a0062d P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.361790  8925 tablet_replica.cc:333] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.362524  8925 raft_consensus.cc:2243] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.363077  8925 raft_consensus.cc:2272] T 6ef9f7c1827047c489b4990aa24cbbb4 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.365175  8925 tablet_replica.cc:333] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.365816  8925 raft_consensus.cc:2243] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.366317  8925 raft_consensus.cc:2272] T f5b0dca477714bdcba7d1a1966d41bbd P 792521f85edc44319aa7bf189556921e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.368278  8925 tablet_replica.cc:333] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.368935  8925 raft_consensus.cc:2243] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.369503  8925 raft_consensus.cc:2272] T 2b0db9e1138e4ea6b777713504155a37 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.372574  8925 tablet_replica.cc:333] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.373281  8925 raft_consensus.cc:2243] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.374280  8925 raft_consensus.cc:2272] T c8c9ac1b66b34d879376b4b86cc87442 P 792521f85edc44319aa7bf189556921e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.376389  8925 tablet_replica.cc:333] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.377034  8925 raft_consensus.cc:2243] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.377573  8925 raft_consensus.cc:2272] T 22bc2a2983fc4a01b8c7d361a9c820be P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.380584  8925 tablet_replica.cc:333] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.381189  8925 raft_consensus.cc:2243] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.382048  8925 raft_consensus.cc:2272] T d21e8f4a2af74ed4a4c460ce24457721 P 792521f85edc44319aa7bf189556921e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.384363  8925 tablet_replica.cc:333] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.385044  8925 raft_consensus.cc:2243] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.385623  8925 raft_consensus.cc:2272] T 7d57583a0d6b4393bd5150e4c28b21e0 P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.387658  8925 tablet_replica.cc:333] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e: stopping tablet replica
I20260709 13:15:54.388230  8925 raft_consensus.cc:2243] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:54.388733  8925 raft_consensus.cc:2272] T 31d1925dbd7a4b018a9b098740dbd71f P 792521f85edc44319aa7bf189556921e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.421945  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:15:54.447467  8925 master.cc:562] Master@127.8.183.126:42731 shutting down...
I20260709 13:15:54.475338  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:15:54.475976  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:54.476344  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3a208c10e923497893da9946ddfe2b5e: stopping tablet replica
I20260709 13:15:54.498992  8925 master.cc:584] Master@127.8.183.126:42731 shutdown complete.
4c2174747b544c0da2b7e6c1a32bbcb0  4
5e6a8136a1f848c5aceed687ee884228  5
792521f85edc44319aa7bf189556921e  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10927 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:15:54.550256  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:40341
I20260709 13:15:54.551829  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:54.558585  9930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.558633  9929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.559077  9932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:54.559908  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:54.560834  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:54.561038  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:54.561249  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602954561215 us; error 0 us; skew 500 ppm
I20260709 13:15:54.561841  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:54.564985  8925 webserver.cc:533] Webserver started at http://127.8.183.126:34021/ using document root <none> and password file <none>
I20260709 13:15:54.565641  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:54.565887  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:54.566474  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:54.568099  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "db0c98fae3aa4b45a39c8914c5e641cc"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:54.573818  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.005s
I20260709 13:15:54.578015  9937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:54.579196  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:15:54.579511  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "db0c98fae3aa4b45a39c8914c5e641cc"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:54.579828  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:54.605280  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:54.606865  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:54.659029  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:40341
I20260709 13:15:54.659220  9988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:40341 every 8 connection(s)
I20260709 13:15:54.663573  9989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:54.674255  9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Bootstrap starting.
I20260709 13:15:54.679283  9989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:54.684116  9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: No bootstrap required, opened a new log
I20260709 13:15:54.686556  9989 raft_consensus.cc:359] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER }
I20260709 13:15:54.687004  9989 raft_consensus.cc:385] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:54.687233  9989 raft_consensus.cc:740] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0c98fae3aa4b45a39c8914c5e641cc, State: Initialized, Role: FOLLOWER
I20260709 13:15:54.687790  9989 consensus_queue.cc:260] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER }
I20260709 13:15:54.688305  9989 raft_consensus.cc:399] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:15:54.688519  9989 raft_consensus.cc:493] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:15:54.688746  9989 raft_consensus.cc:3060] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:54.693243  9989 raft_consensus.cc:515] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER }
I20260709 13:15:54.693818  9989 leader_election.cc:304] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db0c98fae3aa4b45a39c8914c5e641cc; no voters: 
I20260709 13:15:54.695187  9989 leader_election.cc:290] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:15:54.695559  9992 raft_consensus.cc:2804] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:54.696933  9992 raft_consensus.cc:697] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 1 LEADER]: Becoming Leader. State: Replica: db0c98fae3aa4b45a39c8914c5e641cc, State: Running, Role: LEADER
I20260709 13:15:54.697654  9992 consensus_queue.cc:237] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER }
I20260709 13:15:54.698376  9989 sys_catalog.cc:565] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:15:54.704171  9994 sys_catalog.cc:455] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader db0c98fae3aa4b45a39c8914c5e641cc. Latest consensus state: current_term: 1 leader_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER } }
I20260709 13:15:54.704896  9993 sys_catalog.cc:455] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db0c98fae3aa4b45a39c8914c5e641cc" member_type: VOTER } }
I20260709 13:15:54.705217  9994 sys_catalog.cc:458] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:54.705451  9993 sys_catalog.cc:458] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [sys.catalog]: This master's current role is: LEADER
I20260709 13:15:54.708783 10000 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:15:54.716405 10000 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:15:54.717536  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:15:54.727816 10000 catalog_manager.cc:1383] Generated new cluster ID: cf961a758002484ba765ef048643ef97
I20260709 13:15:54.728235 10000 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:15:54.759699 10000 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:15:54.761152 10000 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:15:54.792119 10000 catalog_manager.cc:6070] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Generated new TSK 0
I20260709 13:15:54.792838 10000 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:15:54.849452  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:54.856772 10010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.858559 10011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.860584 10013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:54.865527  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:54.866593  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:54.866865  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:54.867055  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602954867031 us; error 0 us; skew 500 ppm
I20260709 13:15:54.868014  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:54.871603  8925 webserver.cc:533] Webserver started at http://127.8.183.65:37953/ using document root <none> and password file <none>
I20260709 13:15:54.872391  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:54.872634  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:54.872972  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:54.874249  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "01bfaf1ad56a4deea886bbc64dec466c"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:54.879626  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:15:54.884191 10018 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:54.885401  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:15:54.885860  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "01bfaf1ad56a4deea886bbc64dec466c"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:54.886234  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:54.901867  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:54.903422  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:54.905220  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:54.908118  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:54.908448  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:54.908751  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:54.908970  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:54.958611  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:43347
I20260709 13:15:54.958774 10080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:43347 every 8 connection(s)
I20260709 13:15:54.964978  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:54.973809 10085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.974087 10086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:54.979523 10088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:54.979763  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:54.981165  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:54.981594  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:54.981900  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602954981870 us; error 0 us; skew 500 ppm
I20260709 13:15:54.982649  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:54.986639  8925 webserver.cc:533] Webserver started at http://127.8.183.66:43681/ using document root <none> and password file <none>
I20260709 13:15:54.986781 10081 heartbeater.cc:344] Connected to a master server at 127.8.183.126:40341
I20260709 13:15:54.987442  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:54.987437 10081 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:54.987841  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:54.988286  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:54.988914 10081 heartbeater.cc:507] Master 127.8.183.126:40341 requested a full tablet report, sending...
I20260709 13:15:54.989688  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "e9a849ac7eda426abfca97cf8b15da0f"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:54.991647  9954 ts_manager.cc:194] Registered new tserver with Master: 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:15:54.993643  9954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47224
I20260709 13:15:54.995481  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:15:54.999963 10093 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.001209  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:15:55.001614  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "e9a849ac7eda426abfca97cf8b15da0f"
format_stamp: "Formatted at 2026-07-09 13:15:54 on dist-test-slave-ch3x"
I20260709 13:15:55.002034  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:55.033505  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:55.034940  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:55.036850  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:55.039646  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:55.039954  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.040266  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:55.040472  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.093086  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:41681
I20260709 13:15:55.093220 10155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:41681 every 8 connection(s)
I20260709 13:15:55.099174  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:15:55.111347 10159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:15:55.112712 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:55.116303 10156 heartbeater.cc:344] Connected to a master server at 127.8.183.126:40341
I20260709 13:15:55.116937 10156 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:55.117846 10156 heartbeater.cc:507] Master 127.8.183.126:40341 requested a full tablet report, sending...
W20260709 13:15:55.119355 10162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:15:55.120426  9954 ts_manager.cc:194] Registered new tserver with Master: e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:55.121588  8925 server_base.cc:1061] running on GCE node
I20260709 13:15:55.122267  9954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47236
I20260709 13:15:55.122767  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:15:55.123054  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:15:55.123323  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602955123303 us; error 0 us; skew 500 ppm
I20260709 13:15:55.124140  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:15:55.127305  8925 webserver.cc:533] Webserver started at http://127.8.183.67:38761/ using document root <none> and password file <none>
I20260709 13:15:55.127877  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:15:55.128062  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:15:55.128446  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:15:55.129664  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "d51ad94710c348889d417f7f00032948"
format_stamp: "Formatted at 2026-07-09 13:15:55 on dist-test-slave-ch3x"
I20260709 13:15:55.135043  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:15:55.139288 10167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.140247  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:15:55.140695  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "d51ad94710c348889d417f7f00032948"
format_stamp: "Formatted at 2026-07-09 13:15:55 on dist-test-slave-ch3x"
I20260709 13:15:55.141074  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:15:55.153861  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:15:55.155448  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:15:55.157160  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:15:55.159978  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:15:55.160274  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.160545  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:15:55.160741  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:15:55.209651  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:40505
I20260709 13:15:55.209792 10229 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:40505 every 8 connection(s)
I20260709 13:15:55.227173 10230 heartbeater.cc:344] Connected to a master server at 127.8.183.126:40341
I20260709 13:15:55.227659 10230 heartbeater.cc:461] Registering TS with master...
I20260709 13:15:55.228484 10230 heartbeater.cc:507] Master 127.8.183.126:40341 requested a full tablet report, sending...
I20260709 13:15:55.230988  9954 ts_manager.cc:194] Registered new tserver with Master: d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:55.231981  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018314681s
I20260709 13:15:55.232702  9954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47238
I20260709 13:15:55.263873  9954 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47242:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:15:55.271163  9954 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:15:55.514889 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 75ddb7f5e3204a51b3dafe4903888dcc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:55.516714 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ddb7f5e3204a51b3dafe4903888dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.516402 10120 tablet_service.cc:1511] Processing CreateTablet for tablet dfacd39092d24ba5a33d0cd9451b91a1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:55.518087 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfacd39092d24ba5a33d0cd9451b91a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.520586 10045 tablet_service.cc:1511] Processing CreateTablet for tablet dfacd39092d24ba5a33d0cd9451b91a1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:55.522231 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfacd39092d24ba5a33d0cd9451b91a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.523995 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 75ddb7f5e3204a51b3dafe4903888dcc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:55.524223 10194 tablet_service.cc:1511] Processing CreateTablet for tablet dfacd39092d24ba5a33d0cd9451b91a1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:15:55.524767 10044 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6c92980e764b60a913b8c5533ed9ad (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:55.525869 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ddb7f5e3204a51b3dafe4903888dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.526125 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6c92980e764b60a913b8c5533ed9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.526048 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfacd39092d24ba5a33d0cd9451b91a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.528537 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6c92980e764b60a913b8c5533ed9ad (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:55.530139 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6c92980e764b60a913b8c5533ed9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.532572 10117 tablet_service.cc:1511] Processing CreateTablet for tablet a353f267acea40eaaf03befcb32892f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:55.532744 10193 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6c92980e764b60a913b8c5533ed9ad (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:15:55.534309 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a353f267acea40eaaf03befcb32892f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.534759 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6c92980e764b60a913b8c5533ed9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.532747 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 75ddb7f5e3204a51b3dafe4903888dcc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:15:55.536454 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 806f35e66c234d569cbafd8b3981518f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:55.538044 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806f35e66c234d569cbafd8b3981518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.537915 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 806f35e66c234d569cbafd8b3981518f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:55.540834 10042 tablet_service.cc:1511] Processing CreateTablet for tablet a353f267acea40eaaf03befcb32892f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:55.541551 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ddb7f5e3204a51b3dafe4903888dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.541797 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 764b8122cc0a4ccc97816d729adafabe (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:55.542363 10116 tablet_service.cc:1511] Processing CreateTablet for tablet bfb08df714384523b94354944a523e54 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:55.542815 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a353f267acea40eaaf03befcb32892f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.544006 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb08df714384523b94354944a523e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.544693 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 2869566cb59547689f49f6e04511ed8a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:55.546312 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2869566cb59547689f49f6e04511ed8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.547885 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806f35e66c234d569cbafd8b3981518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.548892 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 764b8122cc0a4ccc97816d729adafabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.552883 10041 tablet_service.cc:1511] Processing CreateTablet for tablet bfb08df714384523b94354944a523e54 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:55.554783 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb08df714384523b94354944a523e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.557188 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 764b8122cc0a4ccc97816d729adafabe (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:55.562930 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 764b8122cc0a4ccc97816d729adafabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.567906 10192 tablet_service.cc:1511] Processing CreateTablet for tablet 806f35e66c234d569cbafd8b3981518f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:15:55.569085 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 2219e1af873d4a00966a8c5975eb7987 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:55.570263 10039 tablet_service.cc:1511] Processing CreateTablet for tablet 2869566cb59547689f49f6e04511ed8a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:55.570858 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219e1af873d4a00966a8c5975eb7987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.572860 10191 tablet_service.cc:1511] Processing CreateTablet for tablet a353f267acea40eaaf03befcb32892f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:15:55.569701 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806f35e66c234d569cbafd8b3981518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.576447 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 764b8122cc0a4ccc97816d729adafabe (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:15:55.578224 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 764b8122cc0a4ccc97816d729adafabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.580513 10188 tablet_service.cc:1511] Processing CreateTablet for tablet 2869566cb59547689f49f6e04511ed8a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:15:55.582373 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2869566cb59547689f49f6e04511ed8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.583902 10190 tablet_service.cc:1511] Processing CreateTablet for tablet bfb08df714384523b94354944a523e54 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:15:55.585592 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb08df714384523b94354944a523e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.587929 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2869566cb59547689f49f6e04511ed8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.591637 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a353f267acea40eaaf03befcb32892f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.595192 10112 tablet_service.cc:1511] Processing CreateTablet for tablet 371654d6e0a94cba825fa080a9998940 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:55.597622 10112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371654d6e0a94cba825fa080a9998940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.601209 10186 tablet_service.cc:1511] Processing CreateTablet for tablet 371654d6e0a94cba825fa080a9998940 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:55.604552 10187 tablet_service.cc:1511] Processing CreateTablet for tablet 2219e1af873d4a00966a8c5975eb7987 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:55.605974 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371654d6e0a94cba825fa080a9998940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.613984 10038 tablet_service.cc:1511] Processing CreateTablet for tablet 2219e1af873d4a00966a8c5975eb7987 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:15:55.617007 10037 tablet_service.cc:1511] Processing CreateTablet for tablet 371654d6e0a94cba825fa080a9998940 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:15:55.630239 10038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219e1af873d4a00966a8c5975eb7987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.633260 10037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 371654d6e0a94cba825fa080a9998940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.635492 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2219e1af873d4a00966a8c5975eb7987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.676667 10120 tablet_service.cc:1511] Processing CreateTablet for tablet b09e10904af344988a1a11ae9e576799 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:55.678506 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09e10904af344988a1a11ae9e576799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.704912 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd90bb718ac49bf84e09fe79d462240 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:55.709571 10186 tablet_service.cc:1511] Processing CreateTablet for tablet b09e10904af344988a1a11ae9e576799 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
W20260709 13:15:55.712208 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:55.713161 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09e10904af344988a1a11ae9e576799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.720011 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd90bb718ac49bf84e09fe79d462240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.670848 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 98f63a37309c453f9fd7bc3f75659536 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:55.730173 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f63a37309c453f9fd7bc3f75659536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.738428 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 98f63a37309c453f9fd7bc3f75659536 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:55.738428 10192 tablet_service.cc:1511] Processing CreateTablet for tablet 98f63a37309c453f9fd7bc3f75659536 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:15:55.745340 10248 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:55.751209 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd90bb718ac49bf84e09fe79d462240 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:55.750938 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 1c7c855af77d40ab995a18b5609de8f5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:55.755648 10249 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:55.752702 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f63a37309c453f9fd7bc3f75659536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.753806 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c7c855af77d40ab995a18b5609de8f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.766058 10115 tablet_service.cc:1511] Processing CreateTablet for tablet c3664f114a52481c8f97e06e9b2d2068 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:55.768203 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3664f114a52481c8f97e06e9b2d2068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.773440 10118 tablet_service.cc:1511] Processing CreateTablet for tablet cf88b6d5c6504b3cb2cbd21cff68e57e (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:55.775098 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf88b6d5c6504b3cb2cbd21cff68e57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.775337 10250 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:55.793579 10248 tablet_bootstrap.cc:654] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:55.797410 10046 tablet_service.cc:1511] Processing CreateTablet for tablet b09e10904af344988a1a11ae9e576799 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:15:55.801222 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 1c7c855af77d40ab995a18b5609de8f5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:55.801271 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd90bb718ac49bf84e09fe79d462240 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:15:55.804993 10121 tablet_service.cc:1511] Processing CreateTablet for tablet f3220b9037e44cae95e49af43de3523a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:55.818310 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 2188d855354542c984f249b94175c660 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:55.801743 10191 tablet_service.cc:1511] Processing CreateTablet for tablet c3664f114a52481c8f97e06e9b2d2068 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:55.825702 10045 tablet_service.cc:1511] Processing CreateTablet for tablet c3664f114a52481c8f97e06e9b2d2068 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:15:55.827443 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b09e10904af344988a1a11ae9e576799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.828869 10116 tablet_service.cc:1511] Processing CreateTablet for tablet e14489a089594151b58748865ea36e81 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:55.830265 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd90bb718ac49bf84e09fe79d462240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.830922 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 2188d855354542c984f249b94175c660 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:55.831140 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3220b9037e44cae95e49af43de3523a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.833232 10044 tablet_service.cc:1511] Processing CreateTablet for tablet cf88b6d5c6504b3cb2cbd21cff68e57e (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:55.834704 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf88b6d5c6504b3cb2cbd21cff68e57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.835286 10193 tablet_service.cc:1511] Processing CreateTablet for tablet e14489a089594151b58748865ea36e81 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:55.839221 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3664f114a52481c8f97e06e9b2d2068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.840894 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188d855354542c984f249b94175c660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.843178 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f63a37309c453f9fd7bc3f75659536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.844051 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd90bb718ac49bf84e09fe79d462240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.844795 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3664f114a52481c8f97e06e9b2d2068. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.845530 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c7c855af77d40ab995a18b5609de8f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.804447 10188 tablet_service.cc:1511] Processing CreateTablet for tablet cf88b6d5c6504b3cb2cbd21cff68e57e (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:15:55.811977 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 1c7c855af77d40ab995a18b5609de8f5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:15:55.847340 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf88b6d5c6504b3cb2cbd21cff68e57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.847889 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c7c855af77d40ab995a18b5609de8f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.813443 10187 tablet_service.cc:1511] Processing CreateTablet for tablet f3220b9037e44cae95e49af43de3523a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:55.850970 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3220b9037e44cae95e49af43de3523a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.858064 10119 tablet_service.cc:1511] Processing CreateTablet for tablet aca8a04bf8df4212bfdf5fd990503c0a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:55.858729 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14489a089594151b58748865ea36e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.845486 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14489a089594151b58748865ea36e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.877983 10195 tablet_service.cc:1511] Processing CreateTablet for tablet aca8a04bf8df4212bfdf5fd990503c0a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:55.888494 10250 tablet_bootstrap.cc:654] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:55.895938 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188d855354542c984f249b94175c660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.923032 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca8a04bf8df4212bfdf5fd990503c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.899896 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca8a04bf8df4212bfdf5fd990503c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.927878 10248 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:55.928660 10248 ts_tablet_manager.cc:1403] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.184s	user 0.011s	sys 0.004s
I20260709 13:15:55.930353 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 2188d855354542c984f249b94175c660 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:15:55.937987 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188d855354542c984f249b94175c660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.936408 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 79954d719dc94974b4864ff715e9ef07 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:55.941988 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79954d719dc94974b4864ff715e9ef07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.942199 10249 tablet_bootstrap.cc:654] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:55.948472 10248 raft_consensus.cc:359] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:55.949419 10248 raft_consensus.cc:385] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:55.949735 10248 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:55.952000 10041 tablet_service.cc:1511] Processing CreateTablet for tablet aca8a04bf8df4212bfdf5fd990503c0a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:15:55.953727 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aca8a04bf8df4212bfdf5fd990503c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.940512 10038 tablet_service.cc:1511] Processing CreateTablet for tablet e14489a089594151b58748865ea36e81 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:15:55.934780 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 79954d719dc94974b4864ff715e9ef07 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:55.956086 10038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e14489a089594151b58748865ea36e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.956162 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79954d719dc94974b4864ff715e9ef07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.919523 10039 tablet_service.cc:1511] Processing CreateTablet for tablet f3220b9037e44cae95e49af43de3523a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:15:55.955559 10248 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:55.960928 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3220b9037e44cae95e49af43de3523a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:55.977154 10112 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce48110a5fb4ffc97e81117165e3ffd (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:56.005456 10250 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:56.005095 10037 tablet_service.cc:1511] Processing CreateTablet for tablet 79954d719dc94974b4864ff715e9ef07 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:15:56.006131 10250 ts_tablet_manager.cc:1403] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.253s	user 0.013s	sys 0.002s
I20260709 13:15:55.972369 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecb570a023c42ada8943a1516f3ef4f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:56.008210 10081 heartbeater.cc:499] Master 127.8.183.126:40341 was elected leader, sending a full tablet report...
I20260709 13:15:56.013675 10112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce48110a5fb4ffc97e81117165e3ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.017510 10037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79954d719dc94974b4864ff715e9ef07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.033995 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 73fb38c54abe4cd4bb552eb544020f47 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:56.049340 10254 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.049525 10156 heartbeater.cc:499] Master 127.8.183.126:40341 was elected leader, sending a full tablet report...
I20260709 13:15:56.056408 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:56.058609 10186 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce48110a5fb4ffc97e81117165e3ffd (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:56.060273 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce48110a5fb4ffc97e81117165e3ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.065459 10250 raft_consensus.cc:359] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.066411 10250 raft_consensus.cc:385] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.066234 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecb570a023c42ada8943a1516f3ef4f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:56.074030 10248 ts_tablet_manager.cc:1434] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.145s	user 0.020s	sys 0.033s
I20260709 13:15:56.075381 10248 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:56.045290 10192 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecb570a023c42ada8943a1516f3ef4f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:15:56.080511 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecb570a023c42ada8943a1516f3ef4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.066862 10250 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.072608 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecb570a023c42ada8943a1516f3ef4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.081645 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecb570a023c42ada8943a1516f3ef4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.083074 10254 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.099481 10250 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.103365 10188 tablet_service.cc:1511] Processing CreateTablet for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:56.105587 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.106415 10249 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:56.107024 10249 ts_tablet_manager.cc:1403] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.354s	user 0.010s	sys 0.020s
I20260709 13:15:56.108556 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fb38c54abe4cd4bb552eb544020f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.116441 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.087191 10120 tablet_service.cc:1511] Processing CreateTablet for tablet dc97aed5eb284c0e86c74b20d18b4399 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:56.120172 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc97aed5eb284c0e86c74b20d18b4399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.130908 10187 tablet_service.cc:1511] Processing CreateTablet for tablet dc97aed5eb284c0e86c74b20d18b4399 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:56.132684 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc97aed5eb284c0e86c74b20d18b4399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.137837 10248 tablet_bootstrap.cc:654] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.143837 10250 ts_tablet_manager.cc:1434] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.006s
I20260709 13:15:56.145156 10250 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:56.151839 10249 raft_consensus.cc:359] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.152801 10249 raft_consensus.cc:385] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.153160 10249 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.163043 10116 tablet_service.cc:1511] Processing CreateTablet for tablet 3173f90d855c4371aeafe17b5f760205 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:56.163503 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 3173f90d855c4371aeafe17b5f760205 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:56.164755 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3173f90d855c4371aeafe17b5f760205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.165238 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3173f90d855c4371aeafe17b5f760205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.166500 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:15:56.167995 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.168519 10121 tablet_service.cc:1511] Processing CreateTablet for tablet a0c93dea5060480eb0618f168078f4e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:56.192603 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c93dea5060480eb0618f168078f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.172552 10045 tablet_service.cc:1511] Processing CreateTablet for tablet dc97aed5eb284c0e86c74b20d18b4399 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:15:56.174158 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 3173f90d855c4371aeafe17b5f760205 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:15:56.174536 10191 tablet_service.cc:1511] Processing CreateTablet for tablet a0c93dea5060480eb0618f168078f4e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:56.168452 10193 tablet_service.cc:1511] Processing CreateTablet for tablet 73fb38c54abe4cd4bb552eb544020f47 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:56.177999 10250 tablet_bootstrap.cc:654] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.178606 10119 tablet_service.cc:1511] Processing CreateTablet for tablet d97ed0ec5f214d95abb9e01d368a04b5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:56.182230 10044 tablet_service.cc:1511] Processing CreateTablet for tablet a0c93dea5060480eb0618f168078f4e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:15:56.183624 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce48110a5fb4ffc97e81117165e3ffd (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:15:56.154060 10249 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.188985 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 73fb38c54abe4cd4bb552eb544020f47 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:15:56.189179 10190 tablet_service.cc:1511] Processing CreateTablet for tablet d97ed0ec5f214d95abb9e01d368a04b5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:56.223883 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97ed0ec5f214d95abb9e01d368a04b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.225353 10256 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.208458 10254 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:56.227499 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 986d51d657bc4fac8f4199fbc38600c1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:56.198148 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 986d51d657bc4fac8f4199fbc38600c1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:56.257180 10256 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.249778 10230 heartbeater.cc:499] Master 127.8.183.126:40341 was elected leader, sending a full tablet report...
I20260709 13:15:56.265728 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc97aed5eb284c0e86c74b20d18b4399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.266636 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fb38c54abe4cd4bb552eb544020f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.273568 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3173f90d855c4371aeafe17b5f760205. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.286425 10256 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:56.301292 10193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fb38c54abe4cd4bb552eb544020f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.302196 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d51d657bc4fac8f4199fbc38600c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.303757 10248 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:56.304373 10248 ts_tablet_manager.cc:1403] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.229s	user 0.016s	sys 0.022s
I20260709 13:15:56.311535 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97ed0ec5f214d95abb9e01d368a04b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.312450 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce48110a5fb4ffc97e81117165e3ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.315182 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d51d657bc4fac8f4199fbc38600c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.315708 10118 tablet_service.cc:1511] Processing CreateTablet for tablet c502c18433fb46e2ac2a42d7d8746ce0 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:56.317167 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c502c18433fb46e2ac2a42d7d8746ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.318492 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c93dea5060480eb0618f168078f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.319901 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 2c485d13f66d4ce8bdca72b60d242842 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:56.321692 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c485d13f66d4ce8bdca72b60d242842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.323926 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0c93dea5060480eb0618f168078f4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.341354 10187 tablet_service.cc:1511] Processing CreateTablet for tablet 2c485d13f66d4ce8bdca72b60d242842 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:56.344033 10039 tablet_service.cc:1511] Processing CreateTablet for tablet d97ed0ec5f214d95abb9e01d368a04b5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:15:56.345757 10192 tablet_service.cc:1511] Processing CreateTablet for tablet c502c18433fb46e2ac2a42d7d8746ce0 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:56.346810 10248 raft_consensus.cc:359] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.349479 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 82960478409442549781547254b9d8ea (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:56.352172 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c485d13f66d4ce8bdca72b60d242842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.364838 10248 raft_consensus.cc:385] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.375147 10248 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.376261 10248 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.378764 10248 ts_tablet_manager.cc:1434] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 13:15:56.386284 10250 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:56.389226 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 82960478409442549781547254b9d8ea (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:56.330564 10037 tablet_service.cc:1511] Processing CreateTablet for tablet 986d51d657bc4fac8f4199fbc38600c1 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:15:56.393985 10038 tablet_service.cc:1511] Processing CreateTablet for tablet 2c485d13f66d4ce8bdca72b60d242842 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:15:56.419507 10114 tablet_service.cc:1511] Processing CreateTablet for tablet 161f2d2f2674494bbffbd5ee927cf5e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:56.403100 10250 ts_tablet_manager.cc:1403] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.258s	user 0.008s	sys 0.011s
I20260709 13:15:56.425401 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82960478409442549781547254b9d8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.435294 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c502c18433fb46e2ac2a42d7d8746ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.435433 10044 tablet_service.cc:1511] Processing CreateTablet for tablet c502c18433fb46e2ac2a42d7d8746ce0 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:15:56.436388 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:56.440106 10095 leader_election.cc:343] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.443205 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97ed0ec5f214d95abb9e01d368a04b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.444125 10038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c485d13f66d4ce8bdca72b60d242842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.447542 10037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d51d657bc4fac8f4199fbc38600c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.453501 10248 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:56.454965 10186 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7190358a9f475d82e5f9875677bccf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:56.456390 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7190358a9f475d82e5f9875677bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.465288 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:15:56.467394 10096 leader_election.cc:343] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.467978 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:56.468899 10254 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:56.470158 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7190358a9f475d82e5f9875677bccf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:56.471074 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 82960478409442549781547254b9d8ea (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:15:56.471900 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7190358a9f475d82e5f9875677bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.472172 10188 tablet_service.cc:1511] Processing CreateTablet for tablet 161f2d2f2674494bbffbd5ee927cf5e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:56.473359 10249 ts_tablet_manager.cc:1434] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.364s	user 0.011s	sys 0.023s
I20260709 13:15:56.474555 10249 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:56.475390 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161f2d2f2674494bbffbd5ee927cf5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.475683 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161f2d2f2674494bbffbd5ee927cf5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.479514 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82960478409442549781547254b9d8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.492067 10116 tablet_service.cc:1511] Processing CreateTablet for tablet a169a099da7f404bb7804561c7aca3e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:56.494644 10250 raft_consensus.cc:359] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.500077 10248 tablet_bootstrap.cc:654] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.502734 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:56.512362 10020 leader_election.cc:343] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.502998 10192 tablet_service.cc:1511] Processing CreateTablet for tablet a169a099da7f404bb7804561c7aca3e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:56.503635 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c502c18433fb46e2ac2a42d7d8746ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.509570 10250 raft_consensus.cc:385] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.519747 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:56.529366 10019 leader_election.cc:343] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.530066 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:56.531350 10256 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:56.531595 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82960478409442549781547254b9d8ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.532464 10250 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.533577 10250 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.536676 10250 ts_tablet_manager.cc:1434] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.115s	user 0.006s	sys 0.002s
I20260709 13:15:56.536454 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 161f2d2f2674494bbffbd5ee927cf5e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:15:56.537864 10250 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:56.538334 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 161f2d2f2674494bbffbd5ee927cf5e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.545495 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169a099da7f404bb7804561c7aca3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.538336 10249 tablet_bootstrap.cc:654] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.553742 10191 tablet_service.cc:1511] Processing CreateTablet for tablet ee79852a0bc04642b58b690930f78267 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:56.555727 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee79852a0bc04642b58b690930f78267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.556126 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169a099da7f404bb7804561c7aca3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.559069 10039 tablet_service.cc:1511] Processing CreateTablet for tablet a169a099da7f404bb7804561c7aca3e6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:15:56.560376 10254 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.568364 10254 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.570746 10254 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:56.534864 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 6a7190358a9f475d82e5f9875677bccf (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:15:56.555758 10121 tablet_service.cc:1511] Processing CreateTablet for tablet ee79852a0bc04642b58b690930f78267 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:56.580382 10265 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.581243 10265 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.590013 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7190358a9f475d82e5f9875677bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.589782 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:56.591643 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee79852a0bc04642b58b690930f78267. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:56.606148 10095 leader_election.cc:343] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.614042 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 5391e406039d4af8bf2b17f1b2034ef5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:56.615463 10265 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:56.615875 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5391e406039d4af8bf2b17f1b2034ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.625959 10189 tablet_service.cc:1511] Processing CreateTablet for tablet dbf6e6a0da3d4501acf0e5451cae4577 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:56.621511 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:56.628409 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 5391e406039d4af8bf2b17f1b2034ef5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:56.629971 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5391e406039d4af8bf2b17f1b2034ef5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:56.630007 10096 leader_election.cc:343] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.630719 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:56.632468 10112 tablet_service.cc:1511] Processing CreateTablet for tablet dbf6e6a0da3d4501acf0e5451cae4577 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:56.634111 10112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf6e6a0da3d4501acf0e5451cae4577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.634395 10254 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:56.636895 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf6e6a0da3d4501acf0e5451cae4577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.644377 10038 tablet_service.cc:1511] Processing CreateTablet for tablet ee79852a0bc04642b58b690930f78267 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:15:56.648021 10256 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.648659 10256 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.650084 10250 tablet_bootstrap.cc:654] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.651050 10256 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:56.652988 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:56.654886 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:56.655663 10131 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
W20260709 13:15:56.657092 10020 leader_election.cc:343] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.658354 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:56.659582 10256 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:56.659981 10256 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.660356 10256 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:56.662092 10044 tablet_service.cc:1511] Processing CreateTablet for tablet dbf6e6a0da3d4501acf0e5451cae4577 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:15:56.663225 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169a099da7f404bb7804561c7aca3e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.655678 10043 tablet_service.cc:1511] Processing CreateTablet for tablet 5391e406039d4af8bf2b17f1b2034ef5 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:15:56.668203 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf6e6a0da3d4501acf0e5451cae4577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.668834 10249 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:56.669503 10249 ts_tablet_manager.cc:1403] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.195s	user 0.008s	sys 0.005s
I20260709 13:15:56.681144 10249 raft_consensus.cc:359] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.682118 10249 raft_consensus.cc:385] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.678624 10256 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.682477 10249 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.683840 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5391e406039d4af8bf2b17f1b2034ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.685267 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:56.686220 10131 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:56.687052 10040 tablet_service.cc:1511] Processing CreateTablet for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:56.688717 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.694046 10038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee79852a0bc04642b58b690930f78267. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.696754 10248 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:56.697405 10248 ts_tablet_manager.cc:1403] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.271s	user 0.015s	sys 0.003s
I20260709 13:15:56.700911 10248 raft_consensus.cc:359] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.701776 10248 raft_consensus.cc:385] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.702136 10248 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.708024 10248 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.709514 10118 tablet_service.cc:1511] Processing CreateTablet for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:56.711907 10191 tablet_service.cc:1511] Processing CreateTablet for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:15:56.713768 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.713879 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed07b7c3185a4d5d9fbcba31d80ed8f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.716496 10046 tablet_service.cc:1511] Processing CreateTablet for tablet f09ed53b9cff42a29c1d6c06e009ee94 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:15:56.717653 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:56.718176 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09ed53b9cff42a29c1d6c06e009ee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.719383 10249 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.722370 10249 ts_tablet_manager.cc:1434] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.000s
I20260709 13:15:56.722458 10131 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:56.723692 10249 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:56.724103 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:56.736714 10249 tablet_bootstrap.cc:654] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.728685 10248 ts_tablet_manager.cc:1434] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.002s
I20260709 13:15:56.746611 10248 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:56.754331 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:15:56.756486 10170 leader_election.cc:343] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.777674 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:15:56.779872 10020 leader_election.cc:343] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.780486 10119 tablet_service.cc:1511] Processing CreateTablet for tablet f09ed53b9cff42a29c1d6c06e009ee94 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:56.782023 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09ed53b9cff42a29c1d6c06e009ee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.792119 10042 tablet_service.cc:1511] Processing CreateTablet for tablet d07e888fde29492b84e575218308b6ff (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:56.792122 10194 tablet_service.cc:1511] Processing CreateTablet for tablet f09ed53b9cff42a29c1d6c06e009ee94 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:15:56.793917 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07e888fde29492b84e575218308b6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.795161 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09ed53b9cff42a29c1d6c06e009ee94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.799836 10188 tablet_service.cc:1511] Processing CreateTablet for tablet d07e888fde29492b84e575218308b6ff (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:56.800079 10115 tablet_service.cc:1511] Processing CreateTablet for tablet d07e888fde29492b84e575218308b6ff (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:15:56.801446 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07e888fde29492b84e575218308b6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.771778 10256 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:56.806794 10037 tablet_service.cc:1511] Processing CreateTablet for tablet d22230bc6afb42ceac52420e42db0292 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:56.807615 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d07e888fde29492b84e575218308b6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.810328 10120 tablet_service.cc:1511] Processing CreateTablet for tablet d22230bc6afb42ceac52420e42db0292 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:56.832665 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:56.834897 10168 leader_election.cc:343] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:56.835471 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:56.836376 10265 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:56.837821 10248 tablet_bootstrap.cc:654] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:56.838104 10187 tablet_service.cc:1511] Processing CreateTablet for tablet d22230bc6afb42ceac52420e42db0292 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:15:56.840145 10037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22230bc6afb42ceac52420e42db0292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.844120 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22230bc6afb42ceac52420e42db0292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.848210 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22230bc6afb42ceac52420e42db0292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.855980 10273 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:15:56.803388 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79350.1us
I20260709 13:15:56.872051 10045 tablet_service.cc:1511] Processing CreateTablet for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:56.873683 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a122c89ca68646b8bf88dbb1d5dcbfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.914538 10249 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:56.915246 10249 ts_tablet_manager.cc:1403] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.192s	user 0.029s	sys 0.036s
I20260709 13:15:56.918975 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 66968e5dbd3040d290ce9716c146d46a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:56.920890 10250 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:56.926422 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66968e5dbd3040d290ce9716c146d46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.920890 10273 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:56.936940 10273 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.920437 10112 tablet_service.cc:1511] Processing CreateTablet for tablet 66968e5dbd3040d290ce9716c146d46a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:56.940191 10250 ts_tablet_manager.cc:1403] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.397s	user 0.013s	sys 0.016s
I20260709 13:15:56.945577 10250 raft_consensus.cc:359] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.946475 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 5853a69cde724bc3a143613bf68a63fc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:56.948069 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5853a69cde724bc3a143613bf68a63fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.951804 10112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66968e5dbd3040d290ce9716c146d46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.946485 10250 raft_consensus.cc:385] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.959313 10250 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.960541 10250 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.963280 10250 ts_tablet_manager.cc:1434] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 13:15:56.958274 10191 tablet_service.cc:1511] Processing CreateTablet for tablet 66968e5dbd3040d290ce9716c146d46a (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:15:56.967159 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 5853a69cde724bc3a143613bf68a63fc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:56.967532 10039 tablet_service.cc:1511] Processing CreateTablet for tablet 9a895dc8dee14dfba216f443121474f4 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:56.968782 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 5853a69cde724bc3a143613bf68a63fc (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:15:56.956337 10192 tablet_service.cc:1511] Processing CreateTablet for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:56.979555 10121 tablet_service.cc:1511] Processing CreateTablet for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:15:56.984403 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a895dc8dee14dfba216f443121474f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:56.989239 10250 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:56.991596 10249 raft_consensus.cc:359] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.992506 10249 raft_consensus.cc:385] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:56.992870 10249 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:56.992007  9954 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:56.993964 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 9a895dc8dee14dfba216f443121474f4 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:56.998554 10041 tablet_service.cc:1511] Processing CreateTablet for tablet 2c048cf8265e4264b35912a8caac9936 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:57.000109 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c048cf8265e4264b35912a8caac9936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.000753 10248 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.002717 10248 ts_tablet_manager.cc:1403] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.256s	user 0.022s	sys 0.011s
I20260709 13:15:57.003001 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a122c89ca68646b8bf88dbb1d5dcbfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.003837 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a895dc8dee14dfba216f443121474f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.004951 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5853a69cde724bc3a143613bf68a63fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.006560 10248 raft_consensus.cc:359] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.007459 10248 raft_consensus.cc:385] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.007797 10248 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.008697 10248 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:56.983534 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 9a895dc8dee14dfba216f443121474f4 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:15:57.014884 10250 tablet_bootstrap.cc:654] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.016825 10116 tablet_service.cc:1511] Processing CreateTablet for tablet 2c048cf8265e4264b35912a8caac9936 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:56.994321 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5853a69cde724bc3a143613bf68a63fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.020798 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66968e5dbd3040d290ce9716c146d46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.022132 10248 ts_tablet_manager.cc:1434] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 13:15:57.027626 10194 tablet_service.cc:1511] Processing CreateTablet for tablet 2c048cf8265e4264b35912a8caac9936 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:15:57.035434 10194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c048cf8265e4264b35912a8caac9936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.021632 10249 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.039865 10249 ts_tablet_manager.cc:1434] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.124s	user 0.003s	sys 0.005s
I20260709 13:15:57.041123 10249 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.042786 10248 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.049103 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a122c89ca68646b8bf88dbb1d5dcbfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.059465 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a895dc8dee14dfba216f443121474f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.060778 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 87d36b9784d64ff48d0d4f0224266506 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:57.062388 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c048cf8265e4264b35912a8caac9936. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.063887 10120 tablet_service.cc:1511] Processing CreateTablet for tablet 87d36b9784d64ff48d0d4f0224266506 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:57.065387 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d36b9784d64ff48d0d4f0224266506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.068424 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d36b9784d64ff48d0d4f0224266506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.076186 10250 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.076846 10250 ts_tablet_manager.cc:1403] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.088s	user 0.008s	sys 0.008s
I20260709 13:15:57.079771 10273 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.080382 10273 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.085110 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.085901 10205 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:57.087245 10273 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.087096 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:57.090193 10019 leader_election.cc:343] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.091198 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:57.096227 10273 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:57.096875 10273 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.097381 10273 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.097262 10187 tablet_service.cc:1511] Processing CreateTablet for tablet 87d36b9784d64ff48d0d4f0224266506 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:15:57.099083 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d36b9784d64ff48d0d4f0224266506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.101109 10248 tablet_bootstrap.cc:654] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.115033 10249 tablet_bootstrap.cc:654] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.126874 10192 tablet_service.cc:1511] Processing CreateTablet for tablet cebbf225b866442ab895577b0496de1b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:57.128531 10192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cebbf225b866442ab895577b0496de1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.128228 10120 tablet_service.cc:1511] Processing CreateTablet for tablet 231b54e25cca40f6a3b0625d68e97cf7 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:57.129779 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231b54e25cca40f6a3b0625d68e97cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.131732 10039 tablet_service.cc:1511] Processing CreateTablet for tablet 231b54e25cca40f6a3b0625d68e97cf7 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:57.131814 10112 tablet_service.cc:1511] Processing CreateTablet for tablet cebbf225b866442ab895577b0496de1b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:57.133615 10112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cebbf225b866442ab895577b0496de1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.134652 10250 raft_consensus.cc:359] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.135566 10250 raft_consensus.cc:385] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.135910 10250 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.140734 10041 tablet_service.cc:1511] Processing CreateTablet for tablet cebbf225b866442ab895577b0496de1b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:15:57.142335 10041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cebbf225b866442ab895577b0496de1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.141919 10188 tablet_service.cc:1511] Processing CreateTablet for tablet 231b54e25cca40f6a3b0625d68e97cf7 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:15:57.147514 10188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231b54e25cca40f6a3b0625d68e97cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.147797 10254 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.148384 10254 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.150972 10254 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.151870 10039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 231b54e25cca40f6a3b0625d68e97cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.145509 10265 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.152578 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.137248 10250 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.154696 10265 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.157421 10250 ts_tablet_manager.cc:1434] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 13:15:57.153383 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 2169fe5ebfe5416c94b7d9095d9af6da (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:57.153851 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.159000 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:57.162474 10168 leader_election.cc:343] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.159065 10250 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:57.159415 10056 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
W20260709 13:15:57.162052 10095 leader_election.cc:343] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.171226 10273 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.160766 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.174556 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:57.176844 10254 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:57.177306 10254 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.177763 10254 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:57.179524 10170 leader_election.cc:343] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.180078 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:57.180960 10248 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.160871 10265 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:57.181501 10045 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81db48ec24878966db719b2790127 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:57.182415 10265 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.182034 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2169fe5ebfe5416c94b7d9095d9af6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.177374 10118 tablet_service.cc:1511] Processing CreateTablet for tablet 2169fe5ebfe5416c94b7d9095d9af6da (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:57.181622 10248 ts_tablet_manager.cc:1403] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.003s
I20260709 13:15:57.195093 10045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81db48ec24878966db719b2790127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.200439 10117 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81db48ec24878966db719b2790127 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:57.194319 10254 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.199554 10118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2169fe5ebfe5416c94b7d9095d9af6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.194499 10248 raft_consensus.cc:359] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.203651 10248 raft_consensus.cc:385] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.204105 10248 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.205570 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:57.205194 10248 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.206192 10273 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.205644 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:15:57.207815 10019 leader_election.cc:343] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.206903 10205 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.219671 10286 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.220876 10248 ts_tablet_manager.cc:1434] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.001s
I20260709 13:15:57.220546 10286 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.224536 10117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81db48ec24878966db719b2790127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.237952 10249 tablet_bootstrap.cc:492] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:57.251710 10249 ts_tablet_manager.cc:1403] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.211s	user 0.019s	sys 0.016s
I20260709 13:15:57.241384 10186 tablet_service.cc:1511] Processing CreateTablet for tablet 6278932947044bdbb7b0b1e14eea574b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:57.254038 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6278932947044bdbb7b0b1e14eea574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.243659 10042 tablet_service.cc:1511] Processing CreateTablet for tablet 6278932947044bdbb7b0b1e14eea574b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:57.255316 10249 raft_consensus.cc:359] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.256093 10042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6278932947044bdbb7b0b1e14eea574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.256163 10249 raft_consensus.cc:385] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.256619 10249 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.257426 10249 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.248255 10037 tablet_service.cc:1511] Processing CreateTablet for tablet 7a82842926e645c08dde847bfe182c31 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:57.215026 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 95b81db48ec24878966db719b2790127 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:15:57.250186 10119 tablet_service.cc:1511] Processing CreateTablet for tablet 6278932947044bdbb7b0b1e14eea574b (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:15:57.263487 10037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a82842926e645c08dde847bfe182c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.266472 10115 tablet_service.cc:1511] Processing CreateTablet for tablet 7a82842926e645c08dde847bfe182c31 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:57.269632 10205 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:57.273619 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 7a82842926e645c08dde847bfe182c31 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:15:57.277055 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:57.278025 10273 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:57.278604 10273 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:57.197961 10191 tablet_service.cc:1511] Processing CreateTablet for tablet 2169fe5ebfe5416c94b7d9095d9af6da (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:15:57.279577 10273 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.281068 10254 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.284376 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81db48ec24878966db719b2790127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.286860 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:57.303140 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:57.303474 10248 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.313745 10113 tablet_service.cc:1511] Processing CreateTablet for tablet 87da05694091470284f66a8181207e2f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:15:57.315099 10095 leader_election.cc:343] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.304733 10040 tablet_service.cc:1511] Processing CreateTablet for tablet 87da05694091470284f66a8181207e2f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:57.316226 10043 tablet_service.cc:1511] Processing CreateTablet for tablet c5af05c8079540ec84b0cc3dc1a9e051 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:57.317706 10040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87da05694091470284f66a8181207e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.308409 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a82842926e645c08dde847bfe182c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.304917 10056 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.328301 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a82842926e645c08dde847bfe182c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.331367 10286 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.304734 10190 tablet_service.cc:1511] Processing CreateTablet for tablet 87da05694091470284f66a8181207e2f (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:15:57.332716 10190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87da05694091470284f66a8181207e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.333271 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.333696 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2169fe5ebfe5416c94b7d9095d9af6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.337193 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:57.339035 10095 leader_election.cc:343] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:57.339670 10096 leader_election.cc:343] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.340225 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:57.343185 10114 tablet_service.cc:1511] Processing CreateTablet for tablet c5af05c8079540ec84b0cc3dc1a9e051 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:57.344395 10286 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.352429 10113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87da05694091470284f66a8181207e2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.355444 10043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5af05c8079540ec84b0cc3dc1a9e051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.361411 10250 tablet_bootstrap.cc:654] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.365953 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6278932947044bdbb7b0b1e14eea574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.297513 10254 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.388988 10114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5af05c8079540ec84b0cc3dc1a9e051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.389315 10254 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.396056 10249 ts_tablet_manager.cc:1434] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.144s	user 0.009s	sys 0.000s
I20260709 13:15:57.397452 10249 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.397487 10056 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:57.401088 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:57.402854 10286 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:57.403467 10286 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:15:57.300475 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120956us
I20260709 13:15:57.404435 10286 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.408696 10187 tablet_service.cc:1511] Processing CreateTablet for tablet c5af05c8079540ec84b0cc3dc1a9e051 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:15:57.410244 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.411012 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.410342 10187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5af05c8079540ec84b0cc3dc1a9e051. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:57.412143 10095 leader_election.cc:343] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:57.413076 10096 leader_election.cc:343] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.413172 10254 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.413619 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:57.412215 10038 tablet_service.cc:1511] Processing CreateTablet for tablet 82406372d02c4dd59d4ef5c32436e148 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:57.414944 10038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82406372d02c4dd59d4ef5c32436e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.420424 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 82406372d02c4dd59d4ef5c32436e148 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:57.422009 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82406372d02c4dd59d4ef5c32436e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.427289 10275 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:57.430542 10254 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.452900 10044 tablet_service.cc:1511] Processing CreateTablet for tablet ece395e49d7e474182be2d25642b90de (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:57.454597 10044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece395e49d7e474182be2d25642b90de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.455979 10116 tablet_service.cc:1511] Processing CreateTablet for tablet ece395e49d7e474182be2d25642b90de (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:57.457566 10116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece395e49d7e474182be2d25642b90de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.462994 10248 tablet_bootstrap.cc:654] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.466722  9954 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.481653 10249 tablet_bootstrap.cc:654] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.501535 10284 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.502211 10284 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.502707 10189 tablet_service.cc:1511] Processing CreateTablet for tablet 82406372d02c4dd59d4ef5c32436e148 (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:15:57.504240 10189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82406372d02c4dd59d4ef5c32436e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.508942 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.513208 10284 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:57.517469 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:57.518761 10205 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.524108 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:57.525382 10186 tablet_service.cc:1511] Processing CreateTablet for tablet ece395e49d7e474182be2d25642b90de (DEFAULT_TABLE table=test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:15:57.526952 10186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece395e49d7e474182be2d25642b90de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.533954 10284 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.537577 10248 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.538244 10248 ts_tablet_manager.cc:1403] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.240s	user 0.016s	sys 0.015s
I20260709 13:15:57.541882 10248 raft_consensus.cc:359] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.542742 10248 raft_consensus.cc:385] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.544865  9954 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.546445 10248 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.547515 10248 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.550220 10248 ts_tablet_manager.cc:1434] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:15:57.551498 10248 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.556730 10273 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:57.562624 10250 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.572356 10250 ts_tablet_manager.cc:1403] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.414s	user 0.010s	sys 0.018s
I20260709 13:15:57.595268 10250 raft_consensus.cc:359] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.596158 10250 raft_consensus.cc:385] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.596489 10250 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.599293 10248 tablet_bootstrap.cc:654] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.611253 10250 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.614039 10250 ts_tablet_manager.cc:1434] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:15:57.623889 10249 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:57.624158 10248 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.624639 10248 ts_tablet_manager.cc:1403] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.073s	user 0.003s	sys 0.010s
I20260709 13:15:57.617182 10250 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:57.624560 10249 ts_tablet_manager.cc:1403] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.227s	user 0.026s	sys 0.007s
I20260709 13:15:57.634740 10249 raft_consensus.cc:359] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.635744 10249 raft_consensus.cc:385] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.629988 10248 raft_consensus.cc:359] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.636494 10248 raft_consensus.cc:385] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.636871 10248 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.637899 10248 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.641714 10248 ts_tablet_manager.cc:1434] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:15:57.636137 10249 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.647302 10249 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.650116 10249 ts_tablet_manager.cc:1434] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:15:57.650709 10248 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.655094 10249 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.659421 10248 tablet_bootstrap.cc:654] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.665021 10249 tablet_bootstrap.cc:654] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.670045 10250 tablet_bootstrap.cc:654] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.681984 10248 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.682240 10131 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.682561 10248 ts_tablet_manager.cc:1403] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.008s
I20260709 13:15:57.684186 10249 tablet_bootstrap.cc:492] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:57.684774 10249 ts_tablet_manager.cc:1403] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:15:57.685746 10248 raft_consensus.cc:359] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.686514 10248 raft_consensus.cc:385] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.686818 10248 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.688186 10249 raft_consensus.cc:359] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.688390 10248 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.689190 10249 raft_consensus.cc:385] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.689548 10249 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.690442 10249 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.693876 10250 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.694496 10250 ts_tablet_manager.cc:1403] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.003s
I20260709 13:15:57.695093 10286 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.696334 10249 ts_tablet_manager.cc:1434] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:15:57.696085 10286 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.697583 10249 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.697956 10250 raft_consensus.cc:359] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.698885 10250 raft_consensus.cc:385] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.699296 10250 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.700265 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.703860 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.704128 10284 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.704927 10248 ts_tablet_manager.cc:1434] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.004s
I20260709 13:15:57.704820 10284 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.706113 10248 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
W20260709 13:15:57.706225 10095 leader_election.cc:343] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.706575 10249 tablet_bootstrap.cc:654] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:57.707073 10096 leader_election.cc:343] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.707585 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:57.708452 10254 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.709326 10286 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.710726 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.711190 10250 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:15:57.713040 10170 leader_election.cc:343] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.713752 10250 ts_tablet_manager.cc:1434] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:15:57.715067 10284 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:57.716571 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:57.717064 10248 tablet_bootstrap.cc:654] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.714897 10273 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:15:57.728214 10168 leader_election.cc:343] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.728780 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:57.730067 10284 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:57.717087 10250 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
W20260709 13:15:57.721040 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:57.726411 10249 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:57.737382 10249 ts_tablet_manager.cc:1403] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.008s
I20260709 13:15:57.740772 10250 tablet_bootstrap.cc:654] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.746598 10249 raft_consensus.cc:359] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.747557 10249 raft_consensus.cc:385] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.747845 10248 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.747956 10249 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.748160 10250 tablet_bootstrap.cc:492] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.748462 10248 ts_tablet_manager.cc:1403] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.015s
I20260709 13:15:57.748862 10250 ts_tablet_manager.cc:1403] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.003s
I20260709 13:15:57.749220 10249 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.751891 10249 ts_tablet_manager.cc:1434] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.005s
I20260709 13:15:57.751698 10248 raft_consensus.cc:359] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.752418 10250 raft_consensus.cc:359] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.753224 10249 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.753400 10250 raft_consensus.cc:385] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.753974 10250 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.755020 10250 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.756570 10248 raft_consensus.cc:385] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.757004 10248 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.758036 10248 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.761898 10249 tablet_bootstrap.cc:654] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.764036 10248 ts_tablet_manager.cc:1434] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:15:57.771248 10248 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.773180 10273 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:57.777947 10248 tablet_bootstrap.cc:654] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.784679 10250 ts_tablet_manager.cc:1434] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.035s	user 0.013s	sys 0.010s
I20260709 13:15:57.786019 10250 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:57.793586 10250 tablet_bootstrap.cc:654] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.806651 10250 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.807354 10250 ts_tablet_manager.cc:1403] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.001s
I20260709 13:15:57.811159 10250 raft_consensus.cc:359] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.812011 10250 raft_consensus.cc:385] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.812332 10250 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.813203 10250 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.815595 10250 ts_tablet_manager.cc:1434] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:15:57.816636 10250 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:57.822626 10250 tablet_bootstrap.cc:654] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.830454 10273 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.831171 10273 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.835280 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.837291 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:57.838311 10131 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:57.840353 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:57.841267 10273 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.841823 10273 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:57.842195 10273 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.842542 10273 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:57.845610 10020 leader_election.cc:343] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.852273 10273 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.854773 10273 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.858094 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:57.858173 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:57.860183 10131 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:57.861474 10020 leader_election.cc:343] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.861608 10250 tablet_bootstrap.cc:492] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.863495 10250 ts_tablet_manager.cc:1403] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.001s
I20260709 13:15:57.863577 10248 tablet_bootstrap.cc:492] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.864140 10248 ts_tablet_manager.cc:1403] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.093s	user 0.038s	sys 0.051s
I20260709 13:15:57.865988 10250 raft_consensus.cc:359] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.866566 10250 raft_consensus.cc:385] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.866858 10250 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.866956 10248 raft_consensus.cc:359] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.867765 10248 raft_consensus.cc:385] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.868078 10248 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.867908 10250 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.869197 10249 tablet_bootstrap.cc:492] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:57.869050 10248 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.869805 10249 ts_tablet_manager.cc:1403] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.117s	user 0.018s	sys 0.040s
I20260709 13:15:57.871282 10248 ts_tablet_manager.cc:1434] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:15:57.872539 10250 ts_tablet_manager.cc:1434] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:15:57.872718 10248 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.873626 10250 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:57.877820 10249 raft_consensus.cc:359] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.878737 10249 raft_consensus.cc:385] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.879488 10249 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.880889 10250 tablet_bootstrap.cc:654] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.880568 10249 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.883220 10286 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.883785 10286 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.887662 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.888478 10056 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:57.887667 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:57.889992 10204 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:57.892174 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:57.893178 10254 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:57.893540 10254 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.893896 10254 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.894982 10249 ts_tablet_manager.cc:1434] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:15:57.899637 10249 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:57.900161 10286 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.902575 10284 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.903215 10284 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.905879 10248 tablet_bootstrap.cc:654] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.908938 10284 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:57.908090 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:57.908548 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:57.909950 10130 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:15:57.911405 10254 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.911870 10131 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:57.910233 10056 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:15:57.914242 10254 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.914415 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:57.915431 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:57.915478 10284 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:57.915760 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:57.916283 10284 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.916504 10204 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.916894 10284 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.917268 10273 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:57.917804 10273 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:57.918723 10273 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.925835 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:57.926759 10056 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.930233 10284 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.935801 10249 tablet_bootstrap.cc:654] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:57.935518 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:57.936707 10055 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.926107 10204 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:57.938845 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:57.939944 10254 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:57.940795 10254 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:57.941810 10254 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.947961 10248 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:57.948566 10248 ts_tablet_manager.cc:1403] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.002s
I20260709 13:15:57.951396 10284 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:57.952112 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:57.952669 10131 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:57.952193  9951 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.956562 10248 raft_consensus.cc:359] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.957504 10248 raft_consensus.cc:385] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:57.957899 10248 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:57.958861 10248 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.960562 10055 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:15:57.961251 10248 ts_tablet_manager.cc:1434] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:15:57.950752 10056 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:57.962472 10248 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:57.962821 10254 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.963028 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:57.965620 10254 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:57.966320 10284 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:57.967341 10250 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:57.967876 10250 ts_tablet_manager.cc:1403] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.094s	user 0.025s	sys 0.049s
I20260709 13:15:57.963585  9952 catalog_manager.cc:5697] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.974920 10254 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:57.977350 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:57.982143 10095 leader_election.cc:343] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:57.988123 10131 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:15:57.991516 10284 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:15:57.993014 10284 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.003388 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.005947 10056 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.012283 10250 raft_consensus.cc:359] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.012318 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:58.013325 10250 raft_consensus.cc:385] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.016130 10248 tablet_bootstrap.cc:654] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.017223 10286 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.017792 10286 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.018188 10286 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.023393 10254 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.033448  9952 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:58.013736 10250 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.037228 10250 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:15:58.038777 10095 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:58.042346 10249 tablet_bootstrap.cc:492] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:58.043234 10249 ts_tablet_manager.cc:1403] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.144s	user 0.031s	sys 0.004s
I20260709 13:15:58.045893 10273 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.046507 10273 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.047598 10249 raft_consensus.cc:359] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.048765 10249 raft_consensus.cc:385] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.049245 10249 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.052152 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.051957 10286 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.053071 10131 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:58.052327 10250 ts_tablet_manager.cc:1434] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.015s
I20260709 13:15:58.055141 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.056054 10250 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.056284 10204 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:58.058743 10273 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.058660 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:58.063542 10306 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.063928 10250 tablet_bootstrap.cc:654] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.064079 10306 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.064495 10306 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.066303 10249 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.069921 10249 ts_tablet_manager.cc:1434] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:15:58.074975 10306 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.081353 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:58.082202 10131 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.082537 10306 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.083263 10249 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:58.086127 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.099516 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:58.101039 10056 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.102280 10131 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:58.104111 10249 tablet_bootstrap.cc:654] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.095082 10205 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.104218 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.106012 10306 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.106506 10306 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:58.107668 10286 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.108914 10306 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.110285 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:15:58.112433 10095 leader_election.cc:343] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:58.113763 10320 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:15:58.129756 10056 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:58.133666 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:58.134486 10248 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.132751 10205 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:58.135025 10248 ts_tablet_manager.cc:1403] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.173s	user 0.047s	sys 0.046s
I20260709 13:15:58.136484 10254 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.141004 10250 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:58.140646 10248 raft_consensus.cc:359] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.141789 10248 raft_consensus.cc:385] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.142241 10248 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.141600 10250 ts_tablet_manager.cc:1403] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.007s
I20260709 13:15:58.143352 10248 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.146414 10248 ts_tablet_manager.cc:1434] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:15:58.148018 10248 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.150192 10250 raft_consensus.cc:359] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.151239 10250 raft_consensus.cc:385] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.151618 10250 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.152510 10250 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.151584  9951 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.177269 10250 ts_tablet_manager.cc:1434] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.009s
I20260709 13:15:58.178550 10250 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.182834 10254 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:58.184769 10254 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.187151 10250 tablet_bootstrap.cc:654] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.188505 10248 tablet_bootstrap.cc:654] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.206851 10248 tablet_bootstrap.cc:492] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.207605 10249 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:58.207615 10248 ts_tablet_manager.cc:1403] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.060s	user 0.026s	sys 0.020s
I20260709 13:15:58.208323 10249 ts_tablet_manager.cc:1403] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.125s	user 0.012s	sys 0.011s
I20260709 13:15:58.207453  9952 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.211402 10306 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.211973 10306 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.213996 10306 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.215018 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.215866 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.216295 10254 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.216624 10205 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:58.216835 10254 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:15:58.216981 10019 leader_election.cc:343] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.219996 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.220782 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.221496 10254 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.221557 10056 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.223466 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.223482 10248 raft_consensus.cc:359] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.224537 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.224892 10254 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.225468 10248 raft_consensus.cc:385] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.225570 10254 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.225994 10248 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.226550 10254 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.226109 10306 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.225386 10205 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.227401 10248 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.227433 10306 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.229737 10306 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.229969 10250 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:58.230479 10248 ts_tablet_manager.cc:1434] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260709 13:15:58.231741 10248 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.237159 10250 ts_tablet_manager.cc:1403] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.019s
I20260709 13:15:58.237020 10324 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.237922 10324 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.240676 10248 tablet_bootstrap.cc:654] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.240810 10250 raft_consensus.cc:359] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.241823 10250 raft_consensus.cc:385] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.242268 10250 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.241730 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.243287 10250 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.244913 10254 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.245867 10250 ts_tablet_manager.cc:1434] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:58.247135 10250 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.247843 10324 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.249253 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:58.249413 10254 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.250010 10055 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.249951 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.250702 10205 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.253368 10250 tablet_bootstrap.cc:654] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.253247 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.253980 10205 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.254946 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.256174 10254 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.256583 10254 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.256934 10254 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.260066 10055 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:58.261358 10275 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.261663 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.261951 10275 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.262733 10324 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.263291 10324 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:58.264309 10324 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.265268 10248 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.265841 10248 ts_tablet_manager.cc:1403] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.009s
I20260709 13:15:58.223232 10249 raft_consensus.cc:359] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.267249 10275 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.267431 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.267307 10249 raft_consensus.cc:385] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.270068 10249 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
W20260709 13:15:58.269400 10020 leader_election.cc:343] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.270874 10249 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.272009 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.263259 10205 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:15:58.274626 10019 leader_election.cc:343] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.275192 10019 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.276445 10275 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.276482 10250 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:58.278219 10250 ts_tablet_manager.cc:1403] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.005s
I20260709 13:15:58.278757 10275 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.279376 10248 raft_consensus.cc:359] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.279619 10275 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.280339 10248 raft_consensus.cc:385] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.280699 10248 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.281620 10248 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.283800 10248 ts_tablet_manager.cc:1434] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 13:15:58.284930 10248 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.291317 10254 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.292338 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.293076 10131 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:58.294339 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.295296 10273 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.295678 10273 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.295715 10248 tablet_bootstrap.cc:654] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.296073 10273 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.283911 10249 ts_tablet_manager.cc:1434] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 13:15:58.300015 10249 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:58.303390  9951 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.287997 10306 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.305972 10056 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:15:58.307168 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.307749 10275 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:15:58.309136 10020 leader_election.cc:343] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.312539 10284 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.313078 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:58.314585 10056 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:15:58.315891 10273 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.313169 10284 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.321772 10205 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.321914 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.322149 10284 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:58.325642 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:58.325965 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.326743 10131 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:15:58.326746 10130 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 1: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:15:58.328608 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.329368 10254 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.329706 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.330499 10204 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.335312 10284 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.336511 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.337512 10275 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:58.344208 10275 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.338910 10204 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:58.332638 10250 raft_consensus.cc:359] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.347375 10306 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.349486 10273 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.359309 10250 raft_consensus.cc:385] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.360311 10250 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.348790 10324 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:15:58.348485 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:15:58.361316 10250 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.369796 10324 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.370481 10324 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:58.372299 10324 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.371009 10250 ts_tablet_manager.cc:1434] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.002s
I20260709 13:15:58.374174 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:58.376323 10130 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.382683 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.387409 10130 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:58.388363 10306 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.388502 10250 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.389178 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.391017 10248 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.390864 10306 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.391638 10248 ts_tablet_manager.cc:1403] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.107s	user 0.048s	sys 0.037s
I20260709 13:15:58.392978 10306 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:58.394424 10306 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.394935 10248 raft_consensus.cc:359] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.395830 10248 raft_consensus.cc:385] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.396174 10248 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.397397 10248 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.400301 10248 ts_tablet_manager.cc:1434] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:15:58.401432 10248 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.403765 10275 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.404621 10275 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.408398 10248 tablet_bootstrap.cc:654] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.409262 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:58.411437 10019 leader_election.cc:343] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.412856 10275 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.412963 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.416749 10249 tablet_bootstrap.cc:654] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:58.421167 10020 leader_election.cc:343] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.425217 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.428189 10250 tablet_bootstrap.cc:654] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:58.431831 10020 leader_election.cc:343] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.433884 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.435946 10333 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.449345 10248 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.450244 10248 ts_tablet_manager.cc:1403] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.049s	user 0.025s	sys 0.015s
I20260709 13:15:58.451216 10323 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:15:58.466096 10249 tablet_bootstrap.cc:492] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:58.466643 10249 ts_tablet_manager.cc:1403] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.167s	user 0.022s	sys 0.001s
I20260709 13:15:58.470105 10249 raft_consensus.cc:359] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.470908 10249 raft_consensus.cc:385] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.471249 10249 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.472146 10249 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.473264 10284 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.473817 10284 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.479676  9952 catalog_manager.cc:5697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.481552 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.472352 10248 raft_consensus.cc:359] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.483620 10248 raft_consensus.cc:385] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.483968 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:15:58.488485 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56732.2us
I20260709 13:15:58.483268  9951 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.491801 10248 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.493750 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.493477 10248 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.496187 10316 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.497510 10323 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.498384 10250 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:58.498212 10323 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.502290 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.484534 10284 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:15:58.503918 10096 leader_election.cc:343] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.504628 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.502693 10323 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:15:58.506673 10095 leader_election.cc:343] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.507278 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:58.496198 10248 ts_tablet_manager.cc:1434] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.001s
I20260709 13:15:58.509475 10323 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.509804 10249 ts_tablet_manager.cc:1434] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.005s
I20260709 13:15:58.511530 10249 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:58.520375 10284 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.509565 10248 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.525256 10249 tablet_bootstrap.cc:654] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.535928 10248 tablet_bootstrap.cc:654] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.557137 10250 ts_tablet_manager.cc:1403] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.169s	user 0.024s	sys 0.000s
I20260709 13:15:58.560833 10250 raft_consensus.cc:359] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.561722 10250 raft_consensus.cc:385] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.562011 10250 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.568542 10250 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.572976 10250 ts_tablet_manager.cc:1434] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:15:58.574349 10250 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.576761 10204 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.583675 10250 tablet_bootstrap.cc:654] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.585848 10316 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.589448 10323 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.567611 10342 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:15:58.590085 10323 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.592482 10324 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.593076 10324 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.600136 10254 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.603058 10324 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.607208 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.608047 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.610076 10249 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:58.610841 10249 ts_tablet_manager.cc:1403] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.100s	user 0.030s	sys 0.012s
I20260709 13:15:58.610685 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.614538 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.615326 10254 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.615787 10254 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.616173 10254 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:58.616622 10095 leader_election.cc:343] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.617415 10323 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.623605 10249 raft_consensus.cc:359] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.624886 10249 raft_consensus.cc:385] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.625305 10249 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.623646 10248 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.626777 10248 ts_tablet_manager.cc:1403] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.118s	user 0.029s	sys 0.022s
I20260709 13:15:58.605599 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.634661 10203 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.634174 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.627141 10249 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.638695 10249 ts_tablet_manager.cc:1434] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
W20260709 13:15:58.640281 10096 leader_election.cc:343] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.641273 10096 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:58.643462 10250 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:58.643392 10323 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:58.644306 10250 ts_tablet_manager.cc:1403] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.070s	user 0.023s	sys 0.042s
I20260709 13:15:58.647639 10250 raft_consensus.cc:359] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.648547 10250 raft_consensus.cc:385] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.648872 10250 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.649631 10250 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.654553 10250 ts_tablet_manager.cc:1434] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:15:58.655865 10250 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:58.655733 10205 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.658512 10249 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:58.651669 10248 raft_consensus.cc:359] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.661865 10248 raft_consensus.cc:385] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.662197 10248 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.663086 10248 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.660475 10273 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:15:58.669687 10254 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.700134 10273 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.701041 10273 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.714298 10248 ts_tablet_manager.cc:1434] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.086s	user 0.004s	sys 0.003s
I20260709 13:15:58.716437 10248 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:58.755252 10316 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.755993 10316 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.760599 10323 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.761435 10323 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.785871 10323 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.821775 10248 tablet_bootstrap.cc:654] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.844017 10254 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.832419 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.850656 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.842000 10316 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:58.856230 10170 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:58.861095 10316 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:15:58.861927 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.862800 10054 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
W20260709 13:15:58.870822 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:15:58.873519 10249 tablet_bootstrap.cc:654] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.879578 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:58.880357 10054 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.881194 10273 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.883586 10250 tablet_bootstrap.cc:654] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.889385 10323 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:58.886452 10352 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260709 13:15:58.890663 10254 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:58.899643 10248 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:58.902601 10096 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.903462 10248 ts_tablet_manager.cc:1403] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.007s
I20260709 13:15:58.903856 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:58.899986 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:58.905128 10131 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:58.902104 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.904523 10204 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.905833 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:58.908260 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:58.908736 10248 raft_consensus.cc:359] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.909859 10248 raft_consensus.cc:385] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:58.910307 10248 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:58.911057 10203 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:58.911616 10286 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.912045 10286 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.912487 10286 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.913283 10248 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.915668 10248 ts_tablet_manager.cc:1434] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:15:58.916931 10248 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
W20260709 13:15:58.925137 10020 leader_election.cc:343] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:58.932181 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:58.933305 10254 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:58.932435 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:58.935776 10342 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:58.936224 10342 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.936610 10342 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:58.938661 10323 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.934481 10254 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:58.948971 10323 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.954253 10323 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.959653 10254 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.971180 10342 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.972720 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:58.974534 10248 tablet_bootstrap.cc:654] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:58.981719 10286 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:58.994153 10286 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:15:58.995826 10096 leader_election.cc:343] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:59.003727 10020 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:58.995162 10204 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:15:59.002056 10358 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.000s	sys 0.002s
I20260709 13:15:59.007344 10342 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:58.992620 10323 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:15:59.011312 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.013335 10352 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:15:59.016789 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.016618 10020 leader_election.cc:343] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.018836 10250 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.019451 10250 ts_tablet_manager.cc:1403] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.364s	user 0.070s	sys 0.104s
I20260709 13:15:59.017627  9952 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.025333 10249 tablet_bootstrap.cc:492] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.028179 10249 ts_tablet_manager.cc:1403] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.370s	user 0.024s	sys 0.038s
I20260709 13:15:59.027896 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:59.028894 10131 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.031522 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.032311 10205 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.045348 10342 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.046041 10342 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.048807 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:59.049603 10054 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.050581 10342 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.046824 10250 raft_consensus.cc:359] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.056636 10250 raft_consensus.cc:385] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.056979 10250 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.058720 10250 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.066144 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.062747 10095 leader_election.cc:343] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.067174 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.070343 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.070600 10323 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.062911 10249 raft_consensus.cc:359] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.073508 10249 raft_consensus.cc:385] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.073791 10249 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.074275 10249 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.076331 10131 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
W20260709 13:15:59.076963 10020 leader_election.cc:343] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.079265 10250 ts_tablet_manager.cc:1434] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:15:59.080705 10250 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
W20260709 13:15:59.084954 10019 leader_election.cc:343] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.085608 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:59.086539 10275 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.086952 10249 ts_tablet_manager.cc:1434] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.003s
I20260709 13:15:59.088132 10249 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.090191 10273 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.092703 10248 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.093425 10248 ts_tablet_manager.cc:1403] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.177s	user 0.029s	sys 0.029s
I20260709 13:15:59.079298 10205 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:59.097357 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:59.098569 10273 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.099298 10273 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:59.099575 10248 raft_consensus.cc:359] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.100617 10248 raft_consensus.cc:385] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.100309 10273 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.101059 10248 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.101989 10248 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.103510 10054 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:59.104753 10248 ts_tablet_manager.cc:1434] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:15:59.105881 10248 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.110201 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.113938 10364 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.117137 10364 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:59.119484 10364 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.136873 10250 tablet_bootstrap.cc:654] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.144285 10248 tablet_bootstrap.cc:654] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.158726 10249 tablet_bootstrap.cc:654] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.158816  9952 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.174680 10248 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.175323 10248 ts_tablet_manager.cc:1403] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.003s
I20260709 13:15:59.178809 10248 raft_consensus.cc:359] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.179838 10248 raft_consensus.cc:385] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.180661 10248 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.181843 10248 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.184746 10248 ts_tablet_manager.cc:1434] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:15:59.186241 10248 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.196842 10249 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.197499 10249 ts_tablet_manager.cc:1403] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.003s
I20260709 13:15:59.198096 10248 tablet_bootstrap.cc:654] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.198194  9954 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.200883 10249 raft_consensus.cc:359] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.201666 10249 raft_consensus.cc:385] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.201964 10249 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.202874 10249 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.203343 10250 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.204564 10250 ts_tablet_manager.cc:1403] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.006s
I20260709 13:15:59.205116 10249 ts_tablet_manager.cc:1434] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.003s
I20260709 13:15:59.206377 10249 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.208487 10250 raft_consensus.cc:359] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.209326 10250 raft_consensus.cc:385] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.209607 10250 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.210371 10250 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.211246 10248 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.211772 10248 ts_tablet_manager.cc:1403] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.011s
I20260709 13:15:59.213647 10249 tablet_bootstrap.cc:654] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.214767 10248 raft_consensus.cc:359] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.215720 10248 raft_consensus.cc:385] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.216032 10248 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.216969 10248 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.219748 10248 ts_tablet_manager.cc:1434] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:15:59.220901 10248 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.227458 10250 ts_tablet_manager.cc:1434] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:15:59.228296 10248 tablet_bootstrap.cc:654] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.229354 10250 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.239337 10250 tablet_bootstrap.cc:654] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.241683 10249 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.242249 10249 ts_tablet_manager.cc:1403] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.008s
I20260709 13:15:59.242489 10248 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.243079 10248 ts_tablet_manager.cc:1403] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 13:15:59.245518 10249 raft_consensus.cc:359] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.246353 10249 raft_consensus.cc:385] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.246625 10249 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.246896 10248 raft_consensus.cc:359] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.247594 10249 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.248070 10248 raft_consensus.cc:385] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.248593 10248 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.249728 10248 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.253623 10248 ts_tablet_manager.cc:1434] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:59.253726 10249 ts_tablet_manager.cc:1434] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:15:59.255236 10249 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.255355 10248 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.268692 10248 tablet_bootstrap.cc:654] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.272003 10249 tablet_bootstrap.cc:654] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.283528 10248 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.283854 10250 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.284063 10248 ts_tablet_manager.cc:1403] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.010s
I20260709 13:15:59.284328 10250 ts_tablet_manager.cc:1403] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.026s
I20260709 13:15:59.284370 10249 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.285157 10249 ts_tablet_manager.cc:1403] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.010s
I20260709 13:15:59.286722 10275 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.287288 10275 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.288244 10248 raft_consensus.cc:359] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.289260 10248 raft_consensus.cc:385] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.289391 10275 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.289552 10248 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.288244 10250 raft_consensus.cc:359] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.290315 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.290586 10250 raft_consensus.cc:385] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.291013 10250 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.290634 10249 raft_consensus.cc:359] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.291474 10249 raft_consensus.cc:385] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.291292 10131 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:59.291476 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.292054 10249 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.291857 10250 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.292886 10249 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.293273 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.294493 10250 ts_tablet_manager.cc:1434] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:15:59.294569 10342 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.295054 10249 ts_tablet_manager.cc:1434] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:15:59.295159 10342 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.295240 10248 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.295914 10342 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.297662 10248 ts_tablet_manager.cc:1434] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:15:59.299033 10248 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.303337 10250 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.303376 10205 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:59.306133 10248 tablet_bootstrap.cc:654] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.311055 10364 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.311816 10364 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.315953 10286 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.316866 10286 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.320883 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.327903 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.324903 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.329677 10056 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:59.329999 10248 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.323886 10364 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.330581 10248 ts_tablet_manager.cc:1403] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.018s
I20260709 13:15:59.330626 10352 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.331234 10352 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.321475 10286 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.333003 10249 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:15:59.329731 10095 leader_election.cc:343] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.333750 10248 raft_consensus.cc:359] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.334625 10248 raft_consensus.cc:385] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.334959 10248 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.335737 10248 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.336398 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.337147 10203 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:59.333745 10352 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.338677 10248 ts_tablet_manager.cc:1434] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:15:59.339205 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.340133 10352 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.340569 10352 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.340796 10248 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.341012 10352 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.342372 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.342835 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.344959 10096 leader_election.cc:343] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:59.345052 10095 leader_election.cc:343] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.345564 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.346421 10286 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.346971 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:59.347909 10286 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.348415 10286 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.348938 10286 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.349381 10352 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.351697 10352 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.352566 10250 tablet_bootstrap.cc:654] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.353016 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.353722 10205 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.358196 10249 tablet_bootstrap.cc:654] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.360208 10250 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.360900 10250 ts_tablet_manager.cc:1403] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.003s
W20260709 13:15:59.361824 10096 leader_election.cc:343] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.362430 10205 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:59.363204 10352 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:15:59.363799 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.364319 10250 raft_consensus.cc:359] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.364867 10364 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.365078 10250 raft_consensus.cc:385] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.365412 10364 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:59.365612 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:59.366130 10248 tablet_bootstrap.cc:654] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.366675 10364 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.348415 10342 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.369107 10349 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.369958 10349 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.365430 10250 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.381191 10250 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.387519 10349 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:59.400600 10286 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.407698 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.409353 10131 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:15:59.414561 10096 leader_election.cc:343] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.416378 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.409464 10250 ts_tablet_manager.cc:1434] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.002s
I20260709 13:15:59.419224 10250 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.421706 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:59.422709 10129 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.435725 10342 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.441269 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.442122 10205 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.442795 10286 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.443586 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:15:59.445441 10095 leader_election.cc:343] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.452102 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:15:59.453064 10349 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.459872 10286 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.461372 10349 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.464840 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:59.466146 10054 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.478336 10129 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:59.482074 10205 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:59.482573 10249 tablet_bootstrap.cc:492] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.483139 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.484157 10250 tablet_bootstrap.cc:654] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.485452 10249 ts_tablet_manager.cc:1403] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.189s	user 0.035s	sys 0.076s
I20260709 13:15:59.485522 10273 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.486212 10273 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:59.487355 10273 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.489446 10248 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.489082 10249 raft_consensus.cc:359] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.489814 10249 raft_consensus.cc:385] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.489943 10248 ts_tablet_manager.cc:1403] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.011s
I20260709 13:15:59.490126 10249 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.492722 10054 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:59.493733 10364 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.494259 10364 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.494618 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:59.495889 10286 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.496448 10286 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:59.496714 10364 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.490911 10249 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.497426 10286 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.493310  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.498936 10316 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.499868 10316 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.505658 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.506872 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:15:59.508755 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:59.509452 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.510330 10129 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:15:59.509950 10248 raft_consensus.cc:359] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.510991 10248 raft_consensus.cc:385] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.511451 10248 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.511977 10316 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:59.512614 10349 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.499106 10249 ts_tablet_manager.cc:1434] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:15:59.513183 10349 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.514242 10249 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.514444 10349 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.500923 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.513180 10248 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.500291 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.521929 10096 leader_election.cc:343] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:15:59.522612 10095 leader_election.cc:343] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.523099 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.525038 10352 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.536470 10248 ts_tablet_manager.cc:1434] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.003s
I20260709 13:15:59.538301 10248 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.546686 10249 tablet_bootstrap.cc:654] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.553189 10364 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.555249 10250 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.556205  9954 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.563201 10349 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.569469  9951 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.555850 10250 ts_tablet_manager.cc:1403] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.137s	user 0.017s	sys 0.006s
I20260709 13:15:59.577049 10342 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:15:59.584643 10349 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:59.585996 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:59.586476 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:59.586730 10056 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.587087 10131 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.589648 10250 raft_consensus.cc:359] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.590309 10250 raft_consensus.cc:385] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.590606 10250 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.595347 10250 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.598131 10250 ts_tablet_manager.cc:1434] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:15:59.601034 10248 tablet_bootstrap.cc:654] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.608757 10352 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.609386 10352 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.626832 10286 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.614982 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.611157 10364 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.628997 10286 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.629601 10364 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:15:59.630328 10095 leader_election.cc:343] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.621094 10056 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:15:59.615031 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.632472 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.632997 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:59.622788 10273 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260709 13:15:59.633927 10096 leader_election.cc:343] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.634250 10349 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.634442 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.634817 10349 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:15:59.633185 10054 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:15:59.635788 10349 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.636734 10390 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.637746 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:15:59.638407 10364 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.639976 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.641506 10250 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.611789 10352 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.644631 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.645833 10096 leader_election.cc:343] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.647360 10286 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.647527 10273 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.621162 10131 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:15:59.655511 10364 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.654546 10248 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.657172 10248 ts_tablet_manager.cc:1403] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.119s	user 0.032s	sys 0.028s
I20260709 13:15:59.658371 10364 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.669484 10250 tablet_bootstrap.cc:654] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:59.659376 10095 leader_election.cc:343] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.675348 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.677707 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.681212 10352 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.681010 10248 raft_consensus.cc:359] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.664404 10364 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.658187 10273 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.686268 10273 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.681859 10248 raft_consensus.cc:385] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.691416 10248 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.691579 10250 tablet_bootstrap.cc:492] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.692399 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:15:59.693138 10205 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:59.692202 10248 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.694360 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:59.695351 10248 ts_tablet_manager.cc:1434] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.003s
I20260709 13:15:59.695740 10250 ts_tablet_manager.cc:1403] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.001s
I20260709 13:15:59.696292 10248 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.690579 10364 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.697392 10273 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:15:59.699939 10273 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.700739 10273 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.704416 10249 tablet_bootstrap.cc:492] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.707732 10249 ts_tablet_manager.cc:1403] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.194s	user 0.011s	sys 0.015s
I20260709 13:15:59.712764 10364 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.722934 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.723881 10250 raft_consensus.cc:359] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.724113 10129 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:15:59.724745 10250 raft_consensus.cc:385] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.725137 10250 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.725855 10250 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.728888 10250 ts_tablet_manager.cc:1434] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:15:59.730176 10250 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.731966 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:15:59.732656 10056 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:15:59.735728 10095 leader_election.cc:343] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.735548 10364 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.740942 10249 raft_consensus.cc:359] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.742419 10249 raft_consensus.cc:385] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.743045 10249 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.749078 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.757668 10273 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.759775 10249 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.765499 10249 ts_tablet_manager.cc:1434] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.004s
I20260709 13:15:59.766616 10249 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.771912 10250 tablet_bootstrap.cc:654] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:15:59.781962 10095 leader_election.cc:343] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.782613 10273 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.776932 10056 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:15:59.787091  9954 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:59.790028 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:15:59.792909 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:15:59.794284 10352 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.792438 10203 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.794864 10352 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:15:59.795903 10352 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.792905 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:15:59.797853 10131 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.799618 10323 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.800266 10323 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.804044 10248 tablet_bootstrap.cc:654] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.803537 10323 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.810940 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:15:59.813340 10095 leader_election.cc:343] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.814232 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:15:59.818058 10096 leader_election.cc:343] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:15:59.818536 10203 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:59.818640 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:15:59.822431 10131 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:15:59.822074 10323 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.829873 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:15:59.834009 10275 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:15:59.830657 10249 tablet_bootstrap.cc:654] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.831748  9951 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.834718 10275 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:15:59.837217 10248 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.836827 10275 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.837852 10248 ts_tablet_manager.cc:1403] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.001s
I20260709 13:15:59.841396 10248 raft_consensus.cc:359] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.842173 10248 raft_consensus.cc:385] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.842468 10248 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.842692 10250 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.843569 10250 ts_tablet_manager.cc:1403] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.114s	user 0.012s	sys 0.009s
I20260709 13:15:59.845364 10248 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.846843 10250 raft_consensus.cc:359] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.847793 10250 raft_consensus.cc:385] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.848227 10250 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.849212 10250 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.851065 10248 ts_tablet_manager.cc:1434] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:15:59.852376 10248 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.854748 10250 ts_tablet_manager.cc:1434] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:15:59.856045 10250 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.856259 10054 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:15:59.859560 10248 tablet_bootstrap.cc:654] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.862498 10250 tablet_bootstrap.cc:654] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.867291 10323 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.878479 10316 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.879634 10248 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:15:59.879544 10316 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.880278 10248 ts_tablet_manager.cc:1403] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.003s
I20260709 13:15:59.883428 10248 raft_consensus.cc:359] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.883976 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:15:59.884703 10316 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:15:59.884229 10248 raft_consensus.cc:385] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.887228 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:15:59.887243  9952 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.886965 10248 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.889719 10248 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.893900 10248 ts_tablet_manager.cc:1434] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:15:59.894987 10248 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:15:59.895151 10250 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:15:59.895632 10250 ts_tablet_manager.cc:1403] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.012s
I20260709 13:15:59.903023 10250 raft_consensus.cc:359] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.908123 10250 raft_consensus.cc:385] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.908639 10250 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.909747 10250 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.916527 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:15:59.916249 10095 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:59.936141 10250 ts_tablet_manager.cc:1434] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:15:59.937505 10250 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:15:59.938235 10349 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:15:59.940320 10248 tablet_bootstrap.cc:654] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.949952 10249 tablet_bootstrap.cc:492] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:15:59.950515 10249 ts_tablet_manager.cc:1403] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.184s	user 0.035s	sys 0.026s
I20260709 13:15:59.957576 10249 raft_consensus.cc:359] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.962141 10352 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:15:59.966663 10250 tablet_bootstrap.cc:654] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:15:59.966794 10249 raft_consensus.cc:385] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:15:59.967445 10249 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:15:59.968106 10249 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:15:59.979004 10249 ts_tablet_manager.cc:1434] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.002s
I20260709 13:15:59.981612 10249 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:15:59.983315 10390 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.980790 10323 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.985675 10390 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.003237 10399 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.003790 10399 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.002998 10390 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.006256 10399 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:15:59.998063 10323 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.013726 10248 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.014828 10248 ts_tablet_manager.cc:1403] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.120s	user 0.038s	sys 0.028s
I20260709 13:16:00.012248 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.016601 10254 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.019974 10203 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:00.022410 10323 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.022498 10254 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.069463 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:16:00.075181 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:00.079561 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.083982 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.094206 10131 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:00.087852 10248 raft_consensus.cc:359] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.095057 10248 raft_consensus.cc:385] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.090114 10273 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.095145 10254 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.095537 10248 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.095729 10273 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.096247 10273 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.096678 10248 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.097793 10254 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.085664 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.099901 10248 ts_tablet_manager.cc:1434] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.085s	user 0.003s	sys 0.008s
I20260709 13:16:00.104673 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.109015 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.111104 10250 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:00.112309 10250 ts_tablet_manager.cc:1403] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.175s	user 0.020s	sys 0.008s
I20260709 13:16:00.114837 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.122404 10248 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:00.122225 10273 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.129729 10275 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.130924 10275 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.135828 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:00.138136 10095 leader_election.cc:343] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.139379 10410 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.142570 10202 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.150769 10342 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:16:00.144038 10273 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.151523 10275 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:00.171958 10096 leader_election.cc:343] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.174836 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.175491 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:00.177093 10020 leader_election.cc:343] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.171391 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:00.181820 10205 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.179981 10323 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:00.173321 10095 leader_election.cc:343] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.173969 10273 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:16:00.188009 10249 tablet_bootstrap.cc:700] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent opening tablet: real 0.166s	user 0.002s	sys 0.001s
I20260709 13:16:00.189178 10249 tablet_bootstrap.cc:654] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:00.186038 10096 leader_election.cc:343] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.185272 10250 raft_consensus.cc:359] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.197850 10250 raft_consensus.cc:385] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.198312 10250 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.199707 10275 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.200210 10248 tablet_bootstrap.cc:654] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.200685 10275 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:00.203408 10095 leader_election.cc:343] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.209411 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:00.210156 10129 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.210487 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.211831 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:00.220129 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:00.222352 10020 leader_election.cc:343] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.223446 10275 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.225119 10129 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:00.226465 10349 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.234347 10250 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.236806 10056 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:00.237372 10250 ts_tablet_manager.cc:1434] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.125s	user 0.006s	sys 0.007s
I20260709 13:16:00.255230 10250 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:00.248991 10096 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:00.261507 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84091.7us
I20260709 13:16:00.264072 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.265291 10286 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.266218 10096 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.271644 10323 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.272128 10323 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.272511 10323 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.275935 10249 tablet_bootstrap.cc:492] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.276736 10249 ts_tablet_manager.cc:1403] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.295s	user 0.013s	sys 0.008s
I20260709 13:16:00.282903 10249 raft_consensus.cc:359] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.286293 10249 raft_consensus.cc:385] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.288695 10249 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.296232 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.297943 10275 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.298619 10275 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.299490 10275 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.301631 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:00.305567 10273 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.306459 10273 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:00.307721 10273 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.309857 10249 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.314134 10249 ts_tablet_manager.cc:1434] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.002s
I20260709 13:16:00.316995 10249 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.318224 10248 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.318845 10248 ts_tablet_manager.cc:1403] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.197s	user 0.016s	sys 0.002s
I20260709 13:16:00.324357 10248 raft_consensus.cc:359] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.327960 10248 raft_consensus.cc:385] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.330158 10248 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.327157 10205 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
W20260709 13:16:00.341632 10019 leader_election.cc:343] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.342583 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:00.344821 10342 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.348106 10248 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.363513 10248 ts_tablet_manager.cc:1434] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.044s	user 0.010s	sys 0.000s
I20260709 13:16:00.374593 10248 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:00.378031 10250 tablet_bootstrap.cc:654] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.360458 10323 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.377349 10275 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.386821 10249 tablet_bootstrap.cc:654] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.393893 10412 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:16:00.408181 10095 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.410804 10248 tablet_bootstrap.cc:654] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.413504 10399 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:16:00.419013 10275 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.429919 10323 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.431232 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:00.432245 10053 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.423393 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:00.442526 10020 leader_election.cc:343] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.443379 10202 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.458942 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:00.459877 10131 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.453025 10053 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:00.464632 10096 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:00.465651 10323 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.466218 10323 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:00.467321 10323 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.475176 10249 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.474781 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:00.482772 10275 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.493484 10250 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:00.494573 10250 ts_tablet_manager.cc:1403] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.240s	user 0.014s	sys 0.011s
I20260709 13:16:00.500515 10249 ts_tablet_manager.cc:1403] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.184s	user 0.028s	sys 0.001s
I20260709 13:16:00.513379 10250 raft_consensus.cc:359] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.514346 10250 raft_consensus.cc:385] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.514772 10250 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.515641 10250 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.512012 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:00.530129 10250 ts_tablet_manager.cc:1434] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:16:00.530704 10249 raft_consensus.cc:359] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.531590 10249 raft_consensus.cc:385] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.531944 10249 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.532311 10248 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.532881 10248 ts_tablet_manager.cc:1403] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.159s	user 0.029s	sys 0.023s
I20260709 13:16:00.533658 10249 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.536778 10249 ts_tablet_manager.cc:1434] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:16:00.538616 10249 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.550042 10250 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
W20260709 13:16:00.515949 10095 leader_election.cc:343] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.551007  9951 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.553299 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.554740 10323 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.555361 10323 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.558349 10399 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.558960 10399 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:00.559976 10399 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.561820 10249 tablet_bootstrap.cc:654] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.561309 10248 raft_consensus.cc:359] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.562275 10248 raft_consensus.cc:385] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.562578 10248 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.563495 10248 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.566645 10248 ts_tablet_manager.cc:1434] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 13:16:00.568322 10248 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:00.572486 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.573961 10323 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:00.574863 10096 leader_election.cc:343] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.580175 10248 tablet_bootstrap.cc:654] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.593391 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:00.595505 10095 leader_election.cc:343] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.596385 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:00.597862 10323 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.605934 10250 tablet_bootstrap.cc:654] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.606321 10248 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.606837 10248 ts_tablet_manager.cc:1403] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.004s
I20260709 13:16:00.611708 10249 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.612284 10249 ts_tablet_manager.cc:1403] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.074s	user 0.016s	sys 0.000s
I20260709 13:16:00.610327 10248 raft_consensus.cc:359] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.613897 10248 raft_consensus.cc:385] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.614216 10248 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.615451 10249 raft_consensus.cc:359] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.616211 10249 raft_consensus.cc:385] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.616524 10249 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.617300 10249 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.622197 10248 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.622138 10249 ts_tablet_manager.cc:1434] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:00.625594 10248 ts_tablet_manager.cc:1434] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:16:00.626529 10249 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.627053 10248 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:00.627614 10250 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:00.628129 10250 ts_tablet_manager.cc:1403] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.002s
I20260709 13:16:00.637064  9953 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.647539 10249 tablet_bootstrap.cc:654] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.649887 10250 raft_consensus.cc:359] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.650790 10250 raft_consensus.cc:385] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.651090 10250 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.652046 10250 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.659936 10248 tablet_bootstrap.cc:654] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.663583 10250 ts_tablet_manager.cc:1434] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 13:16:00.664691 10323 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.665977 10323 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.665344 10250 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:00.665026  9953 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.675590 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.677289 10323 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:00.677426 10095 leader_election.cc:343] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.679412 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.689210 10323 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.690059 10399 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.690577 10399 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.690205 10323 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.692790 10399 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.694221 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:00.689400 10096 leader_election.cc:343] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.695487 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:00.698868 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.699987 10131 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:00.698014 10254 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.704490 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:00.705132 10020 leader_election.cc:343] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:00.706126 10095 leader_election.cc:343] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.707041 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.708213 10399 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.708670 10399 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.709062 10399 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.711663 10250 tablet_bootstrap.cc:654] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.712126 10248 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.712731 10248 ts_tablet_manager.cc:1403] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.009s
I20260709 13:16:00.714164 10323 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.712803 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.718696 10051 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:00.720506 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.721689 10249 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.722741 10249 ts_tablet_manager.cc:1403] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.097s	user 0.035s	sys 0.030s
I20260709 13:16:00.723719 10323 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.724316 10323 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.724711 10323 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.726425 10249 raft_consensus.cc:359] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.734846 10399 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.734947 10323 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.737658 10249 raft_consensus.cc:385] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.739370 10249 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.737432 10248 raft_consensus.cc:359] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.740662 10248 raft_consensus.cc:385] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.740424 10249 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.740960 10248 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.741855 10248 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.744246 10248 ts_tablet_manager.cc:1434] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.003s
I20260709 13:16:00.748129 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:00.749215 10131 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.750725 10249 ts_tablet_manager.cc:1434] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 13:16:00.752136 10249 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.757099 10201 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.768978 10249 tablet_bootstrap.cc:654] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.770392 10323 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.770665 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:00.772259 10095 leader_election.cc:343] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.778568 10342 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.779214 10342 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.780272 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:00.780964 10051 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.783175 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.783854 10249 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.784722 10249 ts_tablet_manager.cc:1403] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.003s
W20260709 13:16:00.785033 10020 leader_election.cc:343] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.787369 10342 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.788453 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.789129 10129 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:00.791723 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.792626 10249 raft_consensus.cc:359] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.793553 10273 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.797120 10273 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.797566 10273 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.799917 10399 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.793908 10249 raft_consensus.cc:385] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.796185 10275 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.801340 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:00.801350 10051 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:00.800654 10249 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.804719 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.805797 10254 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.806398 10254 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:00.807505 10254 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.811723 10286 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.812160 10249 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:00.813578 10020 leader_election.cc:343] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.814214 10249 ts_tablet_manager.cc:1434] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:16:00.814791 10250 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:00.815436 10249 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.812391 10286 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.818454 10131 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:00.815435 10250 ts_tablet_manager.cc:1403] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.151s	user 0.026s	sys 0.029s
I20260709 13:16:00.819991 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.822963 10273 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.823904 10275 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.824435 10275 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:00.825377 10273 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.825307 10275 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.826057 10250 raft_consensus.cc:359] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.826731 10250 raft_consensus.cc:385] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.826581 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:00.827167 10250 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
W20260709 13:16:00.828228 10020 leader_election.cc:343] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.829242 10286 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.831437 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:00.833073 10096 leader_election.cc:343] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.833516 10248 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:00.831024 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:00.839798 10131 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.840215 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.841372 10249 tablet_bootstrap.cc:654] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:00.845083 10095 leader_election.cc:343] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.845666 10095 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:00.846992 10254 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:00.842909  9951 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.854169 10131 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:00.855618 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.857220 10248 tablet_bootstrap.cc:654] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.827975 10250 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.860096 10275 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:00.866094 10250 ts_tablet_manager.cc:1434] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:16:00.867821 10250 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:00.889070 10323 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:00.922469 10202 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.929610 10249 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:00.930249 10249 ts_tablet_manager.cc:1403] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.115s	user 0.016s	sys 0.013s
I20260709 13:16:00.940416 10410 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.945361 10410 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.946388 10250 tablet_bootstrap.cc:654] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:00.948223 10286 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.946468 10249 raft_consensus.cc:359] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.949662 10249 raft_consensus.cc:385] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:00.950350 10249 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:00.952621 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.954409  9953 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:00.956305 10170 leader_election.cc:343] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.957690 10323 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:00.964051 10249 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.966713 10249 ts_tablet_manager.cc:1434] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:16:00.950752 10286 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.969228 10286 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:00.971853 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:00.974911 10248 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:00.970610 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.975535 10410 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:00.975827 10095 leader_election.cc:343] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.975597 10053 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:00.976287 10450 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 13:16:00.977013 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.977734 10129 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:16:00.977777 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:00.979458 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:00.980888 10410 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.981247 10249 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:00.981406 10349 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.981523 10410 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.982268 10410 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.982156 10349 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:00.984215 10248 ts_tablet_manager.cc:1403] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.151s	user 0.019s	sys 0.022s
I20260709 13:16:00.985369 10275 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:00.983629 10286 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.986261 10286 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.986661 10286 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:00.987617 10349 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:00.989718 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:00.990929 10129 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:16:00.996305 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:00.996193 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:00.998311 10170 leader_election.cc:343] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:00.998996 10349 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:00.999615 10349 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.000501 10349 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.012935 10248 raft_consensus.cc:359] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.013782 10248 raft_consensus.cc:385] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.013862 10250 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.014665 10250 ts_tablet_manager.cc:1403] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.147s	user 0.016s	sys 0.009s
I20260709 13:16:01.016932 10316 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.017554 10316 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.018606 10250 raft_consensus.cc:359] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.019527 10250 raft_consensus.cc:385] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.019856 10250 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.020085 10316 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:01.022046 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.022238 10250 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.023090 10131 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:01.014138 10248 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.024236 10248 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.021596 10349 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.027366 10248 ts_tablet_manager.cc:1434] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.003s
I20260709 13:16:01.029414 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:01.032066 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:01.032243 10170 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:01.032847 10131 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.032752 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:01.036187 10316 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.037029 10248 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:01.029678 10250 ts_tablet_manager.cc:1434] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
W20260709 13:16:01.038939 10170 leader_election.cc:343] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.036645 10286 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.044185 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:01.051348 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:01.037879 10349 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:01.073442 10249 tablet_bootstrap.cc:654] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.078856 10250 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.028196 10275 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.095626 10286 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.096676 10410 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.097399 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:01.094336 10457 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.099869 10457 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:01.100117 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:01.103734 10248 tablet_bootstrap.cc:654] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.107924 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:01.108289 10095 leader_election.cc:343] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.108753 10129 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:01.107928 10170 leader_election.cc:343] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.115265 10316 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.100961 10457 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.139346 10450 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.157666 10250 tablet_bootstrap.cc:654] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.142704 10129 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:01.165467 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:01.169088 10458 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.169788 10458 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:01.170794 10458 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.174069 10249 tablet_bootstrap.cc:492] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:01.174952 10249 ts_tablet_manager.cc:1403] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.194s	user 0.022s	sys 0.021s
I20260709 13:16:01.182595 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:01.184666 10051 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.189380 10248 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:01.190050 10248 ts_tablet_manager.cc:1403] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.154s	user 0.007s	sys 0.012s
W20260709 13:16:01.202740 10020 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.200619 10410 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:01.205956  9954 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.233433 10248 raft_consensus.cc:359] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.234613 10248 raft_consensus.cc:385] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.235046 10248 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
W20260709 13:16:01.235405 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:01.235834 10248 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.247764 10248 ts_tablet_manager.cc:1434] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.003s
I20260709 13:16:01.251694 10249 raft_consensus.cc:359] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.253844 10249 raft_consensus.cc:385] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.254274 10249 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.255067 10249 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.261808 10051 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:01.265859 10249 ts_tablet_manager.cc:1434] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.003s
I20260709 13:16:01.271874 10248 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:01.273705 10250 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.274310 10250 ts_tablet_manager.cc:1403] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.201s	user 0.042s	sys 0.030s
I20260709 13:16:01.275406 10249 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:01.279392 10250 raft_consensus.cc:359] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.280234 10250 raft_consensus.cc:385] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.280495 10250 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.281107 10250 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.292131 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:01.290633  9954 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:01.293622 10323 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.296072  9954 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:01.298988 10250 ts_tablet_manager.cc:1434] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:16:01.311378 10250 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.311744 10248 tablet_bootstrap.cc:654] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.302958 10323 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:01.313946 10412 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.317833 10323 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.319216 10412 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.322659 10412 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:01.327750 10020 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.345964 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:01.353013 10096 leader_election.cc:343] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.356989 10452 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.357615 10452 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.360131 10452 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.363523 10286 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.366168 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.367914 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.368098 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.368000  9951 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.368860 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:01.372339 10452 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:01.374426 10095 leader_election.cc:343] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.377655 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:01.378330 10249 tablet_bootstrap.cc:654] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:01.375766 10020 leader_election.cc:343] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.378823 10286 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.378901 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:01.381507 10275 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:01.381922 10275 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.382296 10275 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.381047 10250 tablet_bootstrap.cc:654] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:01.398494 10095 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.408778 10275 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.417950 10323 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.418619 10275 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.417999 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:01.419798 10129 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.421082 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:01.418594 10323 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.425318 10323 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:01.425830 10020 leader_election.cc:343] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.427309 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:01.429332 10096 leader_election.cc:343] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.428601 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.430873 10249 tablet_bootstrap.cc:492] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
W20260709 13:16:01.430822 10095 leader_election.cc:343] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.431552 10249 ts_tablet_manager.cc:1403] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.157s	user 0.007s	sys 0.015s
I20260709 13:16:01.431402 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:01.432709 10323 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.437376 10250 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.442409 10250 ts_tablet_manager.cc:1403] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.008s
I20260709 13:16:01.439025 10248 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:01.438597 10249 raft_consensus.cc:359] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.445133 10248 ts_tablet_manager.cc:1403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.180s	user 0.013s	sys 0.007s
I20260709 13:16:01.445391 10249 raft_consensus.cc:385] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.445866 10249 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.446650 10451 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.447254 10451 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.446712 10249 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.450703 10248 raft_consensus.cc:359] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.451632 10248 raft_consensus.cc:385] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.452045 10248 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.452104 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.452917 10130 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:01.453226 10248 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.455497 10249 ts_tablet_manager.cc:1434] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:16:01.456321 10248 ts_tablet_manager.cc:1434] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:16:01.449633 10250 raft_consensus.cc:359] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.456885 10250 raft_consensus.cc:385] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.457321 10250 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.457556 10248 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:01.458447 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:01.458237 10250 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.459906 10452 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:01.460319 10452 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.460717 10452 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.462055 10249 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:01.465159 10248 tablet_bootstrap.cc:654] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.466147 10250 ts_tablet_manager.cc:1434] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:16:01.471657 10249 tablet_bootstrap.cc:654] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.471331 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.472018 10451 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.472353 10202 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:01.477025 10248 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:01.477563 10248 ts_tablet_manager.cc:1403] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
I20260709 13:16:01.442121 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:01.480755 10248 raft_consensus.cc:359] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.481626 10248 raft_consensus.cc:385] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.482185 10248 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.482385 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:01.482977 10248 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.484508 10275 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.485706 10248 ts_tablet_manager.cc:1434] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:16:01.485793 10275 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:01.486923 10250 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.486923 10275 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.493355 10412 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.493995 10412 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.494678 10250 tablet_bootstrap.cc:654] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.493845 10452 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.501302 10412 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.501441 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:01.502336 10202 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.503260 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:01.505995 10096 leader_election.cc:343] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.507639 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.508455 10248 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
W20260709 13:16:01.510113 10095 leader_election.cc:343] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.510839 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:01.512917 10412 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.514775 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:01.523547 10248 tablet_bootstrap.cc:654] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.524035 10129 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.516901 10250 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.531243 10250 ts_tablet_manager.cc:1403] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.003s
I20260709 13:16:01.531245 10452 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.522114 10202 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:01.534593 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:01.535846 10250 raft_consensus.cc:359] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.536329 10452 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.536741 10250 raft_consensus.cc:385] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.536268  9954 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.537305 10250 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.538008 10452 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:01.538157 10250 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.542424 10250 ts_tablet_manager.cc:1434] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:16:01.543740 10250 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.544137 10248 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:01.544634 10248 ts_tablet_manager.cc:1403] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.005s
I20260709 13:16:01.546409 10410 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.547443 10410 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.547592 10458 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:01.553442 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:01.554337 10250 tablet_bootstrap.cc:654] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.540158 10452 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.561621 10410 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:01.561363 10275 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.563599 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.571574 10249 tablet_bootstrap.cc:492] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:01.572250 10249 ts_tablet_manager.cc:1403] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.110s	user 0.040s	sys 0.026s
I20260709 13:16:01.578125 10249 raft_consensus.cc:359] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.578893 10249 raft_consensus.cc:385] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.579233 10249 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.581013 10249 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.584919 10248 raft_consensus.cc:359] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.585846 10248 raft_consensus.cc:385] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.586172 10248 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.587184 10249 ts_tablet_manager.cc:1434] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:16:01.586966 10248 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.608673 10249 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:01.616374 10248 ts_tablet_manager.cc:1434] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.071s	user 0.011s	sys 0.000s
I20260709 13:16:01.628993 10250 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.629664 10250 ts_tablet_manager.cc:1403] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.086s	user 0.009s	sys 0.012s
I20260709 13:16:01.634907 10410 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.635520 10129 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:01.635618 10410 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.636871 10051 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.638244 10410 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:01.641347 10248 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:01.640793  9951 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.646714 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:01.649993 10250 raft_consensus.cc:359] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.655483 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:01.657416 10250 raft_consensus.cc:385] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.657821 10250 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.658371 10410 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.656852 10481 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.660544 10481 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.665241 10481 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.667734 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.669701 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:01.671296 10410 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.675488 10205 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.675873 10250 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.677033 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:01.678328 10250 ts_tablet_manager.cc:1434] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.005s
I20260709 13:16:01.679646 10250 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.675576 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.683786 10249 tablet_bootstrap.cc:654] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.685325 10248 tablet_bootstrap.cc:654] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:01.690121 10096 leader_election.cc:343] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.696460 10410 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:01.702072 10095 leader_election.cc:343] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.702656 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:01.707971 10275 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.712481 10254 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.714120 10399 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.714721 10399 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.717697 10399 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.723284 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.724108 10129 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:01.726372 10250 tablet_bootstrap.cc:654] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.732815 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.737864 10249 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:01.738780 10249 ts_tablet_manager.cc:1403] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.130s	user 0.012s	sys 0.006s
I20260709 13:16:01.745585 10249 raft_consensus.cc:359] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.746430 10249 raft_consensus.cc:385] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.746716 10249 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.747717 10249 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.742763 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:01.776583 10020 leader_election.cc:343] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.778107 10250 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.779982 10250 ts_tablet_manager.cc:1403] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.101s	user 0.018s	sys 0.016s
I20260709 13:16:01.782332 10452 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:01.782853 10452 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.783244 10452 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.786365 10249 ts_tablet_manager.cc:1434] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.001s
I20260709 13:16:01.784262 10250 raft_consensus.cc:359] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.787513 10275 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.787693 10249 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:01.787868 10250 raft_consensus.cc:385] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.788059 10275 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.788390 10250 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.789672 10250 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.790447 10275 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.794565 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.795898 10129 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 1: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:01.798418 10250 ts_tablet_manager.cc:1434] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 13:16:01.799782 10051 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.803448 10203 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.804315 10457 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.798602 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:01.813287 10095 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.816141 10020 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:01.824116 10452 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.799945 10250 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:01.831589 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:01.835755 10202 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.835776 10399 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:01.837481 10457 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:01.826756 10452 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:01.844594 10020 leader_election.cc:343] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.846684 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:01.847772 10131 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:01.858949 10250 tablet_bootstrap.cc:654] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.890169 10131 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:01.893004 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:01.900636 10248 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:01.901249 10248 ts_tablet_manager.cc:1403] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.279s	user 0.048s	sys 0.089s
W20260709 13:16:01.905604 10020 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.905675 10399 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:01.905175 10248 raft_consensus.cc:359] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.906997 10399 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:01.907088 10248 raft_consensus.cc:385] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:01.907593 10248 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:01.908223 10399 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.915879 10249 tablet_bootstrap.cc:654] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:01.915314 10248 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.918159 10248 ts_tablet_manager.cc:1434] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:16:01.920715 10254 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.921852 10254 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.929932 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:01.932363 10096 leader_election.cc:343] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.936187 10248 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:01.936475 10275 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.939512 10275 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:01.941928 10275 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.943617 10254 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:01.944514 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:01.944653 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:01.945788 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:01.950909 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:01.960681 10275 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:01.991894 10095 leader_election.cc:343] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.992808 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:01.997264 10250 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:01.997931 10250 ts_tablet_manager.cc:1403] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.198s	user 0.035s	sys 0.013s
I20260709 13:16:02.000692 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.001405 10412 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:01.999455 10286 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:02.006747 10020 leader_election.cc:343] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:01.970202 10275 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.019079 10249 tablet_bootstrap.cc:492] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.019656 10249 ts_tablet_manager.cc:1403] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.232s	user 0.035s	sys 0.006s
I20260709 13:16:02.021998 10250 raft_consensus.cc:359] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.022522 10249 raft_consensus.cc:359] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.019788 10275 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.023288 10250 raft_consensus.cc:385] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.024585 10250 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.026081 10250 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.026947 10249 raft_consensus.cc:385] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.035622 10249 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.036015 10248 tablet_bootstrap.cc:654] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.036721 10249 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.039299 10249 ts_tablet_manager.cc:1434] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:16:02.030133  9954 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:02.041909 10249 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.037676 10250 ts_tablet_manager.cc:1434] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:16:02.049732 10250 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.056668 10250 tablet_bootstrap.cc:654] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.061743 10275 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.070729 10275 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.071933 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:02.072757 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:02.073378 10129 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:02.074059 10020 leader_election.cc:343] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.062389 10249 tablet_bootstrap.cc:654] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.044306 10248 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.076548 10248 ts_tablet_manager.cc:1403] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.000s
I20260709 13:16:02.080703 10248 raft_consensus.cc:359] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.081651 10248 raft_consensus.cc:385] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.082000 10248 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.082827 10248 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.085143 10248 ts_tablet_manager.cc:1434] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:02.085017 10275 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:02.088033 10248 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.095214 10248 tablet_bootstrap.cc:654] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.099287 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:02.104099 10250 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.104676 10250 ts_tablet_manager.cc:1403] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.004s
I20260709 13:16:02.109128 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.115512 10452 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:02.116389 10452 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:02.117864 10452 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.116597 10248 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.123845 10248 ts_tablet_manager.cc:1403] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.016s
I20260709 13:16:02.109063 10250 raft_consensus.cc:359] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.131554 10250 raft_consensus.cc:385] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.131945 10250 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.128170 10248 raft_consensus.cc:359] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.132712 10250 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.134299 10248 raft_consensus.cc:385] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.135341 10275 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:02.136679 10250 ts_tablet_manager.cc:1434] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.003s
I20260709 13:16:02.137845 10250 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.147286 10250 tablet_bootstrap.cc:654] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.135416 10248 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.148550 10254 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.149070 10254 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.149350 10248 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.152004 10249 tablet_bootstrap.cc:492] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.153672 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.158797 10249 ts_tablet_manager.cc:1403] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.117s	user 0.020s	sys 0.007s
W20260709 13:16:02.159734 10096 leader_election.cc:343] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.162770 10249 raft_consensus.cc:359] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.153767 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.163620 10249 raft_consensus.cc:385] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.164556 10249 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.154585 10254 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.165423 10399 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:02.165331 10249 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.169705 10250 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.170337 10250 ts_tablet_manager.cc:1403] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.004s
W20260709 13:16:02.172377 10095 leader_election.cc:343] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.173038 10095 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:02.172699  9954 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:02.174371 10250 raft_consensus.cc:359] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.175163 10250 raft_consensus.cc:385] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.177856 10481 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.179368 10249 ts_tablet_manager.cc:1434] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:16:02.182879 10248 ts_tablet_manager.cc:1434] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 13:16:02.184494 10248 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.175457 10250 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.190837 10250 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.192893 10250 ts_tablet_manager.cc:1434] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:16:02.195497 10249 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.206525 10248 tablet_bootstrap.cc:654] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.194152 10250 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.212401 10249 tablet_bootstrap.cc:654] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.213371 10254 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.213963 10254 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.219311 10254 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.221609 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.232514 10250 tablet_bootstrap.cc:654] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.226363 10248 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
W20260709 13:16:02.233538 10096 leader_election.cc:343] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.234288 10248 ts_tablet_manager.cc:1403] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.005s
I20260709 13:16:02.223038 10249 tablet_bootstrap.cc:492] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.235562 10249 ts_tablet_manager.cc:1403] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.000s
I20260709 13:16:02.237607 10248 raft_consensus.cc:359] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.238481 10248 raft_consensus.cc:385] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.238775 10248 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.238855 10249 raft_consensus.cc:359] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.227319 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.241560 10249 raft_consensus.cc:385] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.241938 10249 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.243093 10249 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.249215 10249 ts_tablet_manager.cc:1434] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:16:02.250931 10249 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.239645 10248 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.253105 10248 ts_tablet_manager.cc:1434] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 13:16:02.254266 10248 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.261132 10248 tablet_bootstrap.cc:654] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.277503 10410 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:02.278594 10095 leader_election.cc:343] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.278352 10410 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.286180 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.286341 10410 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:02.292150 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.293114 10131 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:02.295789 10399 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.296425 10399 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.299058 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:02.298905 10399 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.301816 10410 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.305382 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:02.312534 10020 leader_election.cc:343] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.282588 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:02.319893 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:02.319818 10286 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.323539 10410 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.324496 10248 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.325021 10248 ts_tablet_manager.cc:1403] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.023s
I20260709 13:16:02.325879 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.326190 10250 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.326633 10249 tablet_bootstrap.cc:654] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.326771 10250 ts_tablet_manager.cc:1403] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.133s	user 0.023s	sys 0.026s
I20260709 13:16:02.328594 10248 raft_consensus.cc:359] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.329418 10248 raft_consensus.cc:385] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.329705 10248 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.326577 10131 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.330469 10248 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.332418 10248 ts_tablet_manager.cc:1434] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:02.330061 10250 raft_consensus.cc:359] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.339284 10248 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.335528 10250 raft_consensus.cc:385] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.346495 10250 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.346539 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.347419 10250 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.351150 10451 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.351886 10451 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.352365 10451 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.358608 10250 ts_tablet_manager.cc:1434] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:16:02.370042 10250 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.374383 10410 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.373500 10451 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.375047 10410 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.379592 10203 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.381067 10410 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:02.382421 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.383401 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.388916 10254 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.389508 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:02.389492 10254 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.390305 10129 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.391906 10248 tablet_bootstrap.cc:654] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.393011 10249 tablet_bootstrap.cc:492] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.393122 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:02.394333 10249 ts_tablet_manager.cc:1403] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.144s	user 0.023s	sys 0.032s
I20260709 13:16:02.399785 10254 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.401479 10452 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.402612 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.403029 10452 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.404582 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:02.404775 10095 leader_election.cc:343] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.405512 10452 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.406076 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.406458 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.406843 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:02.408259 10096 leader_election.cc:343] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.408865 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:02.408990 10020 leader_election.cc:343] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.409896 10249 raft_consensus.cc:359] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.407220 10131 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.410595 10254 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.410668 10249 raft_consensus.cc:385] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.411391 10249 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.412748 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.413749 10254 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.414445 10254 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.415174 10452 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.415652 10452 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.416091 10452 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.416416 10250 tablet_bootstrap.cc:654] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.417392 10451 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.424075 10254 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.420197 10129 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:02.418857 10249 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.425402 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.429191 10205 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.430634 10249 ts_tablet_manager.cc:1434] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 13:16:02.431810 10249 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.434482 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.435671 10458 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.436285 10458 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.437433 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:02.438555 10458 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:02.439498 10342 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:02.439570 10451 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:02.440167 10342 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:02.440582 10205 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:02.440874 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.441064 10342 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.441839 10254 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:02.438494 10095 leader_election.cc:343] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.442301 10254 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:02.443011 10096 leader_election.cc:343] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.444275 10254 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.445125 10096 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:02.448109 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.449448 10286 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.450232 10286 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.450492 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.450848 10286 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:02.452569 10095 leader_election.cc:343] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.453754 10286 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.450233 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:02.456905 10458 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.458981 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:02.460913 10095 leader_election.cc:343] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.463893 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:02.465691 10096 leader_election.cc:343] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.466187 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:02.467495 10254 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.467970 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.468699 10054 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:02.448405 10452 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.469792 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.471385 10452 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.471967 10481 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.472500 10481 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.472844 10481 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.474325  9953 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:02.482234 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:02.483770 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:02.484743 10131 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.492581 10131 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:02.488900 10410 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:02.516968 10020 leader_election.cc:343] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.509428 10249 tablet_bootstrap.cc:654] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.525290 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.526365 10399 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:02.526986 10399 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:02.528021 10399 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.529528 10204 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.531049 10481 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:02.547906 10020 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:02.556830 10481 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.562453 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:02.557410 10250 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.560097 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:02.564082 10053 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.564383 10248 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.564934 10248 ts_tablet_manager.cc:1403] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.226s	user 0.027s	sys 0.026s
I20260709 13:16:02.563979 10250 ts_tablet_manager.cc:1403] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.194s	user 0.017s	sys 0.024s
W20260709 13:16:02.568418 10095 leader_election.cc:343] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.573524 10249 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.574142 10249 ts_tablet_manager.cc:1403] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.143s	user 0.013s	sys 0.010s
I20260709 13:16:02.564105  9954 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:02.575156 10250 raft_consensus.cc:359] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.576006 10250 raft_consensus.cc:385] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.576378 10250 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.577251 10250 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.579020 10250 ts_tablet_manager.cc:1434] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.000s
I20260709 13:16:02.578797 10053 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:02.580849 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.582405 10249 raft_consensus.cc:359] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.583199 10249 raft_consensus.cc:385] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.583499 10249 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.585041 10250 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.589078 10249 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.594880 10249 ts_tablet_manager.cc:1434] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.003s
I20260709 13:16:02.567734 10399 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:16:02.596065 10481 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:02.598217 10481 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:02.599164 10481 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.607676 10249 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.611176 10248 raft_consensus.cc:359] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.619200 10248 raft_consensus.cc:385] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.619578 10248 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.620278 10248 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.625859 10248 ts_tablet_manager.cc:1434] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:16:02.627759 10248 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.642285 10250 tablet_bootstrap.cc:654] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.669691 10249 tablet_bootstrap.cc:654] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.671965 10248 tablet_bootstrap.cc:654] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.681245 10450 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:02.683812 10450 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.684360 10450 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.686574 10450 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.690739 10250 tablet_bootstrap.cc:492] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.691493 10250 ts_tablet_manager.cc:1403] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.004s
I20260709 13:16:02.694294 10250 raft_consensus.cc:359] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.695983 10250 raft_consensus.cc:385] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.696451 10250 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.697541 10248 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.697458 10250 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.698122 10248 ts_tablet_manager.cc:1403] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.007s
I20260709 13:16:02.694419 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.701725 10250 ts_tablet_manager.cc:1434] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:16:02.702195 10248 raft_consensus.cc:359] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.703084 10248 raft_consensus.cc:385] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:02.703028 10020 leader_election.cc:343] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.703572 10248 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.704329 10248 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.705303  9954 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:02.708431 10248 ts_tablet_manager.cc:1434] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:16:02.709367 10250 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.709697 10248 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.714879 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.715785 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.718284 10250 tablet_bootstrap.cc:654] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.722731 10481 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.723627 10481 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.726719 10481 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.729197 10248 tablet_bootstrap.cc:654] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.738173 10250 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.738803 10250 ts_tablet_manager.cc:1403] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.007s
I20260709 13:16:02.741890 10250 raft_consensus.cc:359] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.742659 10250 raft_consensus.cc:385] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.742941 10250 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.743631 10250 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.743970 10399 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.744786 10399 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.746044 10250 ts_tablet_manager.cc:1434] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:16:02.747246 10250 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.752516 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:02.754971 10020 leader_election.cc:343] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.757051 10250 tablet_bootstrap.cc:654] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.758538 10399 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.764505 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:02.768293 10096 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.784451 10019 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.785598 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.785892 10516 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.786736 10516 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.787259 10516 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.795506 10481 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.796344 10481 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.801266 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.801246 10481 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:02.803903 10096 leader_election.cc:343] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:02.816594 10095 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.818291 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:02.818786 10249 tablet_bootstrap.cc:492] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.819403 10249 ts_tablet_manager.cc:1403] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.223s	user 0.048s	sys 0.019s
I20260709 13:16:02.819880 10481 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.822377 10249 raft_consensus.cc:359] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.823196 10249 raft_consensus.cc:385] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.822992 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.823750 10129 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.824529 10249 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.825368 10249 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.832281 10249 ts_tablet_manager.cc:1434] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:16:02.833423 10249 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.836829 10399 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.837491 10399 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.839757 10248 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.840416 10248 ts_tablet_manager.cc:1403] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.131s	user 0.030s	sys 0.026s
I20260709 13:16:02.843752 10248 raft_consensus.cc:359] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.844568 10248 raft_consensus.cc:385] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.844858 10248 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.842651 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.845539 10248 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.846041 10203 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.848026 10248 ts_tablet_manager.cc:1434] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:02.847976 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:02.849081 10450 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.849341 10248 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.849551 10450 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.849931 10450 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.855312 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:02.857524 10399 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.857733 10249 tablet_bootstrap.cc:654] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.864450 10248 tablet_bootstrap.cc:654] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.868165 10410 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.868762 10410 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.870965 10410 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:02.872440 10249 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:02.873235 10249 ts_tablet_manager.cc:1403] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.004s
I20260709 13:16:02.873669 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.876161 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:02.877271 10249 raft_consensus.cc:359] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.878403 10249 raft_consensus.cc:385] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.878186 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:02.878846 10249 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.879913 10249 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.881036 10410 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:02.884793 10249 ts_tablet_manager.cc:1434] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:16:02.886066 10249 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:02.886237 10248 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.886997 10248 ts_tablet_manager.cc:1403] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.013s
I20260709 13:16:02.888877 10019 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:02.890012 10399 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:02.890413 10399 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.890767 10399 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.890856 10248 raft_consensus.cc:359] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.891785 10248 raft_consensus.cc:385] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.892128 10248 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.892915 10248 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.894552 10516 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.897107 10250 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:02.897408 10516 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.897714 10250 ts_tablet_manager.cc:1403] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.151s	user 0.055s	sys 0.043s
I20260709 13:16:02.900899 10250 raft_consensus.cc:359] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.902323 10250 raft_consensus.cc:385] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.902697 10250 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.908744 10250 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.911571 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:02.916210 10399 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:02.915663 10020 leader_election.cc:343] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.919693 10399 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.921123 10248 ts_tablet_manager.cc:1434] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.001s
I20260709 13:16:02.922480 10248 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:02.924372 10250 ts_tablet_manager.cc:1434] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.001s
I20260709 13:16:02.924614 10249 tablet_bootstrap.cc:654] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.925631 10250 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:02.929365 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:02.930193 10203 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.933087 10204 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.893290 10450 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.936333 10250 tablet_bootstrap.cc:654] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.941215 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:02.950124 10275 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:02.950614 10128 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:02.942678 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:02.951726 10129 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.942821 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:02.955473 10131 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:02.957402 10342 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.957988 10342 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.945150 10248 tablet_bootstrap.cc:654] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:02.960227 10342 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.960536 10450 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:02.965714 10129 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
W20260709 13:16:02.946149 10095 leader_election.cc:343] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.971074 10131 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:02.982391 10248 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:02.983071 10248 ts_tablet_manager.cc:1403] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.006s
I20260709 13:16:02.986405 10248 raft_consensus.cc:359] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.987753 10248 raft_consensus.cc:385] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:02.988073 10248 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:02.989030 10248 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.993450 10248 ts_tablet_manager.cc:1434] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:16:02.975499 10250 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.000605 10250 ts_tablet_manager.cc:1403] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.006s
I20260709 13:16:02.979211 10095 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:02.978248 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.001993 10286 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:03.002161 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56053.4us
I20260709 13:16:03.008430 10248 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
W20260709 13:16:03.007875 10020 leader_election.cc:343] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:02.978681 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:03.011289 10202 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.013247 10250 raft_consensus.cc:359] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.014092 10250 raft_consensus.cc:385] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.014417 10250 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.015419 10250 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:02.975066 10203 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.024101 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:03.025242 10399 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.025699 10248 tablet_bootstrap.cc:654] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.025791 10399 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.026897 10399 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.027801 10250 ts_tablet_manager.cc:1434] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:16:03.029129 10250 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:03.031659 10019 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.034861 10452 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.034745 10529 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:03.035602 10452 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.036573 10452 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.037781 10249 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.038319 10249 ts_tablet_manager.cc:1403] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.003s
I20260709 13:16:03.041437 10249 raft_consensus.cc:359] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.042315 10249 raft_consensus.cc:385] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.042610 10249 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.045723 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.050442 10131 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
W20260709 13:16:03.057742 10020 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.059209 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.060034 10131 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.056867 10249 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.052430 10202 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.064541 10250 tablet_bootstrap.cc:654] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.073516 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:03.074899 10450 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.075644 10450 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.076838 10249 ts_tablet_manager.cc:1434] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:16:03.076859 10450 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.078514 10529 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.079169 10529 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.081004 10249 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:03.084699 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:03.090034 10096 leader_election.cc:343] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.082500 10529 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.102066 10529 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.103626 10249 tablet_bootstrap.cc:654] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.104503  9952 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.102778 10529 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.106568  9952 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.110737 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:03.112485 10096 leader_election.cc:343] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.140957 10452 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:03.151749 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.152704 10450 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:03.153542 10451 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.154276 10451 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.154779 10451 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.155701 10529 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.157262 10249 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.157862 10249 ts_tablet_manager.cc:1403] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.002s
I20260709 13:16:03.161142 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.162261 10249 raft_consensus.cc:359] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.163245 10249 raft_consensus.cc:385] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.163901 10249 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.164906 10249 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.168210 10249 ts_tablet_manager.cc:1434] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:16:03.169493 10249 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:03.169265  9952 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:03.175537 10231 tablet.cc:2406] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:03.178283 10249 tablet_bootstrap.cc:654] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.183497 10451 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.187574 10451 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.188329 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:03.190760 10249 tablet_bootstrap.cc:492] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.194644 10249 ts_tablet_manager.cc:1403] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.012s
W20260709 13:16:03.195755 10020 leader_election.cc:343] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.211489 10249 raft_consensus.cc:359] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.212457 10249 raft_consensus.cc:385] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.212742 10249 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.213498 10249 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.217808 10249 ts_tablet_manager.cc:1434] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:16:03.219039 10249 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:03.223081 10451 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.223742 10451 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.226707 10249 tablet_bootstrap.cc:654] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.227278 10250 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.227280 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.227962 10250 ts_tablet_manager.cc:1403] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.199s	user 0.018s	sys 0.020s
I20260709 13:16:03.228168 10201 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.229651 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:03.230510 10451 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.230770 10248 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:03.230530 10399 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.230965 10250 raft_consensus.cc:359] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.231367 10248 ts_tablet_manager.cc:1403] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.232s	user 0.032s	sys 0.058s
I20260709 13:16:03.231235 10399 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.231775 10250 raft_consensus.cc:385] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.232158 10399 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.232152 10250 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.236394 10248 raft_consensus.cc:359] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.237622 10248 raft_consensus.cc:385] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.238034 10248 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.238968 10248 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.242169 10399 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.246392 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:03.247087 10201 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.257946 10248 ts_tablet_manager.cc:1434] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.003s
I20260709 13:16:03.235961 10250 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.259430 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.260077 10248 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:03.260143 10129 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.265407 10250 ts_tablet_manager.cc:1434] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:16:03.269843 10129 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.270493 10250 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:03.261786 10201 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.272503 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:03.273452 10450 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.274174 10450 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.275056 10450 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.238911 10131 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.287473 10399 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.291288 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.292235 10129 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.296283 10248 tablet_bootstrap.cc:654] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.298530 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.300460 10128 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.309051 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.314002 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:03.320396 10095 leader_election.cc:343] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.324648  9952 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.320358 10450 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.327297 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:03.328430 10254 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:03.330327 10450 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.331913 10450 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.341398 10249 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.342002 10249 ts_tablet_manager.cc:1403] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.123s	user 0.019s	sys 0.028s
I20260709 13:16:03.345183 10286 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.345788 10286 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.346252 10249 raft_consensus.cc:359] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.347081 10249 raft_consensus.cc:385] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.347421 10249 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.348284 10249 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.349185 10286 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.350451 10128 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.350713 10249 ts_tablet_manager.cc:1434] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.007s
I20260709 13:16:03.351945 10248 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:03.352567 10249 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:16:03.353824 10095 leader_election.cc:343] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.352598 10248 ts_tablet_manager.cc:1403] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.004s
I20260709 13:16:03.354391 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:03.357654 10254 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:03.358925 10095 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:03.359675 10250 tablet_bootstrap.cc:654] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.361541 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:03.378039 10096 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.358417 10248 raft_consensus.cc:359] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:03.378216 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:03.378731 10248 raft_consensus.cc:385] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.365126 10249 tablet_bootstrap.cc:654] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.379206 10248 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.380348 10248 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.381459 10250 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.382077 10250 ts_tablet_manager.cc:1403] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.112s	user 0.009s	sys 0.002s
I20260709 13:16:03.382825 10248 ts_tablet_manager.cc:1434] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:16:03.384678 10248 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:03.385794 10250 raft_consensus.cc:359] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.386787 10250 raft_consensus.cc:385] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.387146 10250 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.387478 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.387909 10250 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.390331 10250 ts_tablet_manager.cc:1434] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:16:03.393812 10250 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:03.395035 10450 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.395668 10450 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.398985 10450 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.401995 10250 tablet_bootstrap.cc:654] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.402935  9952 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.409710 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:03.410164 10095 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.411207 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:03.412341 10254 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:03.414429 10020 leader_election.cc:343] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.418116 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.418838 10248 tablet_bootstrap.cc:654] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.418962 10129 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.420444 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.421046 10249 tablet_bootstrap.cc:492] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.422111 10450 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.422602 10450 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.422640 10249 ts_tablet_manager.cc:1403] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.020s
I20260709 13:16:03.423043 10450 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.425033 10458 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.425889 10458 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.429397 10250 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.430105 10250 ts_tablet_manager.cc:1403] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.005s
I20260709 13:16:03.430527 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:03.436854 10458 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:03.436115 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.440217 10129 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:03.441815 10250 raft_consensus.cc:359] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.442214 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:03.442684 10250 raft_consensus.cc:385] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.443064 10250 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.444892 10451 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.445027 10410 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.445533 10451 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.449079 10451 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.450946 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.443917 10250 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.454595 10250 ts_tablet_manager.cc:1434] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
W20260709 13:16:03.455101 10020 leader_election.cc:343] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.455955 10250 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:03.462535 10342 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:03.463207 10250 tablet_bootstrap.cc:654] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.481071 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.481854 10129 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.484535 10250 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.485105 10250 ts_tablet_manager.cc:1403] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.012s
I20260709 13:16:03.487715 10410 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:03.482990 10450 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.488845 10250 raft_consensus.cc:359] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.502379 10250 raft_consensus.cc:385] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.502735 10250 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.503501 10250 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.505336 10250 ts_tablet_manager.cc:1434] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:16:03.507544 10250 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:03.517443 10450 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.518846 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:03.519701 10250 tablet_bootstrap.cc:654] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:03.520692 10020 leader_election.cc:343] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.491271 10249 raft_consensus.cc:359] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.499933 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.525820 10451 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.526288 10451 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.526643 10451 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.531327 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.532109 10129 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.541396 10249 raft_consensus.cc:385] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.541885 10249 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.546003 10410 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.546924 10452 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:03.554174 10451 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.558975 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:03.561120 10020 leader_election.cc:343] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.562004 10452 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.563287 10452 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.565899 10410 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.568548 10452 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.570571 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.571548 10129 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.571765 10203 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.573724 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:03.575445 10560 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.576746 10560 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.551759 10249 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.575999 10451 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.578382 10458 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.578961 10458 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.584564 10458 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:03.584697 10560 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.585012 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:03.611171 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 83771.1us
I20260709 13:16:03.612257 10128 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:03.612866 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.613147 10275 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:03.614521 10275 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.615012 10275 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.615779 10275 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.594385 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:03.617702 10410 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:03.619215 10249 ts_tablet_manager.cc:1434] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.196s	user 0.008s	sys 0.000s
I20260709 13:16:03.621634 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:03.623719 10555 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.624281 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:03.626648 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.630951 10128 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:16:03.633220 10168 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:03.634683 10410 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.635320 10410 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.635784 10410 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.626309 10560 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.655375 10053 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:03.656455 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:03.659148 10249 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:03.665027 10286 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.665834 10286 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.656229 10560 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.670257 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.671339 10131 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.673856 10458 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.674481 10458 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.676445 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.677654 10129 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.676915 10458 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:03.678905 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.687072 10249 tablet_bootstrap.cc:654] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:03.681150 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:03.694584 10248 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:03.695163 10248 ts_tablet_manager.cc:1403] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.311s	user 0.056s	sys 0.062s
I20260709 13:16:03.695211 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:03.661090 10555 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:03.701646 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:03.702785 10555 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:03.716408 10129 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:03.716173 10275 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:03.721429 10286 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.716817  9952 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:03.742152 10248 raft_consensus.cc:359] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.743471 10248 raft_consensus.cc:385] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.744001 10248 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.747679 10200 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.750545 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:03.740365 10516 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:16:03.747046 10248 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.759963 10450 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.799463 10450 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.800096 10451 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:03.800699 10451 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:03.812572 10451 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:03.822012 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102396us
I20260709 13:16:03.812629 10450 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:03.762439 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:03.822952 10096 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.835454 10248 ts_tablet_manager.cc:1434] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.140s	user 0.004s	sys 0.009s
I20260709 13:16:03.819260 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:03.852876 10248 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
W20260709 13:16:03.852828 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93529.6us
I20260709 13:16:03.858274 10250 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:03.858888 10250 ts_tablet_manager.cc:1403] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.352s	user 0.069s	sys 0.079s
W20260709 13:16:03.861666 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46892.8us
I20260709 13:16:03.867251 10399 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:03.868804 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:03.869690 10130 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:03.870844 10249 tablet_bootstrap.cc:492] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:03.871471 10249 ts_tablet_manager.cc:1403] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.220s	user 0.026s	sys 0.030s
I20260709 13:16:03.863173 10124 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.896464 10249 raft_consensus.cc:359] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.904744 10249 raft_consensus.cc:385] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:03.905355 10249 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:03.909893 10249 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:03.879837 10019 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:03.923463 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43751.5us
I20260709 13:16:03.927345 10249 ts_tablet_manager.cc:1434] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.004s
I20260709 13:16:03.875887 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:03.935561 10566 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:03.936153 10566 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:03.925560 10095 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:03.936558 10566 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:03.937461 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:03.938198 10286 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:03.948024 10020 leader_election.cc:343] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:03.994602 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67550.6us
W20260709 13:16:03.998955 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60291.9us
W20260709 13:16:03.999833 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49852.8us
I20260709 13:16:04.205281 10574 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.206144 10574 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.232746 10568 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.233486 10568 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.048328 10410 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.296047 10205 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 1: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:04.305474 10577 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.307312 10575 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.336s
I20260709 13:16:04.339277 10250 raft_consensus.cc:359] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.340541 10250 raft_consensus.cc:385] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:04.340938 10250 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:04.306447 10577 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.359465 10250 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.494163 10250 ts_tablet_manager.cc:1434] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.635s	user 0.011s	sys 0.012s
I20260709 13:16:04.447738 10574 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:04.494621 10250 ts_tablet_manager.cc:1467] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet startup took 635ms
I20260709 13:16:04.587281 10248 tablet_bootstrap.cc:654] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:04.647702 10568 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:04.649372 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 98041.6us
W20260709 13:16:04.662168 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:16:04.674844 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.012s	sys 0.010s
I20260709 13:16:04.653147 10410 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:04.675348 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.013s	sys 0.010s
I20260709 13:16:04.680560 10577 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:04.681232 10249 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:16:04.554374 10250 ts_tablet_manager.cc:1469] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Trace:
0709 13:15:56.997233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.997428 (+   195us) service_pool.cc:224] Handling call
0709 13:15:56.999588 (+  2160us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:15:56.999703 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:15:57.063078 (+ 63375us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:57.101631 (+ 38553us) tablet_replica.cc:206] Creating consensus instance
0709 13:15:57.102184 (+   553us) inbound_call.cc:177] Queueing success response
0709 13:16:03.506923 (+6404739us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:16:03.859229 (+352306us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:16:04.043887 (+184658us) tablet_replica.cc:261] Starting instrumentation
0709 13:16:04.046987 (+  3100us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":97469,"raft.queue_time_us":116,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":35724,"tablet-open.queue_time_us":6404879}
W20260709 13:16:04.696413 10566 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.759s	user 0.008s	sys 0.004s
I20260709 13:16:04.700470 10250 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:04.701015 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:04.707690 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:04.699153 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:04.714783 10054 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:04.704320 10566 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.712412 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:04.725881 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.022s	sys 0.008s
W20260709 13:16:04.726351 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.022s	sys 0.008s
W20260709 13:16:04.726651 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 672026us
W20260709 13:16:04.729446 10124 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.002s
I20260709 13:16:04.729970 10124 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:04.733444 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 197) took 1077 ms. Trace:
I20260709 13:16:04.734114 10205 rpcz_store.cc:276] 0709 13:16:03.655921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.656077 (+   156us) service_pool.cc:224] Handling call
0709 13:16:04.733103 (+1077026us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:04.733301 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468140928}
I20260709 13:16:04.730688 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 204) took 1031 ms. Trace:
I20260709 13:16:04.753579 10566 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:04.736821 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:04.735383 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:04.751227 10249 tablet_bootstrap.cc:654] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:04.757947 10558 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:04.766616 10555 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:04.773940 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:04.776190 10342 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:04.776633 10451 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:04.809716 10020 leader_election.cc:343] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:04.811216 10020 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:04.812980 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:04.814236 10056 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:04.816116 10201 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:04.757020 10124 rpcz_store.cc:276] 0709 13:16:03.699239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.750244 (+ 51005us) service_pool.cc:224] Handling call
0709 13:16:04.730573 (+980329us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:04.780754 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45482.6us
W20260709 13:16:04.845419 10020 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:04.856221 10248 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:04.857043 10248 ts_tablet_manager.cc:1403] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 1.011s	user 0.024s	sys 0.018s
I20260709 13:16:04.862923 10249 tablet_bootstrap.cc:492] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:04.863562 10249 ts_tablet_manager.cc:1403] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.183s	user 0.039s	sys 0.018s
I20260709 13:16:04.873095 10458 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:04.873710 10458 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.888738 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:04.877568 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.028s	sys 0.019s
I20260709 13:16:04.909225 10529 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:04.927965 10529 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.929528 10529 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:04.922973 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.030s	sys 0.020s
I20260709 13:16:04.938774 10555 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:04.944350 10560 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:04.947150 10593 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:16:04.967168 10570 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:04.968017 10570 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.969236 10560 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.965751 10555 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:04.985816 10555 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:04.961220 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:04.996220 10458 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.083863  9952 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 95) took 1135 ms. Trace:
I20260709 13:16:05.091655  9954 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:05.127974 10586 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.128585 10458 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.128793 10586 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.129067 10458 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.131042 10586 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.137310 10286 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.138022 10286 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.084512  9952 rpcz_store.cc:276] 0709 13:16:03.932798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.932997 (+   199us) service_pool.cc:224] Handling call
0709 13:16:05.067757 (+1134760us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:16:05.149852 10019 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:05.156687 10589 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.288s	user 0.000s	sys 0.002s
I20260709 13:16:05.157085 10249 raft_consensus.cc:359] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.157899 10249 raft_consensus.cc:385] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:05.158156 10249 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:05.159013 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.158907 10249 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.166322 10286 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.169803 10254 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.170763 10254 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.173527 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.158792 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 181) took 1424 ms. Trace:
I20260709 13:16:05.148100 10560 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.151481 10586 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.151146 10342 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.202111 10248 raft_consensus.cc:359] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.207492 10248 raft_consensus.cc:385] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:05.208050 10248 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:05.209115 10248 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.220861 10249 ts_tablet_manager.cc:1434] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.357s	user 0.018s	sys 0.055s
I20260709 13:16:05.248256 10254 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.248842 10254 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.249296 10254 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.171940 10399 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
W20260709 13:16:05.268025 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 97916.5us
I20260709 13:16:05.178221 10592 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.295858 10248 ts_tablet_manager.cc:1434] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.438s	user 0.011s	sys 0.005s
I20260709 13:16:05.213531 10342 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.216303 10583 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.312295 10254 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:05.320709 10020 leader_election.cc:343] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:05.255518 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:05.213964 10586 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.334687 10250 tablet_bootstrap.cc:654] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:05.330605 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53439.9us
I20260709 13:16:05.312179 10582 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.308349 10573 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:05.347441 10248 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:05.361080 10249 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:05.339659 10560 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.342767 10580 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.346285 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:05.345892 10597 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.336282 10583 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.327807 10592 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.375329 10573 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:05.354918 10582 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.401103 10578 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.369220 10560 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.406322 10578 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.383944 10580 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.386005 10597 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:05.642728 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 270003us
I20260709 13:16:05.181766 10200 rpcz_store.cc:276] 0709 13:16:03.734659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.735831 (+  1172us) service_pool.cc:224] Handling call
0709 13:16:03.740740 (+  4909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.984650 (+243910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.985517 (+   867us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.728103 (+742586us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:16:04.728584 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.767404 (+ 38820us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:04.865039 (+ 97635us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.893043 (+ 28004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.951920 (+ 58877us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.158348 (+206428us) raft_consensus.cc:1685] finished
0709 13:16:05.158505 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.158691 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":413606,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":4039,"raft.queue_time_us":30940,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":22890,"spinlock_wait_cycles":2557824,"thread_start_us":130810,"threads_started":2,"wal-append.queue_time_us":98293}
I20260709 13:16:05.409482 10600 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:05.495445 10594 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.181s	user 0.000s	sys 0.002s
I20260709 13:16:05.414456 10581 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:05.446875 10560 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.656472 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.467020 10481 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.618257 10586 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.656880 10458 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.566377 10601 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.517155 10516 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.661877 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.521034 10342 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.459189 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:05.638895 10608 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.451202 10602 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.416918 10555 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.673179 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.460021 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:05.705350 10555 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.695802 10586 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.448669 10583 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:05.746492 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87891.5us
I20260709 13:16:05.680426 10592 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.459491 10573 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.705088 10342 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.699069 10605 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.705514 10582 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.781878 10342 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.696060 10481 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.699869 10516 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.664486 10600 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.799161 10342 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.775111 10583 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.712821 10601 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.815039 10516 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.815171 10481 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.817344 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:05.784052 10575 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:05.783593 10592 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.783924 10202 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:05.851857 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 232) took 1047 ms. Trace:
I20260709 13:16:05.853322 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:05.855195 10580 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.757772 10586 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.806144 10605 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.819345 10458 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.799368 10611 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:05.810652 10606 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.865662 10203 rpcz_store.cc:276] 0709 13:16:04.805047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.851720 (+ 46673us) service_pool.cc:224] Handling call
0709 13:16:04.852124 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.854301 (+  2177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.854978 (+   677us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.858018 (+  3040us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.892498 (+ 34480us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:16:04.892546 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.892628 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.851532 (+958904us) raft_consensus.cc:1685] finished
0709 13:16:05.851561 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.851765 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":938,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"thread_start_us":34084,"threads_started":1,"wal-append.queue_time_us":345419}
I20260709 13:16:05.865712 10481 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.697821 10581 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.727346 10602 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:05.882699 10529 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.948s	user 0.019s	sys 0.000s
I20260709 13:16:05.874234 10605 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.874902 10127 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:05.858356 10583 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.882977 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 203) took 1044 ms. Trace:
I20260709 13:16:05.874182 10586 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.885747 10578 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.713471 10608 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.875813 10592 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.883569 10458 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.875526 10575 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.907052 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:05.927979 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:05.892740 10481 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.904070 10529 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.923084 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:05.924422 10581 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.917882 10610 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.931591 10580 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.917496 10602 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.934126 10578 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.914351 10586 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.951629 10248 tablet_bootstrap.cc:654] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:05.902622 10611 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.938817 10555 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:05.903674 10606 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.939175 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 205) took 1073 ms. Trace:
W20260709 13:16:05.945529 10609 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:16:05.945367 10614 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.970171 10614 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.970847 10608 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.939551 10597 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.968061 10249 tablet_bootstrap.cc:654] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:05.942008 10570 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.969007 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 233) took 1160 ms. Trace:
I20260709 13:16:05.943877 10621 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.947335 10607 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.949918 10600 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.982479 10481 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.910221 10056 rpcz_store.cc:276] 0709 13:16:04.857970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.861563 (+  3593us) service_pool.cc:224] Handling call
0709 13:16:05.710785 (+849222us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.758089 (+ 47304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.804126 (+ 46037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.806406 (+  2280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.810092 (+  3686us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:05.836709 (+ 26617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.861261 (+ 24552us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.871446 (+ 10185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18129280}
I20260709 13:16:05.992060 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.004551 10597 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.004879 10481 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.005653 10481 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.005501 10597 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.009119 10250 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:06.011091 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.012303 10481 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.013693 10250 ts_tablet_manager.cc:1403] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 1.313s	user 0.025s	sys 0.021s
I20260709 13:16:05.973366 10602 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.015267 10592 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.015372 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:05.959228 10580 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.964996 10578 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.965634 10586 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:05.954147 10622 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.020933 10601 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.967021 10055 rpcz_store.cc:276] 0709 13:16:04.862391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.864362 (+  1971us) service_pool.cc:224] Handling call
0709 13:16:05.180223 (+315861us) spinlock_profiling.cc:243] Waited 926 us on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.881029 (+700806us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.891089 (+ 10060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.903579 (+ 12490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.903618 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.913922 (+ 10304us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:05.920890 (+  6968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.927008 (+  6118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.934017 (+  7009us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14816000}
I20260709 13:16:06.048084 10616 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:05.977859 10198 rpcz_store.cc:276] 0709 13:16:04.808282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.918304 (+110022us) service_pool.cc:224] Handling call
0709 13:16:04.962930 (+ 44626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.979845 (+ 16915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.979895 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.192891 (+212996us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.627815 (+434924us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:16:05.627892 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.628030 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.954923 (+326893us) raft_consensus.cc:1685] finished
0709 13:16:05.954948 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.968916 (+ 13968us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":190,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":165,"thread_start_us":294113,"threads_started":1,"wal-append.queue_time_us":683254}
I20260709 13:16:06.036036 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.038316 10127 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:05.967432 10610 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.025897 10602 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.038725 10583 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.050689 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 204) took 1190 ms. Trace:
I20260709 13:16:06.054190 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.024309 10607 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.067935 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.070699 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 68) took 1135 ms. Trace:
I20260709 13:16:06.071210 10607 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.072288 10606 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.074885 10250 raft_consensus.cc:359] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.078927 10250 raft_consensus.cc:385] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:06.079990 10250 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:06.077329 10614 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.078439 10597 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.076368 10580 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:05.982527 10621 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.085269 10250 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.035218 10622 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.081269 10529 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.071290 10052 rpcz_store.cc:276] 0709 13:16:04.947741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.947967 (+   226us) service_pool.cc:224] Handling call
0709 13:16:06.067748 (+1119781us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:06.070588 (+  2840us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14844160}
I20260709 13:16:06.087248 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.097221 10611 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.102931 10250 ts_tablet_manager.cc:1434] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.000s
I20260709 13:16:06.050336 10616 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.110674 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.066404 10051 rpcz_store.cc:276] 0709 13:16:04.860190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.861070 (+   880us) service_pool.cc:224] Handling call
0709 13:16:05.999888 (+1138818us) spinlock_profiling.cc:243] Waited 7.74 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:06.017394 (+ 17506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.026726 (+  9332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.028432 (+  1706us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.030884 (+  2452us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:06.034490 (+  3606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.037327 (+  2837us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.049228 (+ 11901us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17793408}
I20260709 13:16:06.112516 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.117151 10610 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.118758 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.125105 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.125989 10051 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:06.129371 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 199) took 1338 ms. Trace:
I20260709 13:16:06.130160 10053 rpcz_store.cc:276] 0709 13:16:04.791307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.804839 (+ 13532us) service_pool.cc:224] Handling call
0709 13:16:05.027197 (+222358us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.195800 (+168603us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.196153 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.351521 (+155368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.354488 (+  2967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.380812 (+ 26324us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:05.385367 (+  4555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.128955 (+743588us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400086454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:06.129024 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.129255 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364116352}
I20260709 13:16:06.122499 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 206) took 1247 ms. Trace:
I20260709 13:16:06.123633 10578 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.115289 10049 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:06.167601 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.166540 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:06.164198 10583 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.173053 10583 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.173456 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.148294 10050 rpcz_store.cc:276] 0709 13:16:04.865451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.876779 (+ 11328us) service_pool.cc:224] Handling call
0709 13:16:06.108003 (+1231224us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:06.110734 (+  2731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.110935 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.110962 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.111009 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:06.111043 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.111140 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.112232 (+  1092us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18297216}
I20260709 13:16:06.174503 10607 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.177060 10607 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.177325 10601 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:06.182240 10623 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:06.162025 10611 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:16:06.183969 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.175057 10581 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:06.184695 10199 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:06.185891 10607 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.186967 10583 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.190448 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.191501 10581 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.194180 10581 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:06.194528 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.195312 10198 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:06.195621 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.195873 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.196362 10204 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:06.195546 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:06.199018 10095 leader_election.cc:343] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.191372 10602 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.202294 10583 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.216104 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.224561 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.225257 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.224720 10622 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.206157 10555 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.229492 10019 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.215240 10621 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.232950 10570 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.230782 10555 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.233322 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.227751 10565 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:06.236923 10565 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.227304 10616 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.237767 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 230) took 1448 ms. Trace:
I20260709 13:16:06.245471 10249 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:06.246018 10249 ts_tablet_manager.cc:1403] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.892s	user 0.025s	sys 0.011s
I20260709 13:16:06.239199 10529 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:06.251544 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.253278 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.254052 10128 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:06.247751 10529 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.256415 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.256800 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.257189 10529 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.257846 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.263943 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:06.264820 10019 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.266662 10342 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:06.267194 10342 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.267580 10342 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:06.254838 10581 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.272284 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.275553  9954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 98) took 1426 ms. Trace:
I20260709 13:16:06.262405 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.257983 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.244835 10622 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:06.283025 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80973.4us
I20260709 13:16:06.270393 10581 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.284528 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.280023 10576 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.298067 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.300872 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.302945 10586 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.291498 10622 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.238902 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.311842 10248 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:06.312736 10248 ts_tablet_manager.cc:1403] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.972s	user 0.023s	sys 0.017s
I20260709 13:16:06.302760 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:06.342830 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58456.6us
I20260709 13:16:06.343674 10576 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.365141 10250 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:06.242424 10201 rpcz_store.cc:276] 0709 13:16:04.789642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.815250 (+ 25608us) service_pool.cc:224] Handling call
0709 13:16:04.815873 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.168981 (+353108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.174935 (+  5954us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.179210 (+  4275us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.815517 (+636307us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b48001e0258. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:16:05.969993 (+154476us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b18001cd560. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:16:06.003800 (+ 33807us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:06.032214 (+ 28414us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.034140 (+  1926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.035593 (+  1453us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.237526 (+201933us) raft_consensus.cc:1685] finished
0709 13:16:06.237555 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.237703 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2427,"prepare.queue_time_us":56573,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":1933,"raft.run_cpu_time_us":37043,"raft.run_wall_time_us":649086,"spinlock_wait_cycles":24625792,"thread_start_us":49151,"threads_started":2,"wal-append.queue_time_us":857969}
I20260709 13:16:06.370450 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.373508 10054 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:06.370999 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.415220 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.416321 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.416765 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.417011 10565 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.276168  9954 rpcz_store.cc:276] 0709 13:16:04.849128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.849375 (+   247us) service_pool.cc:224] Handling call
0709 13:16:06.275463 (+1426088us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:05.330459 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:05.521336 (+190877us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:05.523965 (+  2629us) write_op.cc:435] Acquired schema lock
0709 13:16:05.556330 (+ 32365us) tablet.cc:662] Decoding operations
0709 13:16:05.614977 (+ 58647us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:05.628547 (+ 13570us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:05.634427 (+  5880us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:05.662343 (+ 27916us) tablet.cc:701] Row locks acquired
0709 13:16:05.665718 (+  3375us) write_op.cc:260] PREPARE: finished
0709 13:16:05.684940 (+ 19222us) write_op.cc:270] Start()
0709 13:16:05.696376 (+ 11436us) write_op.cc:276] Timestamp: P: 1783602965680055 usec, L: 0
0709 13:16:05.699568 (+  3192us) op_driver.cc:348] REPLICATION: starting
0709 13:16:05.788809 (+ 89241us) log.cc:844] Serialized 537 byte log entry
0709 13:16:06.050211 (+261402us) op_driver.cc:464] REPLICATION: finished
0709 13:16:06.094831 (+ 44620us) write_op.cc:301] APPLY: starting
0709 13:16:06.094923 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:06.095025 (+   102us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:06.100450 (+  5425us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:06.103079 (+  2629us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:06.107559 (+  4480us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:06.107624 (+    65us) write_op.cc:312] APPLY: finished
0709 13:16:06.117806 (+ 10182us) log.cc:844] Serialized 22 byte log entry
0709 13:16:06.118724 (+   918us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:06.118841 (+   117us) write_op.cc:454] Released schema lock
0709 13:16:06.118892 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:06.119350 (+   458us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:06.119390 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":222632,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":44280,"apply.run_cpu_time_us":3155,"apply.run_wall_time_us":24522,"num_ops":1,"prepare.queue_time_us":171849,"prepare.run_cpu_time_us":91419,"prepare.run_wall_time_us":650285,"raft.queue_time_us":101222,"raft.run_cpu_time_us":29441,"raft.run_wall_time_us":38822,"replication_time_us":350455,"thread_start_us":325310,"threads_started":4,"wal-append.queue_time_us":156738}]]}
W20260709 13:16:06.403234 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 138448us
I20260709 13:16:06.405082 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.425148 10249 raft_consensus.cc:359] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.431984 10249 raft_consensus.cc:385] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:06.432385 10249 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:06.433213 10249 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.431412 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 69) took 1397 ms. Trace:
I20260709 13:16:06.455029 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.458132 10249 ts_tablet_manager.cc:1434] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.212s	user 0.004s	sys 0.006s
I20260709 13:16:06.440248 10248 raft_consensus.cc:359] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.460220 10248 raft_consensus.cc:385] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:06.460649 10248 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:06.448464 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.461073 10565 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:06.476275 10558 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.476836 10565 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.477175 10558 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.476260 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.437083 10054 rpcz_store.cc:276] 0709 13:16:05.047456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.048592 (+  1136us) service_pool.cc:224] Handling call
0709 13:16:05.712525 (+663933us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:06.363767 (+651242us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:06.425298 (+ 61531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45378176}
I20260709 13:16:06.466614 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.483992 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.464658 10248 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.471864 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:06.479764 10555 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.469776 10342 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.490258 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:06.490911 10602 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.485827 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.490970 10601 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:06.639277 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.647320 10601 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.630066 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 248) took 1029 ms. Trace:
I20260709 13:16:06.641081 10254 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.496603 10250 tablet_bootstrap.cc:654] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:06.648166 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 193279us
I20260709 13:16:06.654795 10254 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.657634 10248 ts_tablet_manager.cc:1434] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.344s	user 0.014s	sys 0.020s
W20260709 13:16:06.623273 10127 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.559s	user 0.014s	sys 0.011s
I20260709 13:16:06.490671 10558 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.611508 10458 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.587332 10631 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.491407 10565 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.612035 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.688951 10020 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.672500 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.589567 10622 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.658023 10601 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.510458 10198 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:06.673692 10130 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:06.669775 10555 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.654188 10202 rpcz_store.cc:276] 0709 13:16:05.600189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.649695 (+ 49506us) service_pool.cc:224] Handling call
0709 13:16:06.629757 (+980062us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b54002dd7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:06.629954 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206427904}
W20260709 13:16:06.697590 10095 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.735682 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:06.674688 10127 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:06.668598 10602 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.632048 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 67) took 1770 ms. Trace:
I20260709 13:16:06.691418 10558 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.696000 10458 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:06.742304 10095 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:06.756640 10095 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.757699 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:06.739478 10555 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.766875 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.768577 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.769053 10555 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.770270 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.747414 10558 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:06.766098 10095 leader_election.cc:343] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.748255 10616 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.735209 10631 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.777930 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:06.759685 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.753656 10458 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.784226 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.785022 10616 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.772089 10529 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.772104 10634 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:06.788524 10095 leader_election.cc:343] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.790145 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.790892 10254 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:06.791139 10170 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.711s (SENT) [suppressed 3 similar messages]
I20260709 13:16:06.793052 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 211) took 1635 ms. Trace:
I20260709 13:16:06.793637 10051 rpcz_store.cc:276] 0709 13:16:05.157658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.128161 (+970503us) service_pool.cc:224] Handling call
0709 13:16:06.169197 (+ 41036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.173981 (+  4784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.174020 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.174083 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:06.174124 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.174197 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.792790 (+618593us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b540010b3d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:06.792966 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92134784}
I20260709 13:16:06.796366 10558 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.797089 10558 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.796947 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.797878 10558 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:06.789423 10095 leader_election.cc:343] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:06.780817 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.780373 10342 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.781738 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.817462 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.832671 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.834066 10558 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.834164 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.824903 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.836058 10631 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.837858 10638 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:06.838356 10636 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:16:06.745836 10048 rpcz_store.cc:276] 0709 13:16:04.871580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.871882 (+   302us) service_pool.cc:224] Handling call
0709 13:16:05.383212 (+511330us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:05.389618 (+  6406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.922869 (+533251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.924869 (+  2000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.931747 (+  6878us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:05.933499 (+  1752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.631638 (+698139us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400391554. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:06.631720 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.631948 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1207521280}
I20260709 13:16:06.840341 10248 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:06.842115 10640 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.842312 10602 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.816033 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:06.860644 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 159320us
I20260709 13:16:06.855476 10130 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:06.868579 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.845881 10640 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.881085 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.888053 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.889065 10048 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:06.890492 10607 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.891391 10458 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.891253 10607 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.893657 10249 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:16:06.895699 10637 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:06.896972 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.898509 10605 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.898891 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.899251 10605 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.900338 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.901134 10642 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:06.901621 10605 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.901715 10642 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:06.902222 10642 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.898555 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.873811 10599 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:06.875450 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84367.7us
W20260709 13:16:06.915351 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:06.916460 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:06.911554 10575 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.924801 10638 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.935266 10638 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.936693 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:06.937599 10248 tablet_bootstrap.cc:654] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:06.925478 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.939320 10254 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:06.939759 10254 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:06.940253 10254 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:06.943943 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:06.944787 10196 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:06.925961 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.911846 10599 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.949225 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.950027 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.951691 10599 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.949311 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.952737 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.953099 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.934787 10607 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.927439 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.927798 10640 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.928188 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.919508 10342 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.930440 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.920099 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.932407 10601 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:06.934038 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:06.937721 10638 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:06.925206 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.938570 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:06.925412 10644 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:06.950724 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:06.953519 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.034228 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.161123 10019 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:07.212563 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.018860 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.213997 10644 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:07.210932 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.085969 10249 tablet_bootstrap.cc:654] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:06.954401 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.024700 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.039745 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.153825 10645 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:07.214833 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.229466 10645 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.157761 10254 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.014727 10170 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.230841 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:07.231945 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 263156us
I20260709 13:16:07.232484 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.232683 10055 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:07.233291 10170 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.233963 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.234366 10638 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.235080 10638 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.062809 10605 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.237190 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.019773 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.238865 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.109782 10458 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.240907 10645 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.241195 10458 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.242911 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.244441 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.039680 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.206410 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.245916 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.019624 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.020946 10642 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.250095 10642 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:06.923941 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.236347 10605 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.255734 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.014758 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 238) took 1854 ms. Trace:
I20260709 13:16:07.126379 10248 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:07.022872 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.208693 10599 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.258999 10248 ts_tablet_manager.cc:1403] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.419s	user 0.013s	sys 0.011s
I20260709 13:16:07.260022 10125 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:07.262194 10055 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.262593 10248 raft_consensus.cc:359] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.263501 10248 raft_consensus.cc:385] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:07.263799 10248 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:07.264907 10248 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.267890 10248 ts_tablet_manager.cc:1434] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:07.269439 10248 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:07.269583 10640 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.270150 10640 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.272571 10640 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.269439 10168 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:07.274468 10640 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.275017 10640 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.275535 10640 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.258603 10205 rpcz_store.cc:276] 0709 13:16:05.160306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.192002 (+ 31696us) service_pool.cc:224] Handling call
0709 13:16:06.026587 (+834585us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:06.026859 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.100099 (+ 73240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.100123 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.100153 (+    30us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:06.100174 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.014384 (+914210us) spinlock_profiling.cc:243] Waited 72.9 ms on lock 0x7b540039e254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:07.014478 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.014673 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186806912}
W20260709 13:16:07.209434 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 270913us
I20260709 13:16:07.209645 10601 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.280051 10248 tablet_bootstrap.cc:654] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:07.281180 10095 leader_election.cc:343] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:07.282642 10095 leader_election.cc:343] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.033596 10250 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
W20260709 13:16:07.212623 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51588.1us
I20260709 13:16:07.283722 10250 ts_tablet_manager.cc:1403] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.919s	user 0.049s	sys 0.067s
I20260709 13:16:07.284107 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:07.246285 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.285447 10640 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:06.991645 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44748.2us
I20260709 13:16:06.954723 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.285820 10254 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.287750 10054 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:07.287602 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.219200 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.214720 10644 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:07.288565 10604 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:07.289287 10645 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.289834 10645 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:07.290288 10645 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.290072 10644 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.287104 10250 raft_consensus.cc:359] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.058701 10096 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.223685 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.222750 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.214392 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:07.217777 10052 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:07.235507 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:07.292155 10250 raft_consensus.cc:385] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:07.296517 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:07.287603 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54500.1us
I20260709 13:16:07.297456 10122 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.294080 10640 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.298283 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.291728 10604 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:07.310834 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.316507 10643 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.319736 10643 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:07.319528 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:07.320363 10643 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.288959 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.296907 10050 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:07.296886 10250 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
W20260709 13:16:07.301115 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 242560us
I20260709 13:16:07.301872 10621 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.306025 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.302968 10599 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.310003 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:07.310086 10640 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.311374 10458 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.313915 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.314428 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.288234 10605 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.020184 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.325218 10250 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.338699 10249 tablet_bootstrap.cc:492] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:07.349332 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:07.350034 10249 ts_tablet_manager.cc:1403] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.457s	user 0.027s	sys 0.025s
W20260709 13:16:07.352190 10020 leader_election.cc:343] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:07.353277 10020 leader_election.cc:343] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.354141 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.353780 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.357216 10250 ts_tablet_manager.cc:1434] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.002s
W20260709 13:16:07.360339 10020 leader_election.cc:343] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.359838 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:07.366863 10095 leader_election.cc:343] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.316203 10604 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.370802 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:07.373128 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.374500 10250 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:07.378549 10286 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.356861 10605 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.359014 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.333336 10621 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.362363 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:07.342747 10640 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.383447 10286 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.358786 10568 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.370783 10599 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:07.366035 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52201.8us
I20260709 13:16:07.383424 10605 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:07.394747 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70345.3us
I20260709 13:16:07.379882 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.404418 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.407094 10254 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.408442 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.409209 10254 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.408866 10286 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:07.409685 10096 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.390978 10201 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.410916 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.411806 10286 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.412297 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.414812 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.418907 10286 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.419325 10096 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.419843 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.420552 10254 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.421397 10096 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.422322 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.399305 10599 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.406947 10202 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:07.424160 10096 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.406226 10605 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.436590 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.424168 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.442333 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.429616 10660 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.418538 10342 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.445238 10607 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.435851 10599 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.423075 10529 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.464797 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:07.465499 10129 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:07.485368 10248 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:07.485939 10248 ts_tablet_manager.cc:1403] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.003s
I20260709 13:16:07.495792 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:07.466532 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53418.3us
I20260709 13:16:07.418028 10645 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.463893 10660 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.464128 10607 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:07.498955 10248 raft_consensus.cc:359] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.517722 10248 raft_consensus.cc:385] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:07.518151 10248 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:07.519243 10248 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.484501 10529 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.468607 10568 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 13:16:07.503304 10645 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:07.530795 10657 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:16:07.523389 10607 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:07.547348 10658 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:16:07.540774 10661 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.500524 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.523269 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:07.568655 10656 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:16:07.571174 10621 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.574627 10638 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.574426 10661 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.578162 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.604800 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:07.606272 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:07.608764 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.585822 10249 raft_consensus.cc:359] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.609560 10249 raft_consensus.cc:385] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:07.609874 10249 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:07.611898 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.612548 10131 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:07.610632 10249 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.593320 10250 tablet_bootstrap.cc:654] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:07.595863 10568 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:07.603241 10661 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.604943 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:07.579403 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78987.3us
I20260709 13:16:07.608601 10605 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.616112 10129 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
W20260709 13:16:07.582170 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 139984us
I20260709 13:16:07.623426 10249 ts_tablet_manager.cc:1434] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.273s	user 0.007s	sys 0.002s
I20260709 13:16:07.635195 10248 ts_tablet_manager.cc:1434] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.149s	user 0.008s	sys 0.006s
I20260709 13:16:07.657202 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.588238 10640 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.673197 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.623893 10568 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:07.667011 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.668987 10254 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.669943 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.670513 10248 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:07.682344 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.686597 10286 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.687052 10096 leader_election.cc:343] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.687628 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.688716 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.689740 10096 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:07.651768 10128 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.696182 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.706992 10286 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.708504 10096 leader_election.cc:343] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.709367 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.711365 10096 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.665639 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.730815 10286 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.728129 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45440.3us
I20260709 13:16:07.731207 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.734297 10096 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.685308 10640 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.691408 10529 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.700290 10621 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.754266 10254 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.696048 10644 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:07.761252 10249 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:07.749085 10660 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.762159 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.736214 10659 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.773900 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.741739 10198 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.729728 10665 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.775687 10516 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.776497 10516 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.771013 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 71977.9us
I20260709 13:16:07.674839 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:07.782394 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:07.827903 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45641.6us
I20260709 13:16:07.712450 10664 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.832304 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:07.824131 10568 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.746431 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.836146 10568 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.836359 10676 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.836663 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 122387us
W20260709 13:16:07.827067 10636 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:16:07.846333 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 107090us
I20260709 13:16:07.834220 10575 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.773846 10644 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.781885 10659 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.837954 10638 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:07.845482 10342 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:16:07.832196 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 179982us
I20260709 13:16:07.868503 10170 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.871003 10659 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.833432 10664 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:07.878929 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.882534 10342 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:07.883440 10664 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.883723 10342 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.883989 10249 tablet_bootstrap.cc:654] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:07.874821 10607 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.889714 10019 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.889372 10649 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.868101 10638 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:07.891898 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.892253 10638 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.893524 10250 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:07.894356 10250 ts_tablet_manager.cc:1403] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.526s	user 0.021s	sys 0.011s
I20260709 13:16:07.884644 10605 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.898327 10458 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.901505 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 131211us
I20260709 13:16:07.897087 10170 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.895036 10640 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.908018 10673 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:07.909278 10675 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:07.909036 10599 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.903550 10583 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.909801 10675 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:07.899492 10050 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.014s
I20260709 13:16:07.903026 10201 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.934764 10644 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.909806 10674 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:07.912436 10458 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.935528 10643 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.614s	user 0.017s	sys 0.008s
I20260709 13:16:07.936316 10529 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:07.936787 10529 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:07.937734 10529 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.939652 10529 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:07.917956 10248 tablet_bootstrap.cc:654] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:07.923185 10096 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:07.905526 10122 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.606s	user 0.030s	sys 0.008s
I20260709 13:16:07.930847 10599 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:07.936298 10643 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.959894 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.960762 10643 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:07.947718 10095 leader_election.cc:343] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.967566 10250 raft_consensus.cc:359] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.968456 10250 raft_consensus.cc:385] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:07.968726 10250 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:07.970162 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:07.972592 10675 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.973980 10675 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:07.973992 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:16:07.977490 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54485.3us
I20260709 13:16:07.978637 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.975373 10250 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:07.923595 10050 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:07.919319 10170 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:07.944135 10122 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:07.960762 10607 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:07.985306 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.987725 10286 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:07.990604 10095 leader_election.cc:343] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:07.994165 10677 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.994449 10131 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.994791 10286 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.994217 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.996376 10661 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:07.996838 10096 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:07.996977 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:07.997678 10661 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:07.997752 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:07.999445 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.002358 10125 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:07.986017 10642 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.007367 10640 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.011394 10640 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.011859 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:07.987725 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 230) took 1046 ms. Trace:
I20260709 13:16:08.003763 10643 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.013728 10640 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:08.013916 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94729.2us
I20260709 13:16:08.015682 10250 ts_tablet_manager.cc:1434] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.121s	user 0.012s	sys 0.000s
I20260709 13:16:07.981952 10678 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:16:08.022281 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.024247 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.013389 10050 rpcz_store.cc:276] 0709 13:16:06.941793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.235151 (+293358us) service_pool.cc:224] Handling call
0709 13:16:07.986361 (+751210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.028112 10643 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:08.028648 10643 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:08.010684 10642 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.029124 10643 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.034904 10250 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:08.035866 10659 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:08.038733 10659 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.039194 10659 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:08.043860 10516 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:07.984856 10664 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.049717 10126 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.030530 10640 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.040126 10659 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.053297 10659 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.046975 10516 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:08.060768 10020 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.063089 10642 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.063351 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.064540 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.065070 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.065976 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.066537 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.067430 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.068090 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.068492 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.074106 10249 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:08.074914 10249 ts_tablet_manager.cc:1403] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.314s	user 0.016s	sys 0.008s
I20260709 13:16:08.077975 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.081867 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.089565 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.092998 10053 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.095381 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.096921 10516 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.097651 10664 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.087875 10599 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:08.098327 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.098932 10599 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:08.097517 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.100033 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.117471 10665 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.118005 10665 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:08.118965 10665 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.120436 10665 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.126878 10168 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:08.127856 10249 raft_consensus.cc:359] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.128769 10249 raft_consensus.cc:385] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.129272 10249 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.128535 10458 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:08.130136 10249 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.130631 10458 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:08.132030 10458 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.072947 10638 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.136830 10205 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.141649 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.141489 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.142724 10122 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:08.143991 10053 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.147902 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.149935 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.150328 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.153810 10056 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:16:08.150358 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:08.101807 10020 leader_election.cc:343] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:08.167037 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65308us
I20260709 13:16:08.163414 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.154986 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.110821 10248 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:08.173985 10248 ts_tablet_manager.cc:1403] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.504s	user 0.019s	sys 0.014s
I20260709 13:16:08.098402 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.175117 10205 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.178802 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:08.183465 10249 ts_tablet_manager.cc:1434] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.001s
I20260709 13:16:08.184629 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:08.185710 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 94027.4us
I20260709 13:16:08.186625 10019 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.187171 10250 tablet_bootstrap.cc:654] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.188352 10672 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:16:08.173882 10638 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:08.192740 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.196862 10049 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:08.198339 10248 raft_consensus.cc:359] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.199110 10248 raft_consensus.cc:385] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.204113 10248 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.203613 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.201776 10675 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.206570 10248 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.207861 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.212143 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.205915 10342 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.211263 10638 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.214417 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.216331 10672 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.216822 10672 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:08.217658 10672 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.213224 10605 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.218952 10675 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.221499 10675 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.222138 10675 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:08.224128 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.225907 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.186615 10643 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.234377 10643 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:08.236559 10605 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:08.237792 10638 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.226725 10607 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.216547 10249 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:08.225625 10583 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.100226 10599 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.238276 10638 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:08.251901 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.257341 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.253474 10660 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:08.234655 10644 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 13:16:08.261917 10679 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:16:08.262512 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50471.3us
I20260709 13:16:08.282372 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.262981 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.226672 10050 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:16:08.266812 10643 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.288328 10643 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.272693 10286 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.273221 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.276224 10248 ts_tablet_manager.cc:1434] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
W20260709 13:16:08.272172 10096 leader_election.cc:343] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.291846 10096 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.291833 10129 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:08.293418 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.294094 10286 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.284997 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.295357 10053 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:08.295224 10286 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.296223 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.296279 10676 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.284548 10607 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.297510 10676 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.295099 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.298727 10661 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.299396 10643 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:08.305328 10020 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.313657 10642 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.331954 10170 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.333171 10458 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.333600 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.316089 10638 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.334571 10676 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.335150 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.336874 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.337298 10607 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.339234 10607 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.343369 10249 tablet_bootstrap.cc:654] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.345003 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.347697 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.353899 10676 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.352519 10607 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.360080 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.363168 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.364073 10130 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:08.368278 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.369506 10127 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:08.378165 10250 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:08.378919 10250 ts_tablet_manager.cc:1403] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.344s	user 0.017s	sys 0.017s
W20260709 13:16:08.383246 10096 leader_election.cc:343] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.385819 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.366981 10672 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.386695 10125 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:08.389038 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:08.390708 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.390878 10632 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:08.388749 10665 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.391537 10632 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.392715 10632 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.399755 10250 raft_consensus.cc:359] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.402166 10250 raft_consensus.cc:385] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.402567 10250 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.403450 10250 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.393357 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.401222 10576 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:08.420719 10576 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.396303 10645 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:08.428251 10250 ts_tablet_manager.cc:1434] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.001s
W20260709 13:16:08.438562 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90903.8us
I20260709 13:16:08.426250 10645 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.445142 10170 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.446827 10686 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.448630 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.449990 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:08.462860 10020 leader_election.cc:343] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.449175 10645 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.451467 10686 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.456005 10643 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.480901 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.489769 10198 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.475428 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.498160 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.508406 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.514808 10249 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:08.517853 10249 ts_tablet_manager.cc:1403] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.302s	user 0.017s	sys 0.029s
I20260709 13:16:08.514230 10643 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.514078 10678 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:08.520524 10678 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.520927 10678 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:08.521870 10678 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:08.523190 10020 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.512954 10458 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.524428 10678 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.530968 10249 raft_consensus.cc:359] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.539652 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.549783 10249 raft_consensus.cc:385] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.550177 10249 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.555619 10664 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.556231 10664 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:08.556484 10249 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.557252 10664 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.558717 10664 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.562314 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.563234 10122 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:08.563807 10053 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:08.563968 10096 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.565997 10661 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.542804 10645 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.571579 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.544924 10250 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:08.559486 10249 ts_tablet_manager.cc:1434] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 13:16:08.548049 10576 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.577665 10199 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:08.581079 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.544059 10643 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.583333 10649 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:08.583781 10649 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:08.584587 10649 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.585940 10249 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:08.544135 10632 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.587997 10632 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.591408 10645 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.593442 10570 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:08.598104 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:08.600698 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:08.602407 10250 tablet_bootstrap.cc:654] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.608851 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.609401 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:08.608855 10020 leader_election.cc:343] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:08.610534 10020 leader_election.cc:343] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.610881 10640 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:08.611632 10640 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:08.612919 10640 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.614571 10638 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.580893 10575 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.620411 10649 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.620530 10575 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.624540 10248 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:08.632426 10575 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.632591 10249 tablet_bootstrap.cc:654] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.638931 10248 tablet_bootstrap.cc:654] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.642962  9954 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.647455 10570 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:08.648007 10570 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:08.648511 10570 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:08.656073 10050 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:16:08.656692 10050 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 0
I20260709 13:16:08.657236 10050 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:16:08.657755 10050 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:08.658658 10249 tablet_bootstrap.cc:492] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:08.659320 10249 ts_tablet_manager.cc:1403] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.005s
I20260709 13:16:08.660207 10640 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.673727 10659 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.674708 10659 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.677928 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.678123 10659 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.677592  9954 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:08.663010 10249 raft_consensus.cc:359] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.681696 10249 raft_consensus.cc:385] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.682197 10249 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.683035 10249 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.683025 10570 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.683713 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.684803 10250 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:08.685381 10250 ts_tablet_manager.cc:1403] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.141s	user 0.013s	sys 0.013s
I20260709 13:16:08.685626 10570 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:08.686228 10095 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:08.690380 10020 leader_election.cc:343] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.692145 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:08.693135 10199 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:08.693571 10250 raft_consensus.cc:359] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.694403 10250 raft_consensus.cc:385] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.694739 10250 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.696090 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.697616 10126 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:08.699798 10640 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.701915 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.702737 10124 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.704960 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.705755 10128 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.706301 10249 ts_tablet_manager.cc:1434] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
W20260709 13:16:08.709857 10096 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.710487 10096 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:08.711443 10659 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.711964 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:08.721341 10249 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:08.727589 10248 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:08.728387 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:08.729743 10249 tablet_bootstrap.cc:654] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.728662 10248 ts_tablet_manager.cc:1403] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.027s
W20260709 13:16:08.756285 10019 leader_election.cc:343] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.756418 10575 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.757085 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:08.758198 10575 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:08.760978 10248 raft_consensus.cc:359] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.761897 10248 raft_consensus.cc:385] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.762218 10248 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.763003 10248 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.764966 10661 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:08.766263 10248 ts_tablet_manager.cc:1434] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:16:08.783306 10248 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:08.784724 10250 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.804631 10128 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:08.810442 10250 ts_tablet_manager.cc:1434] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.125s	user 0.008s	sys 0.001s
I20260709 13:16:08.818507 10249 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:08.819231 10249 ts_tablet_manager.cc:1403] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.028s
I20260709 13:16:08.824668 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.825768 10645 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:08.826329 10645 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:08.831825 10645 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.853745 10678 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.864089 10250 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:08.862550 10678 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.876145 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:08.876941 10054 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:08.863081 10201 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.883646 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:08.884918 10664 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:08.887425 10664 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.887828 10664 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.920305 10664 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:08.926602 10705 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:16:08.885888 10249 raft_consensus.cc:359] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.928369 10249 raft_consensus.cc:385] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:08.928859 10249 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:08.944175 10678 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.932698 10124 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:08.950205 10249 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.959492 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.959995 10342 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.960994 10581 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:08.960680 10342 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.961613 10581 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:08.947160 10123 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:08.962679 10581 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:08.961563 10248 tablet_bootstrap.cc:654] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.965071 10249 ts_tablet_manager.cc:1434] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.009s
I20260709 13:16:08.965783 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:08.966315 10342 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:08.970537 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:08.974346 10249 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:16:08.974696 10020 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:08.984510 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:08.985337 10054 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:08.972563 10570 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:08.971024 10196 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.004798 10250 tablet_bootstrap.cc:654] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:08.976686 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.009101 10054 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:16:09.006469 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:08.977205 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.007293 10570 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:09.011482 10095 leader_election.cc:343] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.015271 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:09.017117 10095 leader_election.cc:343] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.020648 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:09.011219 10123 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:09.019819 10664 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.024488 10659 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:09.034144 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:09.026968 10570 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.035313 10601 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:09.035833 10601 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.036330 10601 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.036201 10659 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:09.037729 10659 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.039621 10123 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:09.045858 10054 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:09.066044 10249 tablet_bootstrap.cc:654] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.054688 10676 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:16:09.076179 10708 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:09.079067 10638 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:16:09.094103 10601 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.099092 10248 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:09.104275 10248 ts_tablet_manager.cc:1403] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.321s	user 0.016s	sys 0.009s
I20260709 13:16:09.127888 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:09.128834 10125 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.131336 10640 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.132007 10640 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.136549 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.138360 10601 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.138878 10640 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.136549 10249 tablet_bootstrap.cc:492] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:09.140259 10249 ts_tablet_manager.cc:1403] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.167s	user 0.022s	sys 0.011s
I20260709 13:16:09.144699 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:09.147071 10020 leader_election.cc:343] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.149451 10248 raft_consensus.cc:359] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.150369 10248 raft_consensus.cc:385] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.150693 10248 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.154207 10249 raft_consensus.cc:359] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.167846 10249 raft_consensus.cc:385] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.168206 10249 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.159384 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.174235 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:09.175798 10640 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:09.168988 10249 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.178102 10248 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.180229 10599 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:09.182688 10249 ts_tablet_manager.cc:1434] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.001s
I20260709 13:16:09.184182 10249 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:09.189944 10248 ts_tablet_manager.cc:1434] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.072s	user 0.010s	sys 0.000s
I20260709 13:16:09.196030 10248 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Bootstrap starting.
I20260709 13:16:09.189970 10676 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.206123 10676 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.189088  9953 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:09.235878  9953 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:09.239176  9953 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:09.294889 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46512.4us
I20260709 13:16:09.299911 10676 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.303689 10250 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:09.304355 10250 ts_tablet_manager.cc:1403] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.451s	user 0.049s	sys 0.038s
I20260709 13:16:09.304996 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.293931 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:09.308333 10645 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:16:09.330742 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.331902 10124 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:09.339910 10249 tablet_bootstrap.cc:654] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.341727 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:09.361653 10286 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.362291 10286 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.364832 10286 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.373250 10248 tablet_bootstrap.cc:654] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.373934 10676 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.378350  9952 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.379384 10676 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.385771 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:09.387813 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46244.7us
I20260709 13:16:09.387074 10676 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.391377 10721 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.392935 10250 raft_consensus.cc:359] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.393826 10250 raft_consensus.cc:385] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.394156 10250 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.394675 10581 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:09.394964 10250 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.396821 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.397964 10250 ts_tablet_manager.cc:1434] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.093s	user 0.014s	sys 0.007s
I20260709 13:16:09.399387 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.400322 10130 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:09.402446 10250 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:09.408725 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.408699 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:09.411214 10640 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.413590 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.421414 10581 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:09.424950 10581 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.428464 10205 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:09.423851 10095 leader_election.cc:343] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:09.438234 10096 leader_election.cc:343] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.438800 10096 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:09.439761 10640 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:09.442503 10599 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:09.443755 10644 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.462884 10601 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:09.447412 10721 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:09.475255  9954 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.478662 10249 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:09.479460 10249 ts_tablet_manager.cc:1403] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.296s	user 0.017s	sys 0.011s
I20260709 13:16:09.478650  9954 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.488752  9954 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:09.490729  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 2 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.497627 10248 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: No bootstrap required, opened a new log
I20260709 13:16:09.498322 10248 ts_tablet_manager.cc:1403] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Time spent bootstrapping tablet: real 0.303s	user 0.030s	sys 0.001s
I20260709 13:16:09.502208 10130 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:09.502982 10605 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:09.504005 10605 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.507872 10250 tablet_bootstrap.cc:654] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.509760 10605 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.510433 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.518412 10248 raft_consensus.cc:359] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.519476 10248 raft_consensus.cc:385] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.519801 10248 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.520613 10248 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.521392 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.522410 10130 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:09.522215 10649 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:09.523692 10248 ts_tablet_manager.cc:1434] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:16:09.524144 10516 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:09.511327 10056 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:09.530741 10664 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:09.542261 10249 raft_consensus.cc:359] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.543020 10249 raft_consensus.cc:385] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.543383 10249 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.538851 10665 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.552106 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:09.556082 10605 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.556664 10605 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:09.557091 10605 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.558816 10599 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.559461 10599 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.561918 10599 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.566190 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.567034 10199 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:09.569274 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:09.575898 10599 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.576411 10599 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.576778 10599 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.578663 10665 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.591524 10665 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.602108 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.604070  9953 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:09.610422 10095 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.616290 10249 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:09.633561 10095 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.620357 10645 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.641631 10645 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.686934 10249 ts_tablet_manager.cc:1434] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.207s	user 0.010s	sys 0.000s
W20260709 13:16:09.696060 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66575.5us
I20260709 13:16:09.667523 10640 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.701606 10249 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:09.704118 10250 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:09.703611 10605 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.704767 10250 ts_tablet_manager.cc:1403] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.306s	user 0.033s	sys 0.020s
W20260709 13:16:09.697854 10728 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:16:09.674013 10638 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.716667 10638 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.724261 10665 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.724987 10665 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.708775 10640 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.729241 10665 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:09.730387 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 101267us
I20260709 13:16:09.735054 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.743655 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.744572 10050 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:09.746814 10638 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.749778 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.729770 10645 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.750914 10050 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:09.753787 10640 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.754545 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:09.754963 10732 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:09.755452 10124 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:09.755805 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.756889 10168 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:09.759025 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.759898 10054 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:09.760531 10640 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.761503 10640 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.762006 10640 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:09.762966 10096 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.763512 10096 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:09.765507 10678 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:09.768317 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.769269 10125 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:09.772506 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.762840 10605 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:09.774621 10170 leader_election.cc:343] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.776204 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.777021 10124 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 0.
I20260709 13:16:09.778630 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:09.778365 10250 raft_consensus.cc:359] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.779736 10250 raft_consensus.cc:385] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:09.780227 10250 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:09.780531 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:09.781251 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:09.781818 10124 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:09.782946 10605 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:09.783525 10605 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.783952 10605 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.787755  9954 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:09.790186 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:09.793709 10095 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:09.800454 10250 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.814177 10250 ts_tablet_manager.cc:1434] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.109s	user 0.011s	sys 0.000s
I20260709 13:16:09.814059 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:09.815256 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:09.815572 10250 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:09.816179 10678 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:09.816648 10678 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.817070 10678 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.820276 10249 tablet_bootstrap.cc:654] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.829396 10054 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:09.831468 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:09.742432 10599 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.832729 10676 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:09.836594 10676 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:09.837751 10676 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.834633 10286 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.839577 10599 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.839490 10286 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.838702 10640 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.841841 10286 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.831882 10644 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.843416 10644 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.844727 10644 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.845252 10581 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:09.852494 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:09.856254 10054 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.860169 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:09.860389 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.861373 10201 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.871471 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.872352 10053 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:09.873536 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:09.874488 10056 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.875198 10721 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:09.875788 10721 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.879617 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:09.855039 10640 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.884807 10678 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.891039 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:09.891839 10721 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.891956 10053 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:09.893452 10170 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:09.895900 10721 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:09.897689 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:09.898460 10123 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.898717 10664 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:09.899212 10664 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.899595 10664 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:09.901595 10250 tablet_bootstrap.cc:654] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:09.903342 10570 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.908270 10570 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.904841 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.911427 10570 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.911832 10126 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:09.912998 10644 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.912662 10605 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.907806 10678 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.896385 10721 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:09.893356 10054 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:09.923792 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:09.923882 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:09.933604 10640 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:09.932261 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:09.934578 10640 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:09.934965 10201 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:09.934759 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:09.930284 10123 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:09.935705 10640 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.950655 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:09.924229 10721 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:09.955565 10605 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:09.953023 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:09.961467 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:09.962543 10054 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.967904 10570 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:09.970093 10570 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:09.970551 10570 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.977473 10056 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:09.936453 10664 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:09.980497 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:09.981637 10125 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:09.982957 10644 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:09.985939 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.067591 10659 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:10.068342 10659 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:10.069407 10659 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.070281 10644 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:10.073776 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 85753.1us
I20260709 13:16:10.075634 10645 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:10.075886 10020 leader_election.cc:343] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.076260 10645 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.076772 10054 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:16:10.081519 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 88331.4us
I20260709 13:16:10.091890 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:10.092784 10203 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.093772 10249 tablet_bootstrap.cc:492] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:10.095070 10249 ts_tablet_manager.cc:1403] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.394s	user 0.024s	sys 0.021s
I20260709 13:16:10.078886 10645 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.097478 10286 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.098060 10286 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.099871 10250 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:10.100092 10286 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.100476 10250 ts_tablet_manager.cc:1403] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.285s	user 0.016s	sys 0.006s
I20260709 13:16:10.103803 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:10.104710 10054 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.110066 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.093566 10664 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.126680 10203 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:10.130121 10249 raft_consensus.cc:359] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.131210 10249 raft_consensus.cc:385] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.131531 10249 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.114534 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.132385 10249 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:10.141070 10096 leader_election.cc:343] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.115047 10665 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:10.135178 10249 ts_tablet_manager.cc:1434] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.040s	user 0.003s	sys 0.003s
I20260709 13:16:10.150959 10054 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
W20260709 13:16:10.152719 10020 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.154003 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.155181 10599 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:10.155742 10742 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.155778 10599 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:10.156471 10742 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.157100 10599 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.160804 10250 raft_consensus.cc:359] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.161902 10250 raft_consensus.cc:385] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.162237 10250 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.163054 10250 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.180284 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:10.183456 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.184379 10202 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:10.189795 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:10.194196 10250 ts_tablet_manager.cc:1434] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.093s	user 0.009s	sys 0.002s
I20260709 13:16:10.114923 10570 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.205756 10249 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:10.205592 10642 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:10.208745 10642 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:10.162483 10721 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.163952 10665 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:10.222062 10250 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:10.200176 10742 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.228821 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:10.215907 10642 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.216099 10664 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.242125 10664 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.256405 10721 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.249464 10665 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.258663 10664 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.234297 10125 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:10.299939 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.304277 10056 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.305097 10051 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:10.335918 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.337399 10125 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.342546 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.347930 10638 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:10.349556 10638 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:10.353267 10638 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.354648 10570 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.361163 10743 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.364709 10286 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:10.362250 10743 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.366422 10286 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.378185 10721 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.379282 10721 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.389246 10286 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:10.394630 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:10.412751 10672 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.413950 10672 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.419920 10672 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.405314  9954 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:10.434099 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.451565 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.452982 10128 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:10.463275 10250 tablet_bootstrap.cc:654] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.463636 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:10.466812 10128 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:10.466816 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:10.468217  9953 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:10.471716 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:10.472386 10249 tablet_bootstrap.cc:654] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.439954 10605 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.458249 10721 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.485349 10124 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:10.470902  9953 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:10.496650 10128 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:10.497567 10096 leader_election.cc:343] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.498917 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:10.498624 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.508857 10054 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:10.509914 10050 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:10.511655 10743 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.499584 10605 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.514142 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:10.515417 10742 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.511101 10130 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:10.517024 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:10.524768 10020 leader_election.cc:343] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.527066 10125 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:10.529735 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.530875 10054 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:10.532428 10050 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:10.536782 10605 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.537376 10721 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:10.539361 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:10.539796 10721 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:10.540365 10721 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:10.541245 10640 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:10.542145 10640 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.541631 10721 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.542616 10640 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.543421 10286 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.543814 10721 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.544198 10286 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.546725 10286 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.562952 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.563906 10196 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:10.565333 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.566164 10054 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:10.565932 10676 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:10.569090 10056 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:10.572750 10096 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.574548 10659 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:10.575058 10659 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:10.575577 10659 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:10.582538 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.583583 10124 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:10.565333 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:10.603551 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.601112 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:10.620826 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:10.630160  9952 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:10.633093 10676 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:10.642210 10047 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:10.635419 10170 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:16:10.640655 10748 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:16:10.647890 10676 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:10.648064 10659 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.648545 10676 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.649021 10676 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.651518 10659 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.655788 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:10.656106 10124 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:10.657521 10249 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:10.660658 10170 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:10.665442 10742 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.677338 10638 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:10.678138 10638 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:10.679342 10638 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.681833 10249 ts_tablet_manager.cc:1403] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.476s	user 0.019s	sys 0.009s
I20260709 13:16:10.671032 10020 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:10.677416 10640 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.688419 10250 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:10.689532 10250 ts_tablet_manager.cc:1403] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.468s	user 0.019s	sys 0.019s
I20260709 13:16:10.692787 10640 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.695338 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.708842 10649 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:10.709436 10649 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.709827 10649 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.713225 10250 raft_consensus.cc:359] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.714520 10250 raft_consensus.cc:385] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.714987 10250 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
W20260709 13:16:10.717087 10095 leader_election.cc:343] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.719867 10250 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.723420 10250 ts_tablet_manager.cc:1434] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.033s	user 0.012s	sys 0.002s
I20260709 13:16:10.725453 10250 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:10.718833 10124 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.741783 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:10.755462 10053 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.756158 10054 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:10.756698 10249 raft_consensus.cc:359] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.757440 10249 raft_consensus.cc:385] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.757709 10249 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.758396 10249 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.759222 10050 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:10.759532 10649 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.762156 10649 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:10.762578 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:10.763686 10049 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.750293 10676 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.766858 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:10.770656 10096 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.778265 10678 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:16:10.780565 10665 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:16:10.782119 10676 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:10.786505 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.787636 10129 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:10.793030 10249 ts_tablet_manager.cc:1434] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.000s
I20260709 13:16:10.788900 10049 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:10.797190 10096 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.797533 10096 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:10.809458 10124 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:10.812103 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:10.823189 10659 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:10.813155 10676 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:10.826728 10676 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:10.829290 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:10.830284 10199 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:10.828437 10053 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:10.826086 10250 tablet_bootstrap.cc:654] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.833832 10249 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:10.844107 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.845459 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.846652 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.847626 10198 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.850878 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.860841 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:10.862135 10199 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.862419 10204 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:10.830176 10676 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.864367 10198 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:10.864971 10198 pending_rounds.cc:85] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 1
I20260709 13:16:10.865351 10198 pending_rounds.cc:107] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:10.865715 10198 raft_consensus.cc:2889] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:10.866147 10129 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:10.866863 10665 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.867606 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:10.871798 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:10.878654 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:10.880494 10742 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:10.881243 10742 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:10.890488 10126 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:10.899736 10095 leader_election.cc:343] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.900830 10095 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:10.915591 10743 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:10.925194 10095 leader_election.cc:343] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.927222 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:10.932731 10665 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.933800 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:10.935895 10640 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:16:10.939195 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:10.940733 10573 raft_consensus.cc:2764] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:10.938971 10249 tablet_bootstrap.cc:654] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:10.928550 10742 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.947264 10250 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:10.947873 10250 ts_tablet_manager.cc:1403] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.223s	user 0.016s	sys 0.011s
I20260709 13:16:10.946942 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:10.957758 10665 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:10.958633 10198 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:10.956717 10747 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:10.961671 10250 raft_consensus.cc:359] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.962790 10250 raft_consensus.cc:385] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:10.963212 10250 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:10.961887 10644 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:10.964263 10250 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:10.973491 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
W20260709 13:16:10.978134 10095 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:10.978968 10095 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:10.979241 10659 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:10.980293 10664 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:10.983708 10743 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:16:11.018970 10250 ts_tablet_manager.cc:1434] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.005s
I20260709 13:16:11.028023 10250 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:11.093221 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:11.094813 10763 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:16:11.118996 10250 tablet_bootstrap.cc:654] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.139083 10676 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:11.144381 10735 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.156111 10640 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.142786 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54422.6us
I20260709 13:16:11.156939 10640 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.188124 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:11.189241 10129 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:11.191372 10249 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:11.191962 10249 ts_tablet_manager.cc:1403] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.378s	user 0.023s	sys 0.007s
I20260709 13:16:11.193588 10250 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:11.194764 10250 ts_tablet_manager.cc:1403] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.024s
I20260709 13:16:11.176358 10649 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.187317 10640 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:11.199898 10056 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:11.203864 10778 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.209657 10735 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:11.217255 10735 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.195907 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:11.228986 10170 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:11.223210 10778 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:11.234269 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80726us
I20260709 13:16:11.237509 10249 raft_consensus.cc:359] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.238391 10249 raft_consensus.cc:385] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.238687 10249 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.212677 10649 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.233937 10721 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.244920  9954 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:11.241494 10605 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:16:11.248101  9954 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:11.251559 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79226.2us
I20260709 13:16:11.247843 10249 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.254784 10249 ts_tablet_manager.cc:1434] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 13:16:11.258966 10126 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:11.262490 10605 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.269191 10250 raft_consensus.cc:359] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.277299 10250 raft_consensus.cc:385] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.277750 10250 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.281045 10649 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.282572 10250 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.293406 10778 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.303529 10129 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:11.307778 10249 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Bootstrap starting.
W20260709 13:16:11.308074 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64991.9us
I20260709 13:16:11.308511 10640 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.307910 10250 ts_tablet_manager.cc:1434] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.000s
I20260709 13:16:11.319020 10250 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Bootstrap starting.
I20260709 13:16:11.326681 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:11.343523 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:11.350379 10198 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
W20260709 13:16:11.355089 10020 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.363874 10019 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:11.364962 10642 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:11.364794 10778 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.366453 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:11.367722 10783 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.368233 10783 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.368556 10783 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.380620 10250 tablet_bootstrap.cc:654] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.387243 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:11.408099 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:11.409085 10127 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:11.388341 10122 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:11.416021 10047 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:11.418788 10605 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.394189 10721 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.399452 10783 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:11.498311 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84004.8us
I20260709 13:16:11.538565 10250 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: No bootstrap required, opened a new log
I20260709 13:16:11.539409 10250 ts_tablet_manager.cc:1403] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent bootstrapping tablet: real 0.221s	user 0.025s	sys 0.028s
I20260709 13:16:11.542201 10782 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.542846 10782 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.545722 10782 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.549010 10249 tablet_bootstrap.cc:654] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.556828 10783 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.565232 10743 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.566164 10743 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.569597 10743 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.570326 10126 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:11.575995 10605 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.579681 10250 raft_consensus.cc:359] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.580565 10250 raft_consensus.cc:385] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.580914 10250 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.584751 10250 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.585600 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:11.595101 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:11.596205 10196 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.598892 10743 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:11.599715 10743 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:11.602072 10020 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.607249 10743 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.613637 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:11.614436 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:11.617183 10095 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.617446 10196 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:11.618927 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:11.619933 10649 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:11.620507 10649 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:11.621418 10649 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.632089 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:11.613956  9952 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.632987 10049 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 0.
I20260709 13:16:11.641152 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:11.645000 10250 ts_tablet_manager.cc:1434] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.000s
I20260709 13:16:11.639608  9952 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.647814  9952 catalog_manager.cc:5697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.656169 10053 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:11.657852 10249 tablet_bootstrap.cc:492] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:11.663884 10249 ts_tablet_manager.cc:1403] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.356s	user 0.019s	sys 0.011s
I20260709 13:16:11.664012 10742 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:11.669854 10249 raft_consensus.cc:359] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.675776 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:11.678364 10249 raft_consensus.cc:385] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.678606 10130 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:11.679046 10249 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.659690  9952 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 1, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:11.684755 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:11.685510 10130 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.685956 10249 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.690693 10249 ts_tablet_manager.cc:1434] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:16:11.692081 10249 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:11.707842 10130 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:11.710283 10019 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:11.715472 10642 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.715981 10642 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.716372 10642 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.718523 10249 tablet_bootstrap.cc:654] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.729030 10096 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:11.733894 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:11.735219 10678 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.736548 10678 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.736930 10678 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.736275  9954 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:11.741767 10286 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:11.753562 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:11.755890 10642 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.760154 10642 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.758347 10019 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:11.762413 10783 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:11.761236 10678 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.762967 10783 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.763468 10783 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.765538 10678 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.769510 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:11.772404 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:11.772148 10783 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:11.774554 10020 leader_election.cc:343] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.774636 10783 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:11.776006 10095 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.778484 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:11.779855 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:11.780568 10131 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.781131 10783 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.781140 10020 leader_election.cc:343] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.781908 10783 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.784157 10783 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:11.786338 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:11.787920 10249 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
W20260709 13:16:11.788229 10020 leader_election.cc:343] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:11.788484 10249 ts_tablet_manager.cc:1403] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.001s
I20260709 13:16:11.792227 10249 raft_consensus.cc:359] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.792979 10249 raft_consensus.cc:385] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.793244 10249 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.797040 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:11.797973 10127 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 1: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:11.799244 10249 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.801957 10249 ts_tablet_manager.cc:1434] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:16:11.803169 10249 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:11.814978 10249 tablet_bootstrap.cc:654] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.817799 10131 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:11.821919 10249 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:11.822654 10249 ts_tablet_manager.cc:1403] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 13:16:11.825964 10249 raft_consensus.cc:359] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.826814 10249 raft_consensus.cc:385] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:11.827241 10249 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:11.836750 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:11.829324 10249 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.973224 10019 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:11.984639 10249 ts_tablet_manager.cc:1434] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.162s	user 0.015s	sys 0.014s
I20260709 13:16:11.984300 10601 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:11.985986 10249 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:11.976490 10196 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:11.987943 10047 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:11.991142 10742 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.991779 10742 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:11.993623 10249 tablet_bootstrap.cc:654] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:11.994670 10742 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:11.994913 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 159294us
W20260709 13:16:11.998317 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52559.2us
I20260709 13:16:11.984841 10019 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:12.007421 10783 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.008965 10644 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.009654 10644 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:12.011886 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:12.012809 10131 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 0.
I20260709 13:16:12.010655 10644 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.018693 10249 tablet_bootstrap.cc:492] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:12.019372 10249 ts_tablet_manager.cc:1403] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.034s	user 0.005s	sys 0.015s
I20260709 13:16:12.020874 10605 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.021621 10605 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.022641 10249 raft_consensus.cc:359] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.023589 10249 raft_consensus.cc:385] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.023897 10249 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.024096 10605 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:12.025467 10249 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.025951 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:12.026935 10049 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:12.027388 10249 ts_tablet_manager.cc:1434] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:16:12.029011 10249 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Bootstrap starting.
I20260709 13:16:12.035813 10096 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:12.036921 10805 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.039994 10249 tablet_bootstrap.cc:654] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:12.054201 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:12.055656 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:12.061244 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:12.062321 10126 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:12.064293 10249 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: No bootstrap required, opened a new log
I20260709 13:16:12.066174 10249 ts_tablet_manager.cc:1403] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.003s
I20260709 13:16:12.070487 10168 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:12.072568 10168 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:12.074385 10605 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.074729 10795 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.076906 10249 raft_consensus.cc:359] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.077768 10249 raft_consensus.cc:385] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:12.078064 10249 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Initialized, Role: FOLLOWER
I20260709 13:16:12.079101 10249 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.081496 10249 ts_tablet_manager.cc:1434] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:16:12.092337  9954 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.095897 10638 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:12.096643 10638 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.099006 10795 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.103442 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:12.104295 10642 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.105736 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:12.107612 10638 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:12.118892 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:12.119448 10783 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:12.120091 10783 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.120541 10783 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.123514 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:12.124676 10638 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:12.137742 10783 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.140377 10783 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:12.154313 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:12.155061 10126 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.163070 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:12.163910 10205 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.176720 10126 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:12.187090 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:12.189539 10649 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:12.193149 10205 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:12.193545 10649 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:12.194555 10196 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.197755 10573 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.203220 10649 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.205567 10730 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.198959  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.264356  9952 catalog_manager.cc:5697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:12.393018 10795 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.393560 10795 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:12.396584 10795 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:12.396912 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:12.399243 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:12.400398 10126 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 1: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:12.401525 10168 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:12.402272 10795 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:12.408268 10795 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:12.475504 10649 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:12.490204 10730 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.773351 10649 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:12.797853 10649 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:12.801769 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:14.202594 10825 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.203265 10825 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:14.206985 10825 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:14.263860 10829 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.265650 10829 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:14.270895 10829 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:14.325909 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:14.329989 10054 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:14.339887 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:14.340844 10205 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:14.346406 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:14.352425 10824 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:14.353097 10824 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.353516 10824 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:14.362910 10002 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:14.387668 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:14.388775 10124 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:14.401453 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:14.402416 10131 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:14.402379 10824 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:14.418919 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:14.419694 10047 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.429592 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:14.430435 10126 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:14.434376 10824 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:14.436344 10019 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:14.443468 10126 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:14.428450 10047 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:14.444698 10827 raft_consensus.cc:2764] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:14.445047 10168 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:14.446002 10824 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:14.446516 10824 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:14.448308 10824 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:14.467057  9952 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 0 to 2, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:14.956909 10130 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:14.956909 10049 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:14.959628 10824 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:14.960783 10825 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:15.204558 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800050060 after lost signal to thread 8928
W20260709 13:16:15.205924 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800037c40 after lost signal to thread 9988
W20260709 13:16:15.207275 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4a00 after lost signal to thread 10080
W20260709 13:16:15.208395 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002bb100 after lost signal to thread 10155
W20260709 13:16:15.209494 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002d27a0 after lost signal to thread 10229
I20260709 13:16:16.358834  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:16.409449 10854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:16.410223 10853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:16.418226  8925 server_base.cc:1061] running on GCE node
W20260709 13:16:16.420042 10857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:16.427718  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:16.428234  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:16.428570  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783602976428540 us; error 0 us; skew 500 ppm
I20260709 13:16:16.429749  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:16.449174  8925 webserver.cc:533] Webserver started at http://127.8.183.68:34889/ using document root <none> and password file <none>
I20260709 13:16:16.450016  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:16.450381  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:16.450837  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:16.453102  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root/instance:
uuid: "db052bef7536453fa2d7f5a00e314a18"
format_stamp: "Formatted at 2026-07-09 13:16:16 on dist-test-slave-ch3x"
I20260709 13:16:16.462306  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:16.468827 10862 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:16.470901  8925 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:16.471580  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
uuid: "db052bef7536453fa2d7f5a00e314a18"
format_stamp: "Formatted at 2026-07-09 13:16:16 on dist-test-slave-ch3x"
I20260709 13:16:16.475360  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:16.513208  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:16.515828  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:16.540395  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:16.556707  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:16.557231  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:16:16.557648  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:16.557964  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:16.773254  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.68:44775
I20260709 13:16:16.773288 10924 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.68:44775 every 8 connection(s)
I20260709 13:16:16.815944  8925 auto_leader_rebalancer-test.cc:384] add a tserver: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:16.842607 10925 heartbeater.cc:344] Connected to a master server at 127.8.183.126:40341
I20260709 13:16:16.843209 10925 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:16.844255 10925 heartbeater.cc:507] Master 127.8.183.126:40341 requested a full tablet report, sending...
I20260709 13:16:16.848012  9952 ts_manager.cc:194] Registered new tserver with Master: db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:16.850283  9952 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43646
I20260709 13:16:17.731997 10196 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:17.751739 10126 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:17.758924 10047 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:17.768445 10934 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:16:17.776062 10824 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:17.803061 10169 consensus_peers.cc:597] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dc97aed5eb284c0e86c74b20d18b4399. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:17.805495 10852 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:17.809021 10047 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:17.813665 10130 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:17.842844  9952 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:17.863494 10925 heartbeater.cc:499] Master 127.8.183.126:40341 was elected leader, sending a full tablet report...
I20260709 13:16:17.895085 10047 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:17.925547 10196 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:17.928200 10936 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:18.037137 10130 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:18.050257 10947 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:16:18.079206 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87605.2us
W20260709 13:16:18.090693 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45703us
I20260709 13:16:18.107365 10837 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.109748 10196 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:16:18.128804 10020 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a82842926e645c08dde847bfe182c31. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:18.136209  9953 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.136436 10127 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.146356 10049 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.183285 10196 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:18.184837 10020 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d97ed0ec5f214d95abb9e01d368a04b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:18.185159 10127 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:18.186575 10837 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:18.194468 10958 ts_tablet_manager.cc:933] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:18.198920 10837 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:18.214951 10958 tablet_copy_client.cc:323] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:18.219749 10837 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.224160 10199 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.265430  9954 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.264434 10127 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.309700 10199 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.333951 10215 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet dc97aed5eb284c0e86c74b20d18b4399 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:18.334715 10215 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet dc97aed5eb284c0e86c74b20d18b4399 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-dc97aed5eb284c0e86c74b20d18b4399
W20260709 13:16:18.351032 10169 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ece395e49d7e474182be2d25642b90de. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:18.364665 10126 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:18.365415 10049 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:18.369660 10824 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:18.371411 10934 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:18.348532 10215 tablet_copy_source_session.cc:215] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:18.393702 10966 ts_tablet_manager.cc:933] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:18.403195 10958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc97aed5eb284c0e86c74b20d18b4399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.408228 10935 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.410926 10047 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.455188 10966 tablet_copy_client.cc:323] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:18.514384 10958 tablet_copy_client.cc:806] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:18.509814  9954 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.516491 10128 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:18.523309 10958 tablet_copy_client.cc:670] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:18.548483 10958 tablet_copy_client.cc:538] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:18.625838 10066 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 7a82842926e645c08dde847bfe182c31 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:18.626685 10066 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 7a82842926e645c08dde847bfe182c31 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-7a82842926e645c08dde847bfe182c31
I20260709 13:16:18.625380 10958 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:18.637820 10066 tablet_copy_source_session.cc:215] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:18.642565 10966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a82842926e645c08dde847bfe182c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.689438 10966 tablet_copy_client.cc:806] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:18.690133 10966 tablet_copy_client.cc:670] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:18.698573 10958 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:18.700964 10966 tablet_copy_client.cc:538] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:18.699985 10958 tablet_bootstrap.cc:492] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:18.715106 10958 ts_tablet_manager.cc:1403] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.094s	user 0.024s	sys 0.029s
I20260709 13:16:18.720764 10958 raft_consensus.cc:359] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.722086 10958 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:18.725310 10966 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:18.723053 10958 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.744983 10958 ts_tablet_manager.cc:1434] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:16:18.747844 10215 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-dc97aed5eb284c0e86c74b20d18b4399 received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:18.748502 10215 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-dc97aed5eb284c0e86c74b20d18b4399 on tablet dc97aed5eb284c0e86c74b20d18b4399 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:18.752578 10958 ts_tablet_manager.cc:933] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:18.768478 10958 tablet_copy_client.cc:323] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:18.776937 10066 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet d97ed0ec5f214d95abb9e01d368a04b5 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:18.777719 10066 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet d97ed0ec5f214d95abb9e01d368a04b5 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-d97ed0ec5f214d95abb9e01d368a04b5
I20260709 13:16:18.781028 10966 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:18.782267 10966 tablet_bootstrap.cc:492] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:18.783812 10966 ts_tablet_manager.cc:1403] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.059s	user 0.029s	sys 0.003s
I20260709 13:16:18.787890 10966 raft_consensus.cc:359] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.790347 10966 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:18.789628 10066 tablet_copy_source_session.cc:215] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:18.791297 10966 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:18.794243 10966 ts_tablet_manager.cc:1434] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:16:18.797307 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-7a82842926e645c08dde847bfe182c31 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:18.797935 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-7a82842926e645c08dde847bfe182c31 on tablet 7a82842926e645c08dde847bfe182c31 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:18.809734 10958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d97ed0ec5f214d95abb9e01d368a04b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.820251 10900 raft_consensus.cc:1217] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:18.835310 10958 tablet_copy_client.cc:806] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:18.836149 10958 tablet_copy_client.cc:670] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:18.840835 10958 tablet_copy_client.cc:538] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:18.855225 10958 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
W20260709 13:16:18.940918 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58905.8us
W20260709 13:16:18.986438 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48130.9us
I20260709 13:16:19.001133 10958 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:19.004405 10958 tablet_bootstrap.cc:492] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:19.005412 10958 ts_tablet_manager.cc:1403] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.150s	user 0.056s	sys 0.059s
W20260709 13:16:19.003984 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59769.4us
I20260709 13:16:19.009075 10958 raft_consensus.cc:359] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:19.010094 10958 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:19.010833 10958 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:19.013361 10958 ts_tablet_manager.cc:1434] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:16:19.024829 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-d97ed0ec5f214d95abb9e01d368a04b5 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:19.025413 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-d97ed0ec5f214d95abb9e01d368a04b5 on tablet d97ed0ec5f214d95abb9e01d368a04b5 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:19.025610 10900 raft_consensus.cc:1217] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:19.044251 10958 ts_tablet_manager.cc:933] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:19.083578 10958 tablet_copy_client.cc:323] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:19.087245 10215 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet ece395e49d7e474182be2d25642b90de from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:19.087996 10215 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet ece395e49d7e474182be2d25642b90de from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-ece395e49d7e474182be2d25642b90de
I20260709 13:16:19.109719 10215 tablet_copy_source_session.cc:215] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:19.117017 10900 raft_consensus.cc:1217] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:19.135764 10958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ece395e49d7e474182be2d25642b90de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.159559 10958 tablet_copy_client.cc:806] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:19.160320 10958 tablet_copy_client.cc:670] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:19.167299 10958 tablet_copy_client.cc:538] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:19.181025 10958 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:19.213863 10958 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:19.215056 10958 tablet_bootstrap.cc:492] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:19.215927 10958 ts_tablet_manager.cc:1403] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.011s
I20260709 13:16:19.219419 10958 raft_consensus.cc:359] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:19.220456 10958 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:19.221252 10958 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:19.223579 10958 ts_tablet_manager.cc:1434] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:19.225682 10215 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-ece395e49d7e474182be2d25642b90de received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:19.226205 10215 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-ece395e49d7e474182be2d25642b90de on tablet ece395e49d7e474182be2d25642b90de with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:19.441619 10852 raft_consensus.cc:1064] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:19.445436 10852 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:19.456853 10049 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.458799 10900 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.461194 10935 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.464938 10824 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.468981 10128 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.486125 10935 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:16:19.511224 10835 raft_consensus.cc:1064] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:19.514786 10835 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:19.562381 10196 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.566102 10899 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.567903 10837 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.572579 10835 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:16:19.577983 10934 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.595058 10127 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.585603 10900 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.597766 10835 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.607312 10898 raft_consensus.cc:1217] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:16:19.641695 10128 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.650872 10832 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.678161 10899 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.700070 10989 raft_consensus.cc:1064] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:19.704531 10989 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:19.732290 10898 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.734786 10127 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.734715 10835 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.746099 10989 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.747983 10196 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:19.751742  9953 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.761461 10989 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:19.780354 10054 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.782503  9951 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.788601 10196 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.792484 10837 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.816649 10900 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.847775 10127 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.895066 10201 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.898435 10131 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:19.987828 10047 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:20.001344 10201 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.004410 10900 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.006316 10988 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.010372 10988 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.010651 10201 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:20.020020 10835 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.021661 10196 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.036271 10900 raft_consensus.cc:2955] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.043226  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7a82842926e645c08dde847bfe182c31 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:20.047554 10127 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.047956 10898 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.055509 10852 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.064749  9953 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.066491 10934 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.073264 10900 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.076221 10935 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.082652 10824 raft_consensus.cc:1064] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:20.085439 10824 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:20.102648 10131 raft_consensus.cc:2955] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.167498  9950 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.166896 10898 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:20.182799 10935 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:20.187811 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a82842926e645c08dde847bfe182c31 with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:20.190196 10131 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:20.198693 10987 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:20.213572 10054 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:20.244544 11001 tablet_replica.cc:333] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:20.245325  9952 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.220222 10987 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:16:20.260536  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dc97aed5eb284c0e86c74b20d18b4399 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:20.248083 10047 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:20.283674 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet dc97aed5eb284c0e86c74b20d18b4399 with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:20.286031 10852 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.290176 10131 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.299491 10900 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.303670 10989 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.307907 10127 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:20.311969 10989 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:20.323351 10835 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.328904 11004 tablet_replica.cc:333] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:20.328613 10126 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.334255  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d97ed0ec5f214d95abb9e01d368a04b5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:20.338508 10898 raft_consensus.cc:2955] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.368285  9952 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.381088 11004 raft_consensus.cc:2243] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:20.379482  9950 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.382055 11004 raft_consensus.cc:2272] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:20.396188 11001 raft_consensus.cc:2243] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:20.397534 11001 raft_consensus.cc:2272] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:20.415597 10898 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:20.422765 11004 ts_tablet_manager.cc:1916] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:20.426031 10047 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:20.627584 11006 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.187s	user 0.004s	sys 0.005s
I20260709 13:16:20.663700 11001 ts_tablet_manager.cc:1916] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:20.845374 11004 ts_tablet_manager.cc:1929] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:20.846480 11004 log.cc:1199] T dc97aed5eb284c0e86c74b20d18b4399 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/dc97aed5eb284c0e86c74b20d18b4399
I20260709 13:16:20.898428 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet d97ed0ec5f214d95abb9e01d368a04b5 with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:20.929117  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet dc97aed5eb284c0e86c74b20d18b4399 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:20.941370 11001 ts_tablet_manager.cc:1929] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:20.942502 11001 log.cc:1199] T 7a82842926e645c08dde847bfe182c31 P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/7a82842926e645c08dde847bfe182c31
I20260709 13:16:20.958820  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet 7a82842926e645c08dde847bfe182c31 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:21.027182 11019 tablet_replica.cc:333] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:21.093662 11019 raft_consensus.cc:2243] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:21.095079 11019 raft_consensus.cc:2272] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:21.102644 11019 ts_tablet_manager.cc:1916] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:21.170006 11019 ts_tablet_manager.cc:1929] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:21.206338 11019 log.cc:1199] T d97ed0ec5f214d95abb9e01d368a04b5 P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/d97ed0ec5f214d95abb9e01d368a04b5
I20260709 13:16:21.213828  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet d97ed0ec5f214d95abb9e01d368a04b5 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:21.445756 10201 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:21.470542 10900 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:21.476217 10934 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:21.489101 10047 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:21.491407 10824 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:21.498413 11014 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:21.502517 10047 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:21.502524 10900 raft_consensus.cc:2955] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:21.512221  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ece395e49d7e474182be2d25642b90de with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:21.532172  9954 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:21.555229 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet ece395e49d7e474182be2d25642b90de with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:21.564393 11025 tablet_replica.cc:333] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:21.567085 11025 raft_consensus.cc:2243] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:21.568259 11025 raft_consensus.cc:2272] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:21.572129 11025 ts_tablet_manager.cc:1916] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:21.589769 11025 ts_tablet_manager.cc:1929] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:16:21.590147 11025 log.cc:1199] T ece395e49d7e474182be2d25642b90de P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/ece395e49d7e474182be2d25642b90de
I20260709 13:16:21.591701  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet ece395e49d7e474182be2d25642b90de (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
W20260709 13:16:22.991787 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84614.8us
W20260709 13:16:23.010147 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 96039.8us
I20260709 13:16:23.139359 10049 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:16:23.147912 10020 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a895dc8dee14dfba216f443121474f4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:23.148455 10201 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.148020 10128 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.150142 11038 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:23.151770 11035 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.162026 11035 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.164499 10205 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.165408 10128 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.190160  9952 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.222189 10128 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:23.236032 10199 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.238255 10053 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.240935 11042 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.255244 11000 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.262332 11040 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.264412 10053 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.299085 10199 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:23.299333 10205 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:16:23.304906 10095 consensus_peers.cc:597] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ce48110a5fb4ffc97e81117165e3ffd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:23.299751  9954 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.313113 10128 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:23.316417 11032 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.317090 10049 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:23.321926 10824 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:23.325707 10169 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a7190358a9f475d82e5f9875677bccf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:23.358124 10824 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.360083 10049 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.363729 10128 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.395432 10199 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:23.400875  9954 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:16:23.433697 10169 consensus_peers.cc:597] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ecb570a023c42ada8943a1516f3ef4f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:23.434952 10128 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.437438 11014 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:23.440425 10050 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:23.443655 11032 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:23.454941 11014 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.462203 10054 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.493724 10127 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:23.494024  9950 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.674315 11064 ts_tablet_manager.cc:933] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:23.676766 11064 tablet_copy_client.cc:323] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:23.679234 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 9a895dc8dee14dfba216f443121474f4 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:23.679919 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 9a895dc8dee14dfba216f443121474f4 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-9a895dc8dee14dfba216f443121474f4
I20260709 13:16:23.686839 10065 tablet_copy_source_session.cc:215] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:23.691071 11064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a895dc8dee14dfba216f443121474f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:23.707245 11064 tablet_copy_client.cc:806] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:23.708036 11064 tablet_copy_client.cc:670] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:23.713954 11064 tablet_copy_client.cc:538] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:16:23.721760 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800282080 after lost signal to thread 8928
W20260709 13:16:23.723333 10226 debug-util.cc:398] Leaking SignalData structure 0x7b080029cb00 after lost signal to thread 9988
W20260709 13:16:23.724913 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800038260 after lost signal to thread 10080
W20260709 13:16:23.726325 10226 debug-util.cc:398] Leaking SignalData structure 0x7b080027d980 after lost signal to thread 10155
W20260709 13:16:23.727828 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08000618a0 after lost signal to thread 10229
I20260709 13:16:23.728957 11064 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:23.760390 11066 ts_tablet_manager.cc:933] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:23.856369 11069 ts_tablet_manager.cc:933] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:23.871423 11069 tablet_copy_client.cc:323] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.66:41681
I20260709 13:16:23.911588 10141 tablet_copy_service.cc:140] P e9a849ac7eda426abfca97cf8b15da0f: Received BeginTabletCopySession request for tablet 4ce48110a5fb4ffc97e81117165e3ffd from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:45470)
I20260709 13:16:23.912281 10141 tablet_copy_service.cc:161] P e9a849ac7eda426abfca97cf8b15da0f: Beginning new tablet copy session on tablet 4ce48110a5fb4ffc97e81117165e3ffd from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:45470: session id = db052bef7536453fa2d7f5a00e314a18-4ce48110a5fb4ffc97e81117165e3ffd
I20260709 13:16:23.922258 10141 tablet_copy_source_session.cc:215] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:23.931802 11069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce48110a5fb4ffc97e81117165e3ffd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:23.941824 11066 tablet_copy_client.cc:323] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:23.974047 11069 tablet_copy_client.cc:806] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:24.069347 11069 tablet_copy_client.cc:670] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:24.232944 11073 ts_tablet_manager.cc:933] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:24.235730 10215 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet 6a7190358a9f475d82e5f9875677bccf from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:24.236586 10215 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet 6a7190358a9f475d82e5f9875677bccf from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-6a7190358a9f475d82e5f9875677bccf
I20260709 13:16:24.344050 11073 tablet_copy_client.cc:323] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
W20260709 13:16:24.349387 11064 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.100s	sys 0.170s
W20260709 13:16:24.350142 11064 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.100s	sys 0.170s
W20260709 13:16:24.385267 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60185.5us
I20260709 13:16:24.407042 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet 5ecb570a023c42ada8943a1516f3ef4f from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:24.407855 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet 5ecb570a023c42ada8943a1516f3ef4f from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-5ecb570a023c42ada8943a1516f3ef4f
I20260709 13:16:24.411701 11069 tablet_copy_client.cc:538] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:16:24.493430 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105375us
W20260709 13:16:24.498899 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 110107us
I20260709 13:16:24.517027 11064 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:24.519398 10215 tablet_copy_source_session.cc:215] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:16:24.529644 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63114.5us
I20260709 13:16:24.529098 11064 tablet_bootstrap.cc:492] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:24.545222 10214 tablet_copy_source_session.cc:215] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:24.549234 11064 ts_tablet_manager.cc:1403] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.821s	user 0.128s	sys 0.180s
I20260709 13:16:24.553372 11064 raft_consensus.cc:359] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.554409 11064 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:24.563692 11069 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:24.580125 11066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a7190358a9f475d82e5f9875677bccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:24.582070 11073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecb570a023c42ada8943a1516f3ef4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:24.652186 11066 tablet_copy_client.cc:806] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:24.660580 11066 tablet_copy_client.cc:670] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:24.628257 11064 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.684022 11064 ts_tablet_manager.cc:1434] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.134s	user 0.009s	sys 0.003s
I20260709 13:16:24.688191 11066 tablet_copy_client.cc:538] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:24.728698 11069 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:24.729862 11069 tablet_bootstrap.cc:492] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:24.730724 11069 ts_tablet_manager.cc:1403] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.167s	user 0.024s	sys 0.019s
I20260709 13:16:24.733841 11066 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:24.735961 11073 tablet_copy_client.cc:806] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:24.739104 11073 tablet_copy_client.cc:670] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:24.741914 11069 raft_consensus.cc:359] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.749928 11069 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:24.750845 11069 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.752624 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-9a895dc8dee14dfba216f443121474f4 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:24.753371 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-9a895dc8dee14dfba216f443121474f4 on tablet 9a895dc8dee14dfba216f443121474f4 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:24.761258 11073 tablet_copy_client.cc:538] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:24.768148 11069 ts_tablet_manager.cc:1434] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.002s
I20260709 13:16:24.773607 10141 tablet_copy_service.cc:342] P e9a849ac7eda426abfca97cf8b15da0f: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-4ce48110a5fb4ffc97e81117165e3ffd received from {username='slave'} at 127.0.0.1:45470
I20260709 13:16:24.774276 10141 tablet_copy_service.cc:434] P e9a849ac7eda426abfca97cf8b15da0f: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-4ce48110a5fb4ffc97e81117165e3ffd on tablet 4ce48110a5fb4ffc97e81117165e3ffd with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:24.777594 11073 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:24.844636 11073 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:24.845696 11073 tablet_bootstrap.cc:492] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:24.846465 11073 ts_tablet_manager.cc:1403] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.009s
I20260709 13:16:24.849936 11073 raft_consensus.cc:359] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.850907 11073 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:24.858464 11073 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:24.861238 11073 ts_tablet_manager.cc:1434] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:16:24.863474 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-5ecb570a023c42ada8943a1516f3ef4f received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:24.864075 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-5ecb570a023c42ada8943a1516f3ef4f on tablet 5ecb570a023c42ada8943a1516f3ef4f with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:24.906955 10898 raft_consensus.cc:1217] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:25.083287 10898 raft_consensus.cc:1217] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:25.118697 11066 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:25.132208 11066 tablet_bootstrap.cc:492] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:25.133224 11066 ts_tablet_manager.cc:1403] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.400s	user 0.038s	sys 0.018s
I20260709 13:16:25.137282 11066 raft_consensus.cc:359] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:25.138613 11066 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:25.139569 11066 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:25.159360 11066 ts_tablet_manager.cc:1434] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:16:25.215890 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-6a7190358a9f475d82e5f9875677bccf received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:25.216488 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-6a7190358a9f475d82e5f9875677bccf on tablet 6a7190358a9f475d82e5f9875677bccf with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:25.300433 11075 raft_consensus.cc:1064] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:25.304970 11075 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:25.327888 10898 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.330051 11081 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.348042 10196 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.366465 11000 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:16:25.391646 10054 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.401439 10898 raft_consensus.cc:1217] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:25.412933 11104 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.430490 11075 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.436786 10049 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.433255 10202 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.466558  9950 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.479655 10898 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.518754 10898 raft_consensus.cc:1217] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:16:25.542091 10128 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:25.551748 10196 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:25.551883 10898 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:25.554296 11087 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:25.556524 11087 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:25.566972 11104 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.573186 10205 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.581818  9938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ce48110a5fb4ffc97e81117165e3ffd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:25.580127 10898 raft_consensus.cc:2955] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.598534  9952 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.611074 11084 raft_consensus.cc:1064] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:25.614217 11084 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:25.620647 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ce48110a5fb4ffc97e81117165e3ffd with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:25.644424 11111 tablet_replica.cc:333] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:25.646883 11111 raft_consensus.cc:2243] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:25.647938 11111 raft_consensus.cc:2272] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:25.650609 10205 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.652963 11111 ts_tablet_manager.cc:1916] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:25.652698 11026 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.660287 10898 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.663230 11037 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.685120 11111 ts_tablet_manager.cc:1929] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:25.685189 10128 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.685838 11111 log.cc:1199] T 4ce48110a5fb4ffc97e81117165e3ffd P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/4ce48110a5fb4ffc97e81117165e3ffd
I20260709 13:16:25.687927  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet 4ce48110a5fb4ffc97e81117165e3ffd (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:25.690459 11035 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.694408 11089 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.697410 10898 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.716282 10205 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.720666  9954 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.732777 10128 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.758579 10054 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:25.771783 10128 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:25.771783 10898 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:25.773788 11037 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:25.775341 11089 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:25.787160 11063 raft_consensus.cc:1064] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:25.795301 11041 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.790341 11063 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:25.797832 10900 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.807811 10128 raft_consensus.cc:2955] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.834839  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a895dc8dee14dfba216f443121474f4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:25.839834 10898 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.842347 11085 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.833850  9951 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.858449 10131 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.861193 10049 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:25.862810 11032 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:25.880581 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a895dc8dee14dfba216f443121474f4 with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:25.884603 11085 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:25.964381 11121 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.010s
W20260709 13:16:25.962718 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57442.9us
I20260709 13:16:25.973181 11120 tablet_replica.cc:333] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:25.974588 11120 raft_consensus.cc:2243] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:25.977365 11063 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:25.980633 10053 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:25.998623 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91983.9us
I20260709 13:16:26.005533 10900 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.011710 11120 raft_consensus.cc:2272] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.020875 11120 ts_tablet_manager.cc:1916] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:26.021581  9951 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.037199 10128 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.047621 10205 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:26.062077 10898 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:26.069339 11085 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:26.079042 10127 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:26.081384 11085 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:26.090380 11032 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.095186 10127 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.104938 10898 raft_consensus.cc:2955] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.108417 11085 raft_consensus.cc:1064] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:26.112421 11120 ts_tablet_manager.cc:1929] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:26.113098 11120 log.cc:1199] T 9a895dc8dee14dfba216f443121474f4 P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/9a895dc8dee14dfba216f443121474f4
I20260709 13:16:26.117715  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet 9a895dc8dee14dfba216f443121474f4 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:26.119589 11085 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:26.137075  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ecb570a023c42ada8943a1516f3ef4f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:26.162329 10049 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:26.164058 10126 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:26.165017  9954 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.166774 10898 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:26.169101 11063 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:26.171864 11123 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:26.176496 11085 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:26.191987 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ecb570a023c42ada8943a1516f3ef4f with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:26.211263 11085 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.218370 10126 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.228250 11126 tablet_replica.cc:333] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:26.229523 10898 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:26.232394 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67875.1us
I20260709 13:16:26.244622 10049 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.241847  9951 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.283605 11126 raft_consensus.cc:2243] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.284516 11126 raft_consensus.cc:2272] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.290681 11126 ts_tablet_manager.cc:1916] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:26.349546 11126 ts_tablet_manager.cc:1929] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:26.350039 11126 log.cc:1199] T 5ecb570a023c42ada8943a1516f3ef4f P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/5ecb570a023c42ada8943a1516f3ef4f
I20260709 13:16:26.351899  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet 5ecb570a023c42ada8943a1516f3ef4f (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:26.714926 10202 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:26.723217 10053 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:26.723235 10898 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:16:26.726317 11085 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:26.732319 11032 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:26.740646 10900 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.738368 11032 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.753664 10049 raft_consensus.cc:2955] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.764873  9954 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:26.766217  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6a7190358a9f475d82e5f9875677bccf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:26.797777 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a7190358a9f475d82e5f9875677bccf with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:26.801082 11132 tablet_replica.cc:333] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:26.802103 11132 raft_consensus.cc:2243] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:26.803215 11132 raft_consensus.cc:2272] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:26.808683 11132 ts_tablet_manager.cc:1916] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:26.825306 11132 ts_tablet_manager.cc:1929] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:16:26.825939 11132 log.cc:1199] T 6a7190358a9f475d82e5f9875677bccf P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/6a7190358a9f475d82e5f9875677bccf
I20260709 13:16:26.843642  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet 6a7190358a9f475d82e5f9875677bccf (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:27.873961 10053 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:27.881762 10196 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:27.882504 10020 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95b81db48ec24878966db719b2790127. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:27.882772 10124 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:27.883608 11037 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:27.884477 11134 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:27.894594 11134 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:27.897301 10124 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:27.898231 10196 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:27.918289  9954 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:27.923064 10196 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:16:27.970175 10169 consensus_peers.cc:597] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbf6e6a0da3d4501acf0e5451cae4577. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:27.974485 10124 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:27.977355 11133 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:28.025960 10053 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:28.028142 11143 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:28.038426 11145 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.041417 10126 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.068710 10196 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.077390 10053 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.089311 10049 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:16:28.090310 10169 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf88b6d5c6504b3cb2cbd21cff68e57e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:28.091194 10127 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:28.075393  9950 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.091791 11032 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:28.095589 11032 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:28.104502 11133 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.110021 10049 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.110841 10127 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.140053  9950 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.143594 10049 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.161345 10196 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:28.163292 11139 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:28.165711 10127 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:28.168143 11139 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:28.168479 10020 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e46fa8cb2404116bf566b4ef9bbb2cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:28.178424 11089 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.180725 10202 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.187974 10127 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:28.203384  9954 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.340248 11155 ts_tablet_manager.cc:933] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:28.344269 11155 tablet_copy_client.cc:323] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:28.346128 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 95b81db48ec24878966db719b2790127 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:28.346767 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 95b81db48ec24878966db719b2790127 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-95b81db48ec24878966db719b2790127
I20260709 13:16:28.354971 10065 tablet_copy_source_session.cc:215] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:28.359565 11155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b81db48ec24878966db719b2790127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:28.381399 11155 tablet_copy_client.cc:806] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:28.382253 11155 tablet_copy_client.cc:670] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:28.388684 11155 tablet_copy_client.cc:538] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:28.402401 11155 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:28.431480 11155 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:28.432701 11155 tablet_bootstrap.cc:492] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:28.433578 11155 ts_tablet_manager.cc:1403] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.012s
I20260709 13:16:28.437207 11155 raft_consensus.cc:359] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.438637 11155 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:28.439563 11155 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.443501 11155 ts_tablet_manager.cc:1434] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.002s
I20260709 13:16:28.445776 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-95b81db48ec24878966db719b2790127 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:28.446316 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-95b81db48ec24878966db719b2790127 on tablet 95b81db48ec24878966db719b2790127 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:28.496272 11155 ts_tablet_manager.cc:933] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:28.498489 11155 tablet_copy_client.cc:323] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:28.501484 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet dbf6e6a0da3d4501acf0e5451cae4577 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:28.502096 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet dbf6e6a0da3d4501acf0e5451cae4577 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-dbf6e6a0da3d4501acf0e5451cae4577
I20260709 13:16:28.513797 10214 tablet_copy_source_session.cc:215] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:28.518374 11155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf6e6a0da3d4501acf0e5451cae4577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:28.543942 11155 tablet_copy_client.cc:806] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:28.544953 11155 tablet_copy_client.cc:670] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:28.550103 11155 tablet_copy_client.cc:538] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:28.561575 11155 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:28.566638 11160 ts_tablet_manager.cc:933] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:28.576161 11160 tablet_copy_client.cc:323] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:28.578435 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:28.579193 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 5e46fa8cb2404116bf566b4ef9bbb2cf from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-5e46fa8cb2404116bf566b4ef9bbb2cf
I20260709 13:16:28.589015 10065 tablet_copy_source_session.cc:215] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:28.591514 11155 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:28.593979 11155 tablet_bootstrap.cc:492] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:28.594900 11155 ts_tablet_manager.cc:1403] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.034s	user 0.027s	sys 0.002s
I20260709 13:16:28.598403 11155 raft_consensus.cc:359] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.600809 11155 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:28.601750 11155 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.600955 11160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:28.604954 11155 ts_tablet_manager.cc:1434] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:16:28.607188 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-dbf6e6a0da3d4501acf0e5451cae4577 received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:28.607784 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-dbf6e6a0da3d4501acf0e5451cae4577 on tablet dbf6e6a0da3d4501acf0e5451cae4577 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:28.621204 11155 ts_tablet_manager.cc:933] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:28.628077 11155 tablet_copy_client.cc:323] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:28.633122 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet cf88b6d5c6504b3cb2cbd21cff68e57e from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:28.633826 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet cf88b6d5c6504b3cb2cbd21cff68e57e from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-cf88b6d5c6504b3cb2cbd21cff68e57e
I20260709 13:16:28.641004 11160 tablet_copy_client.cc:806] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:28.642199 11160 tablet_copy_client.cc:670] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:28.642005 10214 tablet_copy_source_session.cc:215] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:28.648119 11155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf88b6d5c6504b3cb2cbd21cff68e57e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:28.654081 11160 tablet_copy_client.cc:538] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:28.664995 11160 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:28.672165 11155 tablet_copy_client.cc:806] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:28.672894 11155 tablet_copy_client.cc:670] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:28.678897 11155 tablet_copy_client.cc:538] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:28.691651 11155 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:28.700286 11160 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:28.701485 11160 tablet_bootstrap.cc:492] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:28.702310 11160 ts_tablet_manager.cc:1403] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.002s
I20260709 13:16:28.705829 11160 raft_consensus.cc:359] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.706826 11160 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:28.707645 11160 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.713627 11160 ts_tablet_manager.cc:1434] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:16:28.715786 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-5e46fa8cb2404116bf566b4ef9bbb2cf received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:28.716326 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-5e46fa8cb2404116bf566b4ef9bbb2cf on tablet 5e46fa8cb2404116bf566b4ef9bbb2cf with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:28.754700 11155 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:28.756047 11155 tablet_bootstrap.cc:492] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:28.756920 11155 ts_tablet_manager.cc:1403] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.065s	user 0.026s	sys 0.023s
I20260709 13:16:28.760303 11155 raft_consensus.cc:359] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.761320 11155 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:28.761991 11155 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:28.764379 11155 ts_tablet_manager.cc:1434] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:28.767871 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-cf88b6d5c6504b3cb2cbd21cff68e57e received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:28.768388 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-cf88b6d5c6504b3cb2cbd21cff68e57e on tablet cf88b6d5c6504b3cb2cbd21cff68e57e with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:28.896132 10900 raft_consensus.cc:1217] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:28.953658 10900 raft_consensus.cc:1217] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:16:29.102839 10900 raft_consensus.cc:1217] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:16:29.107582 10900 raft_consensus.cc:1217] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:29.387415 11143 raft_consensus.cc:1064] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:29.390599 11143 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.413810 10900 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.416575 10049 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.416368 10127 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.417065 11133 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:29.419708 11145 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:29.423276 11143 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:29.451691 11145 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.463150 10900 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.474913 10127 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.478102 10049 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.480036 11089 raft_consensus.cc:1064] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:29.484351 11089 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.507285 11174 raft_consensus.cc:1064] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:29.508216  9951 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.520231 11174 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.525014 11134 raft_consensus.cc:1064] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:29.531252 10205 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:29.541000 10898 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.541599 10127 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.542555 11138 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.543077 11173 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:29.544281 11133 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:29.554051 10049 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:29.553838 11134 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.556396 11145 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:29.561558 10900 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:16:29.566020 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80269.1us
I20260709 13:16:29.566522 11037 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.570528 10128 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:29.608642 11138 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.623057 11133 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.634016 10898 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.658623 11138 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.673640 10126 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.679877 10054 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.688951 10898 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.692126 10131 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:29.691999 10900 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:29.694849 11175 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.696790 11185 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.708690 10201 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:29.723898 10205 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.736506 11089 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:29.749404  9954 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.772837 10126 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.770315  9951 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.778913 10202 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.795401 10131 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:29.807498 11133 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:16:29.850795 10199 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:29.852823 10898 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:29.859946 11143 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:29.865387 10054 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:29.868942 11145 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:29.873466 10897 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:29.876111 11174 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:29.876224 11037 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.886127 10900 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.906880 11174 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.909673  9951 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.915781 11133 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.919697 10054 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.918951 10898 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.928217 10900 raft_consensus.cc:2955] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:29.966704 10049 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:29.973348 11194 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:16:29.980293  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dbf6e6a0da3d4501acf0e5451cae4577 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:30.022228 10896 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:30.024098  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cf88b6d5c6504b3cb2cbd21cff68e57e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:29.994565 10131 raft_consensus.cc:2955] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.036478 10124 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:30.049867  9939 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet cf88b6d5c6504b3cb2cbd21cff68e57e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 5
I20260709 13:16:30.059160  9951 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 4 to 5, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.091168 10205 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.101629 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet dbf6e6a0da3d4501acf0e5451cae4577 with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:30.116178  9952 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 4 to 5, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.118201 11202 tablet_replica.cc:333] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:30.119659 11202 raft_consensus.cc:2243] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:30.121285 10127 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:30.127777 10049 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:30.145404 11185 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:30.150713 11202 raft_consensus.cc:2272] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:30.152093 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet cf88b6d5c6504b3cb2cbd21cff68e57e with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45958
W20260709 13:16:30.156618 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58696.3us
I20260709 13:16:30.158960 11138 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:30.166862 11203 tablet_replica.cc:333] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:30.167686 10126 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:30.169032 11037 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.168467 11203 raft_consensus.cc:2243] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:30.171103 11203 raft_consensus.cc:2272] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:30.164197 11202 ts_tablet_manager.cc:1916] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:30.185818 11199 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:30.189210  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 95b81db48ec24878966db719b2790127 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:30.189419 10126 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.214386 11199 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.218856 10124 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.222515 10900 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:30.222335 11203 ts_tablet_manager.cc:1916] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:30.244894  9951 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.255398  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:30.272306 11202 ts_tablet_manager.cc:1929] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:16:30.272974 11202 log.cc:1199] T dbf6e6a0da3d4501acf0e5451cae4577 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/dbf6e6a0da3d4501acf0e5451cae4577
I20260709 13:16:30.275017  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet dbf6e6a0da3d4501acf0e5451cae4577 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:30.266250  9952 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.280545 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet 95b81db48ec24878966db719b2790127 with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:30.296895 11204 tablet_replica.cc:333] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:30.296576 11138 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:16:30.314751 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet 5e46fa8cb2404116bf566b4ef9bbb2cf with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:30.316928 10900 raft_consensus.cc:2955] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.324081 10898 raft_consensus.cc:2955] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:30.336505 11204 raft_consensus.cc:2243] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:30.337551 11204 raft_consensus.cc:2272] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:30.342072 11203 ts_tablet_manager.cc:1929] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:16:30.348367 11203 log.cc:1199] T cf88b6d5c6504b3cb2cbd21cff68e57e P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/cf88b6d5c6504b3cb2cbd21cff68e57e
I20260709 13:16:30.355947 11204 ts_tablet_manager.cc:1916] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:30.356086  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet cf88b6d5c6504b3cb2cbd21cff68e57e (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:30.378736 11204 ts_tablet_manager.cc:1929] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:30.379472 11204 log.cc:1199] T 95b81db48ec24878966db719b2790127 P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/95b81db48ec24878966db719b2790127
I20260709 13:16:30.381403  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet 95b81db48ec24878966db719b2790127 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:30.382556 11204 tablet_replica.cc:333] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:30.383708 11204 raft_consensus.cc:2243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:30.384647 11204 raft_consensus.cc:2272] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:30.389936 11204 ts_tablet_manager.cc:1916] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:30.407284 11204 ts_tablet_manager.cc:1929] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:30.407781 11204 log.cc:1199] T 5e46fa8cb2404116bf566b4ef9bbb2cf P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/5e46fa8cb2404116bf566b4ef9bbb2cf
I20260709 13:16:30.409603  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet 5e46fa8cb2404116bf566b4ef9bbb2cf (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:31.459758 10051 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:31.468043 10199 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:31.468847 10020 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5391e406039d4af8bf2b17f1b2034ef5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.469444 10127 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.470100 11138 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.471262 11208 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.484965 11210 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.488830 10199 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.493263 10131 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.514775  9951 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.528044 10199 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:16:31.544514 10169 consensus_peers.cc:597] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6278932947044bdbb7b0b1e14eea574b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.545869 10051 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.548812 10127 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.549397 11207 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.551482 11145 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.579651 11207 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.581408 10131 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.599246 10049 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.606308 10127 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:31.609277  9954 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:16:31.633085 10095 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5af05c8079540ec84b0cc3dc1a9e051. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.640260 10051 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.646801 10198 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.646631 11206 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:16:31.652207 11206 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.665266 11167 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.672474 10051 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.693493 10127 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:31.700657  9954 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.724061 10198 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.746775 10198 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.748086 10051 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:31.749461 11167 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:31.752915 10095 consensus_peers.cc:597] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98f63a37309c453f9fd7bc3f75659536. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.753330 11224 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:31.763957 11226 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.766484 10198 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.777148 10050 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.791086  9951 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:31.937332 11231 ts_tablet_manager.cc:933] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:31.958925 11233 ts_tablet_manager.cc:933] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:31.970875 11231 tablet_copy_client.cc:323] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:31.973482 11233 tablet_copy_client.cc:323] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:31.975666 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet 6278932947044bdbb7b0b1e14eea574b from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:31.976614 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet 6278932947044bdbb7b0b1e14eea574b from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-6278932947044bdbb7b0b1e14eea574b
I20260709 13:16:31.979890 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 5391e406039d4af8bf2b17f1b2034ef5 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:31.980650 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 5391e406039d4af8bf2b17f1b2034ef5 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-5391e406039d4af8bf2b17f1b2034ef5
I20260709 13:16:31.998168 10065 tablet_copy_source_session.cc:215] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:32.000588 10214 tablet_copy_source_session.cc:215] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:32.002821 11233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5391e406039d4af8bf2b17f1b2034ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.009414 11231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6278932947044bdbb7b0b1e14eea574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.041258 11233 tablet_copy_client.cc:806] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:32.042156 11233 tablet_copy_client.cc:670] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:32.051558 11233 tablet_copy_client.cc:538] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:32.066916 11233 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:32.070871 11231 tablet_copy_client.cc:806] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:32.071902 11231 tablet_copy_client.cc:670] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:32.080641 11231 tablet_copy_client.cc:538] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:32.109830 11231 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:32.111778 11233 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:32.113114 11233 tablet_bootstrap.cc:492] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:32.114102 11233 ts_tablet_manager.cc:1403] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.048s	user 0.030s	sys 0.005s
I20260709 13:16:32.118608 11233 raft_consensus.cc:359] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.120090 11233 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:32.121110 11233 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.132637 11233 ts_tablet_manager.cc:1434] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.011s
I20260709 13:16:32.145284 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-5391e406039d4af8bf2b17f1b2034ef5 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:32.146057 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-5391e406039d4af8bf2b17f1b2034ef5 on tablet 5391e406039d4af8bf2b17f1b2034ef5 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:32.163403 11231 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:32.164741 11231 tablet_bootstrap.cc:492] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:32.165808 11231 ts_tablet_manager.cc:1403] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.056s	user 0.034s	sys 0.014s
I20260709 13:16:32.169672 11231 raft_consensus.cc:359] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.170941 11231 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:32.171941 11231 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.175011 11231 ts_tablet_manager.cc:1434] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:16:32.178162 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-6278932947044bdbb7b0b1e14eea574b received from {username='slave'} at 127.0.0.1:50726
I20260709 13:16:32.178812 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-6278932947044bdbb7b0b1e14eea574b on tablet 6278932947044bdbb7b0b1e14eea574b with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:32.224954 11231 ts_tablet_manager.cc:933] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:32.227430 11231 tablet_copy_client.cc:323] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.66:41681
I20260709 13:16:32.230099 10141 tablet_copy_service.cc:140] P e9a849ac7eda426abfca97cf8b15da0f: Received BeginTabletCopySession request for tablet c5af05c8079540ec84b0cc3dc1a9e051 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:45470)
I20260709 13:16:32.230784 10141 tablet_copy_service.cc:161] P e9a849ac7eda426abfca97cf8b15da0f: Beginning new tablet copy session on tablet c5af05c8079540ec84b0cc3dc1a9e051 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:45470: session id = db052bef7536453fa2d7f5a00e314a18-c5af05c8079540ec84b0cc3dc1a9e051
I20260709 13:16:32.240787 10141 tablet_copy_source_session.cc:215] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:32.245617 11231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5af05c8079540ec84b0cc3dc1a9e051. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.282042 11231 tablet_copy_client.cc:806] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:32.287559 11231 tablet_copy_client.cc:670] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:32.308104 11231 tablet_copy_client.cc:538] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:32.318460 11231 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:32.341841 11233 ts_tablet_manager.cc:933] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:32.342187 11231 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:32.343245 11231 tablet_bootstrap.cc:492] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:32.343896 11233 tablet_copy_client.cc:323] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.66:41681
I20260709 13:16:32.344151 11231 ts_tablet_manager.cc:1403] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.026s	user 0.026s	sys 0.000s
I20260709 13:16:32.346244 10141 tablet_copy_service.cc:140] P e9a849ac7eda426abfca97cf8b15da0f: Received BeginTabletCopySession request for tablet 98f63a37309c453f9fd7bc3f75659536 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:45470)
I20260709 13:16:32.346877 10141 tablet_copy_service.cc:161] P e9a849ac7eda426abfca97cf8b15da0f: Beginning new tablet copy session on tablet 98f63a37309c453f9fd7bc3f75659536 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:45470: session id = db052bef7536453fa2d7f5a00e314a18-98f63a37309c453f9fd7bc3f75659536
I20260709 13:16:32.354779 11231 raft_consensus.cc:359] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.356007 11231 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:32.356590 10141 tablet_copy_source_session.cc:215] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:32.356858 11231 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.360353 11233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f63a37309c453f9fd7bc3f75659536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:32.367687 11231 ts_tablet_manager.cc:1434] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:16:32.378991 10141 tablet_copy_service.cc:342] P e9a849ac7eda426abfca97cf8b15da0f: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-c5af05c8079540ec84b0cc3dc1a9e051 received from {username='slave'} at 127.0.0.1:45470
I20260709 13:16:32.379639 10141 tablet_copy_service.cc:434] P e9a849ac7eda426abfca97cf8b15da0f: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-c5af05c8079540ec84b0cc3dc1a9e051 on tablet c5af05c8079540ec84b0cc3dc1a9e051 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:32.384812 11233 tablet_copy_client.cc:806] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:32.385785 11233 tablet_copy_client.cc:670] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:32.390676 11233 tablet_copy_client.cc:538] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:32.401755 11233 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:32.424348 11233 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:32.425559 11233 tablet_bootstrap.cc:492] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:32.426491 11233 ts_tablet_manager.cc:1403] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.004s
I20260709 13:16:32.430377 11233 raft_consensus.cc:359] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.431730 11233 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:32.432565 11233 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:32.435432 11233 ts_tablet_manager.cc:1434] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:16:32.437868 10141 tablet_copy_service.cc:342] P e9a849ac7eda426abfca97cf8b15da0f: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-98f63a37309c453f9fd7bc3f75659536 received from {username='slave'} at 127.0.0.1:45470
I20260709 13:16:32.438369 10141 tablet_copy_service.cc:434] P e9a849ac7eda426abfca97cf8b15da0f: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-98f63a37309c453f9fd7bc3f75659536 on tablet 98f63a37309c453f9fd7bc3f75659536 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:32.446748 10898 raft_consensus.cc:1217] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:32.512265 10898 raft_consensus.cc:1217] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:32.667536 10898 raft_consensus.cc:1217] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:32.956614 10898 raft_consensus.cc:1217] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:32.964434 11220 raft_consensus.cc:1064] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:32.968122 11220 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:32.981151 10898 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:32.983423 11220 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:32.983978 10131 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:32.983530 10051 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:32.993268 11243 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:32.994369 11220 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.004671 11138 raft_consensus.cc:1064] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:33.010383 11138 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.043570 10198 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.046077 10900 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.047456 11210 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.052250 10131 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.066548 10126 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.042037 11244 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.079743 11138 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.100719 11138 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.122391 11224 raft_consensus.cc:1064] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:33.148223 11224 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.179958 11210 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.182279 10898 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.231318 10900 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:33.235397 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46023.4us
I20260709 13:16:33.236039 10198 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.242156 10051 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.265513 10900 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:16:33.266073 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47965.9us
I20260709 13:16:33.268607 11167 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.268998 10202 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.270344  9951 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.278237 10050 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.290706 11224 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.298367 11226 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:33.296449  9952 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.352100 10126 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.367820 11224 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.384654 10900 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.384686 10051 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.388123 10198 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.412556 10202 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.424520 10898 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.427312 10130 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.433538 10050 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:33.434459 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46013.9us
I20260709 13:16:33.436416 11244 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.450230 11207 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.453114 11255 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.460183  9950 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.465108 10130 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.483178 10898 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.484112  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6278932947044bdbb7b0b1e14eea574b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:33.487406 11242 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.475304 10900 raft_consensus.cc:2955] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.495807 10202 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.502908 11250 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.505939 11138 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.508495 10202 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.510650 10896 raft_consensus.cc:2955] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.540573 10128 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.543327  9953 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 1 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.564823 11224 raft_consensus.cc:1064] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:33.570935 10896 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.570354 11224 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.573041 11256 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.579306 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet 6278932947044bdbb7b0b1e14eea574b with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:33.587971  9952 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.585650 11266 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.592798 10900 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.602074 11271 tablet_replica.cc:333] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:33.603737 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet 5391e406039d4af8bf2b17f1b2034ef5 with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:33.611332 10051 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.631918  9950 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.647090 10900 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.648588 11271 raft_consensus.cc:2243] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:33.649358 11256 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.651064  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5391e406039d4af8bf2b17f1b2034ef5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:33.651286 10202 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.664418 11271 raft_consensus.cc:2272] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:33.668735 11267 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.654250 11272 tablet_replica.cc:333] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:33.674295 11272 raft_consensus.cc:2243] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:33.675246 11271 ts_tablet_manager.cc:1916] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:33.675753  9938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c5af05c8079540ec84b0cc3dc1a9e051 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:33.688072 11272 raft_consensus.cc:2272] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:33.700217 11256 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.706266 11272 ts_tablet_manager.cc:1916] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:33.717231 11271 ts_tablet_manager.cc:1929] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:33.718043 11271 log.cc:1199] T 6278932947044bdbb7b0b1e14eea574b P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/6278932947044bdbb7b0b1e14eea574b
I20260709 13:16:33.737691 10049 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:33.738801 11272 ts_tablet_manager.cc:1929] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:33.739500 11272 log.cc:1199] T 5391e406039d4af8bf2b17f1b2034ef5 P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/5391e406039d4af8bf2b17f1b2034ef5
I20260709 13:16:33.720598  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet 6278932947044bdbb7b0b1e14eea574b (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:33.749028  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet 5391e406039d4af8bf2b17f1b2034ef5 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:33.749648 10202 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.746610 11267 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.755927 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet c5af05c8079540ec84b0cc3dc1a9e051 with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:33.767812 11281 tablet_replica.cc:333] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:33.772886 10050 raft_consensus.cc:2955] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.775753 10900 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.781442 11167 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:33.813199  9954 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.830561 11281 raft_consensus.cc:2243] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:33.831734 11281 raft_consensus.cc:2272] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:33.843247 11281 ts_tablet_manager.cc:1916] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:33.844519 10049 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.863335 10128 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:33.875573 10900 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.879072 11224 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:33.880400 10202 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:33.882186 11224 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:33.884435 11281 ts_tablet_manager.cc:1929] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:33.885051 11281 log.cc:1199] T c5af05c8079540ec84b0cc3dc1a9e051 P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/c5af05c8079540ec84b0cc3dc1a9e051
I20260709 13:16:33.888514 11167 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.902241 10900 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.903798 10198 raft_consensus.cc:2955] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:33.921521  9938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 98f63a37309c453f9fd7bc3f75659536 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:33.904531  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet c5af05c8079540ec84b0cc3dc1a9e051 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:33.935705  9950 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 3 to 4, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 1 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:33.946609  9938 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 98f63a37309c453f9fd7bc3f75659536 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:16:33.964522 10040 tablet_service.cc:1558] Processing DeleteTablet for tablet 98f63a37309c453f9fd7bc3f75659536 with delete_type TABLET_DATA_TOMBSTONED (TS 01bfaf1ad56a4deea886bbc64dec466c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:46784
I20260709 13:16:33.965715 11271 tablet_replica.cc:333] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:16:33.967025 11271 raft_consensus.cc:2243] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:33.967895 11271 raft_consensus.cc:2272] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:33.972395 11271 ts_tablet_manager.cc:1916] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:33.988041 11271 ts_tablet_manager.cc:1929] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:33.988530 11271 log.cc:1199] T 98f63a37309c453f9fd7bc3f75659536 P 01bfaf1ad56a4deea886bbc64dec466c: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/wals/98f63a37309c453f9fd7bc3f75659536
I20260709 13:16:33.990401  9940 catalog_manager.cc:5028] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): tablet 98f63a37309c453f9fd7bc3f75659536 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:35.063411 10049 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.071587 10202 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.071761 10130 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:35.072255 10020 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 82406372d02c4dd59d4ef5c32436e148. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.073480 11268 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.074497 11254 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:35.085927 11268 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.088177 10128 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.088291 10201 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.109040  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.121157 10047 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.142156 10202 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:35.143857 10128 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:16:35.144167 10020 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 75ddb7f5e3204a51b3dafe4903888dcc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.144384 11288 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:35.145666 11287 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:35.184613 11288 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.187330 10202 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.189777 10128 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.208527  9951 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.221509 10047 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.237557 10202 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:16:35.239223 10020 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66968e5dbd3040d290ce9716c146d46a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.239408 10130 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.240561 11287 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:35.241673 11268 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:35.253530 11287 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.256184 10130 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.259642 10202 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.278417 10047 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.287932  9951 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.331552 10130 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.334781 11254 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:35.337620 10020 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a122c89ca68646b8bf88dbb1d5dcbfa8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:35.339448 10202 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:16:35.342506 11268 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:35.350455 11268 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.358275 10202 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.359876 10130 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.380966  9953 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 2, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:35.533186 11302 ts_tablet_manager.cc:933] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:35.537097 11302 tablet_copy_client.cc:323] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:35.539198 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 75ddb7f5e3204a51b3dafe4903888dcc from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:35.539845 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 75ddb7f5e3204a51b3dafe4903888dcc from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-75ddb7f5e3204a51b3dafe4903888dcc
I20260709 13:16:35.549957 10065 tablet_copy_source_session.cc:215] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:35.554003 11302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75ddb7f5e3204a51b3dafe4903888dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:35.574975 11302 tablet_copy_client.cc:806] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:35.576331 11302 tablet_copy_client.cc:670] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:35.588721 11302 tablet_copy_client.cc:538] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:35.616508 11302 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:35.663432 11302 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.664968 11302 tablet_bootstrap.cc:492] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:35.666934 11302 ts_tablet_manager.cc:1403] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.051s	user 0.019s	sys 0.011s
I20260709 13:16:35.676048 11302 raft_consensus.cc:359] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.677000 11308 ts_tablet_manager.cc:933] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:35.677295 11302 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:35.678329 11302 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.685783 11308 tablet_copy_client.cc:323] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:35.689216 11302 ts_tablet_manager.cc:1434] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 13:16:35.690864 10065 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 82406372d02c4dd59d4ef5c32436e148 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:35.691593 10065 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 82406372d02c4dd59d4ef5c32436e148 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-82406372d02c4dd59d4ef5c32436e148
I20260709 13:16:35.699700 10066 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-75ddb7f5e3204a51b3dafe4903888dcc received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:35.700304 10066 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-75ddb7f5e3204a51b3dafe4903888dcc on tablet 75ddb7f5e3204a51b3dafe4903888dcc with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:35.700591 10065 tablet_copy_source_session.cc:215] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:35.705891 11308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82406372d02c4dd59d4ef5c32436e148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:35.726327 11308 tablet_copy_client.cc:806] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:35.727150 11308 tablet_copy_client.cc:670] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:35.735878 11302 ts_tablet_manager.cc:933] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:35.743981 11302 tablet_copy_client.cc:323] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:35.746917 10066 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet 66968e5dbd3040d290ce9716c146d46a from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:35.747612 10066 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet 66968e5dbd3040d290ce9716c146d46a from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-66968e5dbd3040d290ce9716c146d46a
I20260709 13:16:35.761023 11308 tablet_copy_client.cc:538] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:35.761511 10066 tablet_copy_source_session.cc:215] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:35.767212 11302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66968e5dbd3040d290ce9716c146d46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:35.772264 11308 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:35.791313 11302 tablet_copy_client.cc:806] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:35.792219 11302 tablet_copy_client.cc:670] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:35.800077 11302 tablet_copy_client.cc:538] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:35.809677 11308 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.810947 11308 tablet_bootstrap.cc:492] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:35.811828 11308 ts_tablet_manager.cc:1403] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.015s
I20260709 13:16:35.811878 11302 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:35.815994 11308 raft_consensus.cc:359] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.817206 11308 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:35.818202 11308 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.822381 11308 ts_tablet_manager.cc:1434] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:16:35.827412 10066 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-82406372d02c4dd59d4ef5c32436e148 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:35.827932 10066 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-82406372d02c4dd59d4ef5c32436e148 on tablet 82406372d02c4dd59d4ef5c32436e148 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:35.842429 11302 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.843693 11302 tablet_bootstrap.cc:492] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:35.844406 11308 ts_tablet_manager.cc:933] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347)
I20260709 13:16:35.844743 11302 ts_tablet_manager.cc:1403] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.015s
I20260709 13:16:35.848488 11302 raft_consensus.cc:359] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.849354 11308 tablet_copy_client.cc:323] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.65:43347
I20260709 13:16:35.849673 11302 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:35.850629 11302 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.851562 10066 tablet_copy_service.cc:140] P 01bfaf1ad56a4deea886bbc64dec466c: Received BeginTabletCopySession request for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:48538)
I20260709 13:16:35.852207 10066 tablet_copy_service.cc:161] P 01bfaf1ad56a4deea886bbc64dec466c: Beginning new tablet copy session on tablet a122c89ca68646b8bf88dbb1d5dcbfa8 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:48538: session id = db052bef7536453fa2d7f5a00e314a18-a122c89ca68646b8bf88dbb1d5dcbfa8
I20260709 13:16:35.854532 11302 ts_tablet_manager.cc:1434] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:35.862529 10065 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-66968e5dbd3040d290ce9716c146d46a received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:35.864315 10065 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-66968e5dbd3040d290ce9716c146d46a on tablet 66968e5dbd3040d290ce9716c146d46a with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:35.864749 10066 tablet_copy_source_session.cc:215] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:35.874157 11308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a122c89ca68646b8bf88dbb1d5dcbfa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:35.899801 11308 tablet_copy_client.cc:806] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:35.900667 11308 tablet_copy_client.cc:670] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:35.904994 11308 tablet_copy_client.cc:538] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:35.914989 11308 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:35.946892 11308 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.948123 11308 tablet_bootstrap.cc:492] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:16:35.948982 11308 ts_tablet_manager.cc:1403] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.011s
I20260709 13:16:35.952190 11308 raft_consensus.cc:359] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.953332 11308 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:16:35.954200 11308 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:35.957413 11308 ts_tablet_manager.cc:1434] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:16:35.960246 10066 tablet_copy_service.cc:342] P 01bfaf1ad56a4deea886bbc64dec466c: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-a122c89ca68646b8bf88dbb1d5dcbfa8 received from {username='slave'} at 127.0.0.1:48538
I20260709 13:16:35.960762 10066 tablet_copy_service.cc:434] P 01bfaf1ad56a4deea886bbc64dec466c: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-a122c89ca68646b8bf88dbb1d5dcbfa8 on tablet a122c89ca68646b8bf88dbb1d5dcbfa8 with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:36.036688 10896 raft_consensus.cc:1217] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:36.037720 10900 raft_consensus.cc:1217] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:16:36.179776 10898 raft_consensus.cc:1217] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:36.308447 10898 raft_consensus.cc:1217] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:16:36.481230 11242 raft_consensus.cc:1064] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:36.484364 11242 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.492617 10898 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.493088 10201 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.494220 10131 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.495024 11288 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.496411 11287 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:36.503233 11268 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.525177 11268 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.528502 10130 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.539438 10198 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.552775 10896 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.559906  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.641723 10054 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.645203 11268 raft_consensus.cc:1064] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:36.649595 10896 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:36.649860 11268 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.653136 10201 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:36.654735 11242 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:36.656149 11317 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:36.668483 11287 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.670787 10202 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:36.672426 10201 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.672261 10898 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:36.675072 11288 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:36.680219 11288 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:36.687531 10130 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:16:36.690147 11242 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:36.700908 10896 raft_consensus.cc:2955] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.730496  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 82406372d02c4dd59d4ef5c32436e148 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:36.772372 11254 raft_consensus.cc:1064] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:36.776094 11254 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.777552  9954 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.792722 10128 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.810405 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet 82406372d02c4dd59d4ef5c32436e148 with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:36.805913 10202 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.799834 11268 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.815439 10896 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.818746 10898 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.828071 11287 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.829628 11330 tablet_replica.cc:333] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:36.834321 11254 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.835412 11330 raft_consensus.cc:2243] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:36.836447 11330 raft_consensus.cc:2272] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:36.852231 11330 ts_tablet_manager.cc:1916] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:36.854323 11317 raft_consensus.cc:1064] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:16:36.858728 11317 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.863443 11288 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.870337 10199 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.884519  9954 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:36.893345 10900 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 LEARNER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.895426 11268 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.896152 10128 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.898228 10201 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.908829 11268 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.913589 10126 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:16:36.914783 10047 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:36.916851 11317 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.921464 10198 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.923270 10897 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:36.920801 11330 ts_tablet_manager.cc:1929] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:36.927469 11330 log.cc:1199] T 82406372d02c4dd59d4ef5c32436e148 P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/82406372d02c4dd59d4ef5c32436e148
I20260709 13:16:36.929997  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet 82406372d02c4dd59d4ef5c32436e148 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:36.934629 11288 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:36.947822 11254 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.953222 10898 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.953440 10897 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.972836  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 75ddb7f5e3204a51b3dafe4903888dcc with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:36.974444 11254 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:36.981133  9950 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.981665 11268 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.992473 10900 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:36.998004 10128 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:37.003744 10054 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:37.012826 10198 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.014762 10900 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.014673 11287 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:37.018236 11287 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:37.027954 11242 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:37.033253 11288 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.053594 10199 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.055423 10900 raft_consensus.cc:2955] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.056010 10202 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.071084 10130 raft_consensus.cc:2955] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.074378 10126 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.084249  9950 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 2 to 3, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.086073  9950 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 4 to 5, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.092003  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66968e5dbd3040d290ce9716c146d46a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:37.104452  9951 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 3 to 4, VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.114230  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:37.130379 10116 tablet_service.cc:1558] Processing DeleteTablet for tablet 66968e5dbd3040d290ce9716c146d46a with delete_type TABLET_DATA_TOMBSTONED (TS e9a849ac7eda426abfca97cf8b15da0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45958
I20260709 13:16:37.131718 11330 tablet_replica.cc:333] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:16:37.132953 11330 raft_consensus.cc:2243] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:37.133736 11330 pending_rounds.cc:70] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Trying to abort 1 pending ops.
I20260709 13:16:37.134054 11330 pending_rounds.cc:77] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305637874785083392 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "66968e5dbd3040d290ce9716c146d46a" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } } }
I20260709 13:16:37.135676 11330 raft_consensus.cc:2917] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:16:37.136210 11330 raft_consensus.cc:2889] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:16:37.136684 11330 raft_consensus.cc:2272] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:37.141762 11330 ts_tablet_manager.cc:1916] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:37.165387 10054 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:37.176271 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet 75ddb7f5e3204a51b3dafe4903888dcc with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:37.181840 11341 tablet_replica.cc:333] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:37.185389 10897 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.185634 11341 raft_consensus.cc:2243] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:37.186509 11341 raft_consensus.cc:2272] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:37.189507 11287 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:16:37.194273 11330 ts_tablet_manager.cc:1929] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:37.194881 11330 log.cc:1199] T 66968e5dbd3040d290ce9716c146d46a P e9a849ac7eda426abfca97cf8b15da0f: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/wals/66968e5dbd3040d290ce9716c146d46a
I20260709 13:16:37.196148 11341 ts_tablet_manager.cc:1916] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:37.199829 11242 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.208904  9938 catalog_manager.cc:5028] TS e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): tablet 66968e5dbd3040d290ce9716c146d46a (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:37.221065 10897 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:37.225384 11341 ts_tablet_manager.cc:1929] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:16:37.230535 10130 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:16:37.226048 11341 log.cc:1199] T 75ddb7f5e3204a51b3dafe4903888dcc P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/75ddb7f5e3204a51b3dafe4903888dcc
W20260709 13:16:37.880577 10897 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.657s	user 0.010s	sys 0.000s
I20260709 13:16:38.067781  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet 75ddb7f5e3204a51b3dafe4903888dcc (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:38.125330 11304 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.126375 11304 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.130491 11304 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.182147 11345 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.183420 11345 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.191056 11345 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.214579 11304 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.215729 11304 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.219619 11304 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.243074 11342 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.244477 11342 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.248533 11345 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.250044 11345 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.298753 11304 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.301080 11220 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.301851 11220 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.304762 11220 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.299909 11304 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.309218 11304 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.320384 11344 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.321538 11344 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.339308 11345 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.346194 11342 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.347002 11342 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.351999 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:38.354152 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:38.354339 11342 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.360692 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:38.366051 11220 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.367035 11220 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.357062 11304 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.376758 11304 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.375679 11343 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.379597 11343 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.384492 11342 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.385663 11342 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.394799 11343 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.426429 11343 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.427524 11343 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.421759 11345 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.429471 11345 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.400893 11342 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.438282 11342 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.451072 11343 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.459638 11343 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.453131 11351 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.460826 11351 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.464326 11351 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:38.452713 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90060.9us
I20260709 13:16:38.468200 11351 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.469170 11351 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.433920 11345 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.471148 11343 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.476802 11351 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.484504 11348 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.499143 11348 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.498075 11343 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:38.505961 11345 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:38.536391 11358 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.534912 11345 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.563247 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:38.564461 10202 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:38.551632 11351 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.553151 11356 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.607642 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:38.621055 10199 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:38.660346 11365 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.660995 11365 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.663280 11365 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.568886 11360 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.569387 11353 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.652979 11348 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.652288 11343 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.658577 11354 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.556691 11358 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.573136 11351 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.677783 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:38.709311 10204 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:38.671265 11365 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.718967 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:38.678650 11343 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.722644 11343 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.722445 11348 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.722813 11349 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.634575 11356 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.723460 11371 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.725806 11373 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.725797 11348 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.734771 11356 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:38.735693 11356 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.675650 11353 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.726255 11371 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.692134 11354 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.673946 11360 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.726999 11343 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:38.773555 10205 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:38.717411 11365 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.736373 11356 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.774127 11345 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.732056 11349 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.836299 11366 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.837020 11366 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.825040 11345 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:38.830957 11349 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:38.726758 11373 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.781098 11343 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:38.831611 11358 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.184185 11385 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.841547 11379 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.229449 11385 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.842743 11360 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.231257 11360 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.231838 11360 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.840947 11366 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.237782 11375 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.238787 11375 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.241000 11369 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.241762 11369 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.243505 11384 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.244246 11384 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.249428  9940 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:39.251235 11353 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.252269 11353 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.252997 11353 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:38.967451 11361 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.256312 11361 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.228753 11358 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:39.025887 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 132931us
I20260709 13:16:39.260361 11358 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.263530 11345 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.041209 11355 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.272585 11370 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.277281 11348 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.273653 11380 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.229827 11379 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.275580 11372 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.272666 11376 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:38.840291 11371 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.277444 11370 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.230249 11343 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.275236 11387 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.223779 11383 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.292379  9950 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New cstate: current_term: 1 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:39.259624 11361 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:39.306579 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.023s	sys 0.037s
I20260709 13:16:39.283782 11349 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.283052 11366 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:39.327268 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.002s	sys 0.006s
W20260709 13:16:39.328086 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.002s	sys 0.007s
W20260709 13:16:39.329056 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.024s	sys 0.037s
I20260709 13:16:39.289218 11383 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.290539 11372 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.369416 11374 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.280678 11355 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.304425 11343 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.285677 11376 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.371352 11348 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.394426 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.336123 11384 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.386276 11365 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.342047 11367 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.360080 11349 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.372445 11366 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.285657 11380 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.387632 11343 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.426192 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.410270 11384 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:39.427500 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800307fa0 after lost signal to thread 8928
I20260709 13:16:39.427965 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.403200 11373 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.396903 11354 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.426941 11380 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.304652 11387 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.435948 11370 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.436555 11345 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.437479 11389 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.403502 11348 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.453016 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.417095 11365 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.457515 11345 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.459069 11349 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.459947 11376 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.450361 11373 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.399083 11374 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.456218 11384 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.473209 11370 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.444054 11367 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.471073 11345 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.485008 11354 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.476552 11365 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.468919 11394 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.487265 11376 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.497318 11353 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.475422 11373 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.509948 11372 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.480831 11375 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.536085 11369 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.539354 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.542059 11349 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.542586 11362 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.542933 11349 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.510823 11382 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.504475 11370 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.548892 11360 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.539256 11395 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.560688 11395 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.468807 11389 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.583344 11396 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.584455 11396 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.593302 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.593851 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.513716 11354 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.543323 11362 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.607606 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.608060 11383 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.608274 11345 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.608443 10048 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:39.520960 11376 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.619828 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.622313 11392 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.622722 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.624284 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.625252 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.624202 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.606285 11379 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.603086 11369 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.525979 11394 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.598733 11375 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.614398 11343 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.608929 11391 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.640801 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.628371 11366 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.631196 11348 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.651608 11374 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.655428 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:39.623268 11392 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.660539 11392 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.670660 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.673053 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.675771 10199 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:39.676115 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.639694 11369 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.609907 11382 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.681149 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.684535 11365 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.640690 11375 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.661947 11348 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.687312 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:39.678869 11368 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.681849 11384 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.663877 11399 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.649237 11391 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.684794 11398 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.699927 11400 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.702490 11348 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.704085 11365 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:39.700073 11366 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.722244 11370 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.737269 11395 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.744289 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.703397 11387 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.740401 11391 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.745261 10900 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.741263 11400 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.743945 11374 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.730068 11389 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.742416 11346 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:16:39.733229 11365 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:39.741654 11366 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.750407 11376 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.735554 11354 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.750550 11367 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.758342 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.726891 11399 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.776890 10898 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.752923 11395 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.735451 11398 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.766881 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.769430 11382 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.759717 11362 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.722581 11368 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.775740 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:39.780225 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105600us
I20260709 13:16:39.781649 11395 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.780848 10900 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.773774 11351 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:39.787361 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.793009 10898 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.776405 11365 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:39.743721 11402 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:39.790091 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86479.1us
I20260709 13:16:39.793936 10896 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.798264 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.770339 11374 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.800506 10898 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.801241 11399 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.804600 11355 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.787416 11349 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.814070 11349 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.814829 11349 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.816110 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.816934 10130 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.818147 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.818897 10128 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.822202 11399 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.823042 11399 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.824769 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.825452 11355 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.825769 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.826241 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.826167 11355 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.828063 11349 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.828364 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.828753 11349 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.829310 11349 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:39.829560 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.826984 10124 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:39.832429 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.833159 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:39.831022 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.833094 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.835291 11399 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.835975 11399 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.822695 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.837455 11399 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:39.844415 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.842458 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:39.836457 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.837777 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.836056 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.837481 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.838707 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.844451 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.847431 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.848675 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:39.802862 11374 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.812693 11375 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.852428 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.858872 11348 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.833330 11349 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.808034 11394 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.859714 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:39.855423 10131 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.855381 10127 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:39.948890 11401 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:39.966027 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2000 after lost signal to thread 9988
I20260709 13:16:39.955420 11383 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:39.960191 11399 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:39.970989 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.004712 11383 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:39.976703 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.007668 11383 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.007751 11343 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.008365 11343 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.008807 11343 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.006923 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.972503 11368 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:39.968076 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:39.791481 11407 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:39.853837 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:39.983441 10196 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:39.966833 11362 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:40.004735 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 145113us
I20260709 13:16:40.009027 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.024012 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:40.024583 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:40.027547 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 173838us
I20260709 13:16:40.028753 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.031766 11400 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.032225 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.032794 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.033862 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.991994 11375 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.059181 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.056244 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:39.843223 11402 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.063745 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.065925 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.056823 11351 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.057091 11389 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.070613 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.063575 11362 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.062523 11407 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.075418 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.058832 11366 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.113319 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:40.112099 11351 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.120820 11351 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.067519 11375 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.025692 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:40.070592 11376 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.123765 11376 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.067749 11395 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.123971 11391 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.124369 11376 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.124625 11391 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.034958 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:40.125195 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 99562.5us
W20260709 13:16:40.112788 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 316270us
I20260709 13:16:40.126039 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:40.127497 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:40.128728 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.129653 10196 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:40.130363 11375 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.130937 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.111682 11389 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:40.142194 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 302994us
W20260709 13:16:40.142809 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 181030us
W20260709 13:16:40.143573 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 175425us
I20260709 13:16:40.126768 11391 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.145329 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:40.024564 11401 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:40.250278 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 122849us
W20260709 13:16:40.211102 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65864.5us
W20260709 13:16:40.150643 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.840s	user 0.007s	sys 0.040s
W20260709 13:16:40.267589 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.957s	user 0.008s	sys 0.051s
I20260709 13:16:40.226166 10129 raft_consensus.cc:2955] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:40.208693 11366 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.268462 11366 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.268661 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:40.264358 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 119161us
W20260709 13:16:40.149716 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1069) took 1494 ms (client timeout 1575 ms). Trace:
I20260709 13:16:40.145927 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1068) took 1569 ms. Trace:
I20260709 13:16:40.149501 11407 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.133554 11354 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.140745 11304 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.145195 11389 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.269091 11366 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.219599 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.274408 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:40.255626 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:40.209333 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64832.7us
I20260709 13:16:40.239147 11398 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.264997 11414 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.279772 11368 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:40.318475 10168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.934s (SENT)
I20260709 13:16:40.324231 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.272884 10202 rpcz_store.cc:276] 0709 13:16:38.608704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.655391 (+ 46687us) service_pool.cc:224] Handling call
0709 13:16:38.655951 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.733521 (+ 77570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:38.733571 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:38.733634 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:38.733681 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.142440 (+1408759us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:40.142490 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.142633 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177588608}
I20260709 13:16:40.326200 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.279980 11354 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:40.309617 11365 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.007s
I20260709 13:16:40.329154 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:40.329656 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.330518 10196 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:16:40.272277 10205 rpcz_store.cc:269] 0709 13:16:38.655642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.657229 (+  1587us) service_pool.cc:224] Handling call
0709 13:16:40.149460 (+1492231us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b540012aa54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:40.149597 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123273600}
I20260709 13:16:40.310155 11407 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:40.281646 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.334216 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.337121 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.337739 11402 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.319583 11362 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.334883 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.321053 11368 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.356317 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.358300 10202 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:40.359479 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.356521 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.334564 11365 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.338405 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.336064 11407 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.332808 11354 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.321535 11414 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.372267 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.372599 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.372670 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.375699 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.370143 10126 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:40.367638 11407 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:40.339373 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.338651 11349 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.345187 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.344574 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.382645 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.373606 11368 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.391405 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.394063 10205 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:40.395401 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.396395 10205 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:40.397976 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.399854 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.399911 10127 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
I20260709 13:16:40.386914 10897 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:40.378937 11414 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.404237 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.384500 11343 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.408795 11343 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.409755 11351 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.363065 11391 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.412125 11351 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.418782 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.421386 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.416217 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.417735 11391 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.435304 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.437470 11344 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.438938 10131 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
I20260709 13:16:40.439182 11401 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.440234 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.442366 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.419502 11376 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.463845 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.465528 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.467677 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.468444 11410 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.470757 11376 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.473493 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.474134 11407 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.475497 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.476641 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.478230 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.456511 10053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:40.493335 10053 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:40.476712 11406 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.497285 10196 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:40.515944 11406 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.516628 11406 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.521315 11350 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:40.484830 11383 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.509698 11410 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.522083 11350 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.550879 11410 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.551837 11365 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.550762 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.553932 11354 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.557441 11356 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.544036 11389 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.618844 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.621088 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:40.621855 10900 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.624884 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.633205 10205 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:40.564952 11368 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:40.636387 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 189027us
I20260709 13:16:40.637390 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.638121 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:40.642817 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:40.629647 11370 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.598905 11366 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.631228 11391 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.651065 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.646785 10900 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:40.648067 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:40.646705 11392 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:40.663524 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 349815us
I20260709 13:16:40.644738 11354 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.662690 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:40.667480 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.648978 11365 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.674147 11366 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.670392 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:40.675894 11376 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:40.676515 11376 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:40.660487 11391 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.677608 11376 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.678941 11391 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:40.679455 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002bbf80 after lost signal to thread 10080
I20260709 13:16:40.684263 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:40.724120 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:40.725606 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:40.726378 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002d8f60 after lost signal to thread 10155
I20260709 13:16:40.726852 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:40.728152 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1340 after lost signal to thread 10229
I20260709 13:16:40.727722 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:40.730786 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 412317us
I20260709 13:16:40.732955 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:40.733664 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:40.736359 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.649356 11370 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.738565 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.682521 11407 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.741801 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:40.739887 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 186376us
I20260709 13:16:40.723162 11392 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.730857 11398 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.738071 11346 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:40.734175 11395 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.737635 11365 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.741658 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.735994 11366 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.756330 10199 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:40.748586 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.738368 11383 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.740777 11349 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.773171 11419 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.773806 11419 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.760412 11365 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.774232 11419 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.753962 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.745720 11391 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.778258 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.765316 10197 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:40.766243 11414 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:40.767086 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.768013 11368 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.749511 11350 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.768587 10893 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:40.768842 11389 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.757287 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.774479 11365 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.780014 10204 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:40.780659 10202 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:40.781883 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:40.808416 11414 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:40.782860 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:40.755287 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:40.791219 11371 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.778081 11391 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.792529 11413 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.793870 11370 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.748378 11398 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.745625 11416 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:40.795840 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.796257 11385 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.796713 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.797048 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.798213 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.798693 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:40.835881 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53215us
I20260709 13:16:40.837005 10049 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:40.800382 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.800745 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.838380 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.799220 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.802428 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.847963 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.804857 11358 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.776378 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:40.810010 10050 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:40.848376 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.829129 11373 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.852573 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.853092 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.854418 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:40.855013 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.827549 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.828382 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:40.829053 11371 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.839205 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:40.841681 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.825076 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:40.850328 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74028us
I20260709 13:16:40.836200 11398 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.852061 11357 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.855274 11407 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:40.826519 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.833024 11391 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.860760 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:40.847656 11385 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.861514 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:40.874240 11385 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:40.875051 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.855875 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.863778 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.877859 11385 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.879060 11367 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:40.879818 11367 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:40.880344 11367 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.838399 11413 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:40.835517 11416 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:40.860895 11371 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.869096 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.839586 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:40.886709 11357 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:40.888675 11220 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.889312 11396 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:40.892586 11413 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.900228 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:40.906544 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 710) took 1111 ms. Trace:
I20260709 13:16:40.907341 10126 rpcz_store.cc:276] 0709 13:16:39.785996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.786191 (+   195us) service_pool.cc:224] Handling call
0709 13:16:39.786657 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.379582 (+592925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.896795 (+517213us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b540014a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:40.897095 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341368704}
I20260709 13:16:40.881615 11355 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.903179 11357 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:40.923197 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2183) took 1293 ms. Trace:
W20260709 13:16:40.924305 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003070a0 after lost signal to thread 10924
I20260709 13:16:40.930204 11369 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:40.892423 11416 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:40.936705 10226 debug-util.cc:398] Leaking SignalData structure 0x7b080030e420 after lost signal to thread 10925
I20260709 13:16:40.954003 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:40.883140 11406 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.006131 11391 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.011884 11406 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:41.014477 11413 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.019356 11419 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.052754 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:40.837419 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:41.059967 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.060155 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:41.068795 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 230573us
W20260709 13:16:41.071511 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 185790us
I20260709 13:16:41.074863 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:41.074965 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.101523 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.095983 10048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:41.107229 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.109683 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.110574 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.111735 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.112064 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.112993 10125 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:41.091334 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2158) took 1270 ms. Trace:
I20260709 13:16:41.114552 10123 rpcz_store.cc:276] 0709 13:16:39.820230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.821289 (+  1059us) service_pool.cc:224] Handling call
0709 13:16:39.821639 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.146527 (+324888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:40.206902 (+ 60375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.209420 (+  2518us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:40.219248 (+  9828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:41.090268 (+871020us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54001c7d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:41.090427 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.090916 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738489088}
I20260709 13:16:41.116920 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.117563 10048 raft_consensus.cc:606] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:41.120572 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.120869 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.123433 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.086702 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.124579 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.125331 10200 rpcz_store.cc:276] 0709 13:16:39.629648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.632247 (+  2599us) service_pool.cc:224] Handling call
0709 13:16:40.440380 (+808133us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:40.921708 (+481328us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:40.921993 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.922324 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:40.922373 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.922566 (+   193us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:40.922630 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.922731 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.922929 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57729280}
I20260709 13:16:41.130064 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.106783 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.132987 10203 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.134665 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2182) took 1505 ms. Trace:
I20260709 13:16:41.137044 10199 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.139801 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.140554 10205 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.140988 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.141929 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.142686 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.143394 10048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:41.144481 10048 raft_consensus.cc:606] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.135483 10198 rpcz_store.cc:276] 0709 13:16:39.629249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.632046 (+  2797us) service_pool.cc:224] Handling call
0709 13:16:40.149973 (+517927us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.150193 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:40.150220 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:40.150263 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:40.150293 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.150348 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:41.134092 (+983744us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540012aa50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:41.134405 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112632192}
I20260709 13:16:41.143674 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:41.132532 10170 connection.cc:256] RPC call timeout handler was delayed by -1.0447081960001015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:16:41.148419 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:41.150298 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.150564 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.151595 10053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:41.152400 10053 raft_consensus.cc:606] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:41.152730 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.153941 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.160956 10200 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:41.155079 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.155222 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.161150 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.155586 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.154237 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.156353 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.156838 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.164508 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.168179 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.169222 10052 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:41.166749 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.160807 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.151647 10197 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:41.172230 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.157851 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.167236 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.166654 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.175702 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.175881 10050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d07e888fde29492b84e575218308b6ff"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:41.178654 10048 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:41.178736 10050 raft_consensus.cc:606] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:41.164601 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.182610  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:40.965868 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:41.204162 10891 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.215870 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.217718 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.232456 11413 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.243538 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.258040 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.266726 11355 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.267547 11355 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:41.268563 11355 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.270828  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:41.271265 11346 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.277631 11370 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.281967 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.284574 11386 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.285622 11386 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.289762 11397 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.291389 11397 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:41.337455 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.296758 11364 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:41.339607 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 154356us
W20260709 13:16:41.337320 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 126145us
I20260709 13:16:41.304834 11419 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:41.286795 11386 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:41.348758 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 108701us
I20260709 13:16:41.363860 11404 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.371585 10126 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:41.364964 11404 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.380580 11396 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.382148 11396 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.382915 11404 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:41.384859 11367 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.009s
I20260709 13:16:41.385614 11367 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.387758 11409 raft_consensus.cc:993] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:41.390189 11403 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.390823 11403 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.392546 11403 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.394763 11403 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.404686 10197 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:41.409282 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:41.376240 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:41.414981 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46010.1us
I20260709 13:16:41.417130 11398 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.419148 10131 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:41.420574 11391 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:41.421700 10130 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.422276 11409 raft_consensus.cc:1081] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer d51ad94710c348889d417f7f00032948 to start an election
W20260709 13:16:41.373842 11421 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.080s	sys 0.106s
I20260709 13:16:41.398890 11396 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.430157 11384 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:41.431346 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51349.2us
I20260709 13:16:41.439356 11358 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.440258 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.440670 11386 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.430297 11354 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.441828 11354 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.442418 11414 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.442684 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:41.430228 11365 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.009s
W20260709 13:16:41.406323 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 200570us
I20260709 13:16:41.443725 11404 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:41.452559 11365 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.454789 10900 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:41.460494 11398 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.460126 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.460080 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.461974 11358 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.462065 11220 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.463215 11220 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.463727 11220 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.467540 11358 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:41.462944 10168 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.472067 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.474171 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.476131 11428 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.476686 11428 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.477350 11358 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.478065 11428 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.481617 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.482028 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:41.483489 11419 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:41.484519 11358 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.457624 11381 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 13:16:41.487385 11416 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.007s
I20260709 13:16:41.515112 11416 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.515846  9953 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 208) took 2264 ms. Trace:
I20260709 13:16:41.490552 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.474195 10891 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:41.524655 11354 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:41.493480 11365 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.498484 11355 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.499954 11367 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:41.500908 11371 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.592s	user 0.001s	sys 0.018s
I20260709 13:16:41.505618 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:41.506477 11402 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.508699  9950 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 101) took 3454 ms. Trace:
I20260709 13:16:41.513671 11381 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.547010 11381 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:41.547247 11371 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.548697 11381 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.517462 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.490038 11396 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.492323 11431 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.520263 11416 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.521262 11419 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:41.518679  9953 rpcz_store.cc:276] 0709 13:16:39.251263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.251478 (+   215us) service_pool.cc:224] Handling call
0709 13:16:41.515583 (+2264105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55829,"reported_tablets":1}
I20260709 13:16:41.519448 11414 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.522400 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.550297 11371 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.555995 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:41.529728 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:41.531010 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49592.2us
I20260709 13:16:41.530840 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:41.553741 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48280.9us
I20260709 13:16:41.553936 11398 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:41.559419 11398 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:41.560104 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.560792 11398 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:41.561335 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87396.7us
I20260709 13:16:41.562796 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.523425 11391 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.555209 11381 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.564639 10197 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.551519 11432 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:41.526096 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.048s	sys 0.127s
I20260709 13:16:41.561318 11370 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:41.571419 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.049s	sys 0.127s
I20260709 13:16:41.546870  9950 rpcz_store.cc:276] 0709 13:16:38.054405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.294477 (+240072us) service_pool.cc:224] Handling call
0709 13:16:41.508417 (+3213940us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:39.489603 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:39.648187 (+158584us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:39.652834 (+  4647us) write_op.cc:435] Acquired schema lock
0709 13:16:39.656440 (+  3606us) tablet.cc:662] Decoding operations
0709 13:16:39.721609 (+ 65169us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:39.725296 (+  3687us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:39.726531 (+  1235us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:39.751719 (+ 25188us) tablet.cc:701] Row locks acquired
0709 13:16:39.753931 (+  2212us) write_op.cc:260] PREPARE: finished
0709 13:16:39.761197 (+  7266us) write_op.cc:270] Start()
0709 13:16:39.764789 (+  3592us) write_op.cc:276] Timestamp: P: 1783602999761099 usec, L: 0
0709 13:16:39.766289 (+  1500us) op_driver.cc:348] REPLICATION: starting
0709 13:16:39.814080 (+ 47791us) log.cc:844] Serialized 532 byte log entry
0709 13:16:40.747414 (+933334us) op_driver.cc:464] REPLICATION: finished
0709 13:16:41.392097 (+644683us) write_op.cc:301] APPLY: starting
0709 13:16:41.392379 (+   282us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:41.392604 (+   225us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:41.392797 (+   193us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:41.393279 (+   482us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:41.393665 (+   386us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:41.393751 (+    86us) write_op.cc:312] APPLY: finished
0709 13:16:41.394330 (+   579us) log.cc:844] Serialized 22 byte log entry
0709 13:16:41.425118 (+ 30788us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:41.437231 (+ 12113us) write_op.cc:454] Released schema lock
0709 13:16:41.447001 (+  9770us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:41.479783 (+ 32782us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:41.479829 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":121491,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":619088,"apply.run_cpu_time_us":4437,"apply.run_wall_time_us":87785,"num_ops":1,"prepare.queue_time_us":131468,"prepare.run_cpu_time_us":66571,"prepare.run_wall_time_us":589267,"raft.queue_time_us":411219,"raft.run_cpu_time_us":209071,"raft.run_wall_time_us":755923,"replication_time_us":963459,"thread_start_us":1286731,"threads_started":4,"wal-append.queue_time_us":319808}]]}
I20260709 13:16:41.573748 11428 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.512326 11220 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.562949 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:41.496250 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45029.9us
I20260709 13:16:41.569621 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.550338 11355 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.553164 11419 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:41.576910 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.583666 10189 tablet_service.cc:1558] Processing DeleteTablet for tablet a122c89ca68646b8bf88dbb1d5dcbfa8 with delete_type TABLET_DATA_TOMBSTONED (TS d51ad94710c348889d417f7f00032948 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57098
I20260709 13:16:41.591552 11419 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.592502 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.593587 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.594172 11433 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:41.595250 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.570684 11414 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.580912 11358 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.597738 11358 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:41.598261 11358 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.581190 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.581457 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.582299 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.582921 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.583734 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.584491 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.584733 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.585399 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.605949 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.606163 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.607576 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.607982 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.591105 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.586148 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.589629 11391 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.609898 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.599457 11371 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:16:41.611848 11371 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.612393 11371 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:41.614028 11371 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.593103 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:41.598613 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:41.579521 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.578424 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.603296 11220 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:41.607465 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.610270 11391 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.582183 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:41.618551 11391 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.593690 11419 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:41.621205 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.586979 11384 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.622356 11384 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.624926 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:41.595686 11357 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.647s	user 0.016s	sys 0.000s
I20260709 13:16:41.607349 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:41.617795 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55176.5us
I20260709 13:16:41.632619 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.634021 11414 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:41.634786 11414 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:41.635304 11351 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.636022 11414 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.636997 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.638559 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.639506 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.641180 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.641340 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.641812 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.642421 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.639386 11357 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.644948 10055 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.644737  9950 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:16:41.634826 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56496.4us
I20260709 13:16:41.629355 11404 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.650367 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.625349 11384 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.661520 10127 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.662374 10129 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.663785 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:41.663939 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.661551 10130 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.676887 11436 tablet_replica.cc:333] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:16:41.677287 10129 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.678571 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.679417 11433 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.679589 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.679836 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.659626 11404 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.690145 11404 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.685186 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:41.672585 11371 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.729308 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.729990 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:41.737715 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 78294.7us
W20260709 13:16:41.736342 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 143363us
I20260709 13:16:41.693154 11433 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:41.739427 11384 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.746001 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:41.747658 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53045.4us
I20260709 13:16:41.751673 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.752640 11369 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:41.753312 11386 raft_consensus.cc:993] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:41.753319 11416 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:41.742597 11433 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.747895 11384 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.758669 11369 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.750656 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.767488 11430 raft_consensus.cc:1081] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:41.767710 11357 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:41.768589 11432 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.761483 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.754776 11409 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:41.751420 11398 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.769447 11416 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.764358 11369 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.764338 11384 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.774582 11409 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.775185 11409 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.775688 11416 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.776459 11409 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.777118 11416 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.778189 11409 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.778625 11416 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.783281 11362 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:41.784699 11352 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.785164 11352 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.788568 11369 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:41.789481 11352 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.790707 11369 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.791169 11352 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:41.792837 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.798444 10203 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.799805 10198 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.805411 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.799898 11389 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.807250 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:41.807696 10900 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.808919 11407 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.808724 11391 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.810279 10202 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.809899 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:41.810578 11430 raft_consensus.cc:993] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:41.813283 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.815575 11430 raft_consensus.cc:1081] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:41.816987 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:41.821983 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.824385 11362 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.769732 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:41.833535 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72135.3us
I20260709 13:16:41.826938 11407 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:16:41.819789 11398 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.851616 10198 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:41.838894 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.855755 10202 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.857352 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.858894 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.859284 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.859562 11436 raft_consensus.cc:2243] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:41.858781 11432 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.860463 11432 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:41.860941 11432 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.847728 11381 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.861872 11381 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:41.863130 11381 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.866843 10202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0"
dest_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:57108
I20260709 13:16:41.867843 10202 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:41.868367 10202 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.871886 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.873747 11436 raft_consensus.cc:2272] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:41.879321 11381 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:41.886529 11403 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:41.890205 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 120853us
I20260709 13:16:41.891405 11391 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:41.892501 11436 ts_tablet_manager.cc:1916] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:41.892836 11404 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.894956 11414 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:41.901396 11414 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:41.902619 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.902820 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.903532 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.902909 11414 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:41.903930 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53967.6us
W20260709 13:16:41.906721 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74264.5us
I20260709 13:16:41.878870 11384 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.916148 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.917892 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:41.886161 11416 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:16:41.920974 11419 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.927196 11357 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:16:41.919395 11362 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:16:41.929924 11430 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:41.938546 11358 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:41.945325 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:41.945245 11428 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:41.948128 10203 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:41.979730 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:41.943641 11430 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.984588 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:41.947825 11428 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:41.995389 11220 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:41.997038 11442 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:41.996740 10202 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.003865 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:41.986641 11407 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:42.007467 11428 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.008958 11428 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.011941 11386 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.012431 11386 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.013471 11386 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.027241 11404 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:42.028144 11404 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.005270 10197 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.035972 11428 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.036507 11428 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.037568 11428 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.042835 11358 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.047787 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:41.947288 11433 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:42.057451 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 127655us
W20260709 13:16:42.060348 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 80746us
I20260709 13:16:42.060889 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:41.994570 11432 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.074374 11381 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:16:42.076565 10894 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:42.076793 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.077494 10202 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:42.078433 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94776.4us
W20260709 13:16:42.079286 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94859.7us
I20260709 13:16:42.080650 11433 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.080708 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.065497 11371 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.063900 11384 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:42.070358 11386 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.087258 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.093868 10900 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.082263 10200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d07e888fde29492b84e575218308b6ff"
dest_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:57108
I20260709 13:16:42.085035 11381 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.097069 11365 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.092352 11391 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.067035 11407 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.103531 10200 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:42.104075 10200 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.105360 11407 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:42.101986 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.107141 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.098212 11432 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.109856 10122 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:42.110175 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:42.110741 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.100085 11443 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.111647 11416 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.112867 11444 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.113260 11444 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.113551 11416 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.114042 11416 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.114289 11444 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.116321 11444 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.118654 11352 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.119185 11352 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.122447 11352 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.125367 11352 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.101188 11381 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.127890 11381 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.128437 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.129056 11381 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.129343 11358 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.130009 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.130995 11402 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.131302 11409 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.131812 11402 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.132540 11402 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.132671 11409 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.133286 11409 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.133451 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.134663 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.135251 11391 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.135700 11407 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.135083 11409 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.139364 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:42.140125 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.140540 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:42.141247 10202 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.088436 11430 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.115103 11416 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.145859 10198 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:42.146083 11430 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.142347 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.164307 11416 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.164587 11432 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.165141 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.166165 11432 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.153420 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.168566 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.163597 10200 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.171343 11428 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.160848 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.162938 10891 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:42.142217 11414 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.148499 11409 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.175864 10122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45970
I20260709 13:16:42.176345 11414 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.176729 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.177207 10122 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:42.177358 11220 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.177822 10122 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.177773 11441 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.178519 11419 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.177814 10201 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:42.180836 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.177474 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.183856 11407 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:42.184891 11406 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.186184 11406 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.186641 11406 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.153935 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.186750 11220 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.187616 11407 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.188414 11406 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.183871 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.190353 11406 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.190887 11220 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.193938 11436 ts_tablet_manager.cc:1929] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:16:42.201833 11436 log.cc:1199] T a122c89ca68646b8bf88dbb1d5dcbfa8 P d51ad94710c348889d417f7f00032948: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/wals/a122c89ca68646b8bf88dbb1d5dcbfa8
I20260709 13:16:42.195765 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.203845 11220 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.205302 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.195098 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.199945 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.209872 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.205417  9939 catalog_manager.cc:5028] TS d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): tablet a122c89ca68646b8bf88dbb1d5dcbfa8 (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:16:42.206416 10200 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.207727 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.208348 10124 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.211124 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.214116 11444 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.221745 10123 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:42.214972 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.222536 11444 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.222711 10126 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:42.224958  9950 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:16:42.226434 11444 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.219405 11381 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.227963 11428 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.228416 11428 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.229423 11428 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.220288 11220 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:42.215847 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.226285 10122 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.220553 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.231729 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.232342 10125 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 3. Ops: []
I20260709 13:16:42.233441 11428 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.233743 10122 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:42.234216 10128 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:42.237473 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.238633 11407 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.239362 11444 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.239564 11407 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.240898 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:42.241665 11416 raft_consensus.cc:993] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:42.242012 10200 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.242267 11416 raft_consensus.cc:1081] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:42.219641 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.244988 11419 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.246410 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.247560 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.251412 11416 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.252416 11416 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:42.253871 11444 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.254601 11444 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.256091 11444 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.251281 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.265071 11407 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.266160 11444 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:42.266384 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.262293 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.273897 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.276288  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:42.278780 11352 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.281433 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.282552 11220 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.283282 11220 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.284677 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.285657 11414 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.287577 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.289199 11407 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.290860 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.294296 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.297200 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.298676 11430 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.300076 11430 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.300601 11430 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.300747 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.301746 11430 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.302453 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.302850 11365 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:42.305357 11432 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:42.306896 10203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47"
dest_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:57108
I20260709 13:16:42.310286 11220 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.312810 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.319595 10047 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:42.324836 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.341755 10203 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:42.342278 10203 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.310457 10200 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:42.343772 11406 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.349279 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:42.357437 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44714.3us
I20260709 13:16:42.357872 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.346788 11428 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.347720 11414 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:42.364712 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:42.368726 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.349617 11419 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.360325 11430 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.370143 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:42.378516 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.380425 11381 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.383447 11433 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.384215 10197 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 3. Ops: []
I20260709 13:16:42.363921 11402 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.368338 11414 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:42.387681 11414 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.346297 11432 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.394127 11402 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:42.394906 11402 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.396108 11402 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.403645 11220 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.417164 11441 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.418537 11409 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.419134 11409 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.420706 11409 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.422540 11398 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:42.427156 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54837.6us
W20260709 13:16:42.430246 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51852.2us
W20260709 13:16:42.431447 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68331.8us
I20260709 13:16:42.432932 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.431700 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.434384 11409 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:42.436652 10203 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.438484 11365 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.440002 11432 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:42.440287 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:42.440346 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.443145 10047 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:42.440069 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:42.446292 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:42.449064 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:42.448990 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.447467 11352 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.450476 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:42.450830 11352 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.451495 11432 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:42.451959 11432 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:42.452404 11432 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.452123 11352 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.451525 11220 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.454509 11398 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.440025 11441 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:16:42.460265 11386 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.469285 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.470283 11220 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.460907 11386 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.472659 11386 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.474514 11386 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.468227 11441 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.475512 11365 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.475723 11441 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.476845 11441 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.480979 10198 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:42.480585 11365 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.470711 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.482015 11398 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.482431 11398 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.482919 11220 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:42.483400 11220 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:42.484109 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.483819 11398 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.483830 11220 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.485383 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.487473 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.492703 10203 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.494545 11398 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.495667 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:42.496497 10051 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:42.499912 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.500746 10122 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.507531 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.509245 10050 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:42.509457 11432 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.511188 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.511296 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.520808 10048 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.532613 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:42.528936 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.548534 11220 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.528733 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:42.553179 11220 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.555884 10055 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:42.557336 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.564440 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.565661 10052 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.571410 10053 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.575454 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.583734 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.588107 10053 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.588503 10053 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:42.589988 10053 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.592227 10053 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.591027 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:42.596827 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68185.2us
I20260709 13:16:42.597314 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.598069 11371 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.528445 11433 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:42.583693 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.608827 11220 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:42.609362 10049 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.609406 11220 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:42.610983 11220 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.611488 11452 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:42.612557 11452 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.602886  9950 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 2 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.615412 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.617653 10047 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.619621 10053 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.626964 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:42.575065 10122 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.627444 10054 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:42.631104 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.640661 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:42.623572 11419 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.642227 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:42.632138 11406 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.666819 11220 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.676895 10052 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:42.678658 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:42.679757 11389 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:42.625835 11391 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:16:42.682430 11391 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.683053 11391 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.687747 11220 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.641405 11452 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:42.641674 11402 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.635897 11358 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:16:42.700438 11433 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.703266 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:42.710817 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72662.9us
I20260709 13:16:42.705489 11452 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:42.716571 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81282.9us
I20260709 13:16:42.724027 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.732311 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.729892 11457 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.735033 11458 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.705498 11391 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.752739 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.723438 11414 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:42.723645 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45114.8us
I20260709 13:16:42.772321 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.775983 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.782021 10053 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:42.770471  9950 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 2 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.791474 10203 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.792551 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.795336 11358 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.795676 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:42.796402 10126 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
W20260709 13:16:42.776785 11455 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:16:42.799294 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.799865 11414 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:42.802871 10198 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:42.797183 11456 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.806346 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:42.806419 10200 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:42.774448  9953 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 3, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.811851  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:42.812851 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.797441 11416 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.802706 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:42.821080 10054 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:42.825420 11416 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.825078 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.814822 11452 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:42.829792 11414 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.829952 11416 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:42.810724 11365 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:16:42.830871 11452 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.820689 11456 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.836938 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:42.842832 11381 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.846570 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:42.847332 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.847807 11414 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.848506 11414 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.848601 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.849067 11414 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.849670 11457 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.850299 11457 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.851517 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:42.851449 11457 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.852538 10200 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:42.853088 11402 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:42.853744 11398 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:16:42.854363 11402 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.854668 11398 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:42.854848 11402 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.855248 11398 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.855937  9951 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.856287 11398 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.868222 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.882107 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:42.852167 11371 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.834744 11432 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:42.889449 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62901.8us
I20260709 13:16:42.890206 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.890992 10054 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.899156 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:42.846565 11381 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.825337 11358 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.887423 11398 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:42.884827 11430 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:42.894034 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:42.850135 11414 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.836984 11456 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.924250 10054 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:42.887929 11371 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:42.914193 11381 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:42.856314 11402 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.895057 11419 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:42.936741 11220 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:42.965664 11220 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:42.941071 11358 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.944236 11369 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:42.949477 10128 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:42.930807 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:42.967552 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:42.973547 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 123246us
I20260709 13:16:42.937685 11406 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:42.992959 11362 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.993768 11362 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.994501 11220 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:42.996115 11220 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:42.993224 10052 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:16:42.986320 11419 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:43.001835 11452 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:43.016835 11391 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:43.023065 11402 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:43.020258 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87166.3us
I20260709 13:16:43.014739 11369 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.029218 11452 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:43.017769 11471 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:16:43.068325 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92538.5us
W20260709 13:16:43.050450 11461 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:16:43.057364 11452 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:43.105895 11464 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:16:43.102067 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:43.118675 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52351.3us
W20260709 13:16:43.119637 11474 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:43.120056 11460 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:43.120589 11432 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:43.121296 11432 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.128518 11362 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.135999 11371 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.137799 11371 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.147687 11381 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:43.109469 11419 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.151137 11432 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.167086 11381 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.168107 11381 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:43.170321 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 147279us
I20260709 13:16:43.171681 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.174926 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:43.176324 10894 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:43.180830 11459 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.181569 11459 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.187014 11459 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.202677 10048 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:43.194263 11419 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:43.230855 11389 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:43.204082 11369 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.232651 11462 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:43.196601 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.240077 10128 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:43.202924 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.233135 11462 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:43.243407 11407 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.233553  9951 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.243731 10122 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.243871 11407 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.244560 11407 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.244196 11462 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.199031 11398 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.245635 11462 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.245456 11398 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.247556 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.248000 11398 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.248643 11432 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:43.249317 11432 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:43.249740 11432 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.249752 11419 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:43.212064 11381 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.264629 10048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:43.279826 10048 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:43.281111  9954 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 3, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:43.287480 10052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:43.288344 10052 raft_consensus.cc:606] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Received request to transfer leadership to TS d51ad94710c348889d417f7f00032948
I20260709 13:16:43.288748 10052 raft_consensus.cc:614] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:43.292533 11419 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.276362 11451 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.295454 11451 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.297900 10052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:43.299077 11369 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.299661 11369 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.300454 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:43.301900 11369 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.303090 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.303578 11369 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.304282 11369 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.298719 10052 raft_consensus.cc:606] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.307075 10052 raft_consensus.cc:614] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:43.307806 11460 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.308645 11460 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.259521 11452 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.313975 11452 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.277837 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.315542 10200 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.318171 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:43.324292 10892 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:43.325785 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:43.317920 11369 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:43.326730 11369 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.320371 10052 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.328442 11369 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.328995 11369 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.333892 10122 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:43.336068 11475 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.347429 11402 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.348291 11402 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.298115 11451 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:43.360746 10049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:43.366945 10049 raft_consensus.cc:606] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.367761 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:43.370355 10201 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.380911 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.383034 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.384161 10203 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:43.354192 11480 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:43.409338 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 301525us
I20260709 13:16:43.358444 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.411855 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.412707 10201 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:43.355988 11402 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.415570 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.422591 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.423527 10201 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:43.357688 11460 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.427937 10203 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:43.424343 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.435763 10199 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.440086 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:43.423703 10131 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:43.346304 11432 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.352298 11407 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.451267 10127 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:43.456928 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.458137 10201 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:43.460426 10894 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:43.462703 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.463279 10131 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.463613 10122 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:43.464615 11479 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.468217 11479 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.470824 10051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:43.471638 10051 raft_consensus.cc:606] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.475345  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:43.475526 10123 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:43.414392 11480 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.478035  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:43.479918 11432 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.478649 10202 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:43.471022 10054 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.484434 10049 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.484470 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.486091 10053 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:43.442901 11416 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.488049 10051 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:43.488082 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.491907 10053 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:43.492885 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.493371 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:43.493785 10056 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.494661 10050 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.496565 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.497730 10055 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.496764 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.499534 10202 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.500633 11358 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.501045 11482 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.502276 11451 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.449122 11452 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:16:43.506286 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:43.506860 10055 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:43.519066 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.520757 11414 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:43.521373 11414 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.521868 11414 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:43.510095 11416 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.523816 11441 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:43.524466 11441 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.527624 10130 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.527482 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.541647 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:43.543943 10051 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:43.480070 11480 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:43.551204 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.556281 10123 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:43.556581 11441 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.555778 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.491000 11407 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:43.540838 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50352.8us
I20260709 13:16:43.569908 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.578471 11441 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.591729 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.569015 11451 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.592931 10202 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:43.593389 11451 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.594343 11451 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.595738  9954 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.598937  9954 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.601679  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 4, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.572336 11358 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:16:43.604084  9954 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.606137  9954 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.537142 11460 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:16:43.612870 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.610782  9954 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.582906 11416 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.586789 11441 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.620194  9954 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:43.628098 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.642278 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:43.643798 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.649217 10198 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.652009 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:43.652937 10202 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.653580 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.654498 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.655164 10199 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:43.620771 11362 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:43.611456 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 201732us
I20260709 13:16:43.655315 10203 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:43.641987 11441 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.651592 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:43.669750 10131 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.648276 11371 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:43.641144 11472 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:16:43.670122 10130 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.621999 11369 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.679854 10128 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.659006 11362 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.690117 11362 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.693112 10200 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:43.693759 11472 raft_consensus.cc:993] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:43.694483 11470 raft_consensus.cc:1081] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:43.705623 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:43.705999 10048 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:43.719365 11490 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:43.720116 11490 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:43.720484 10200 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:43.721081 11490 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:43.724314 11489 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:43.707271 11452 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:16:43.707235 11479 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.705610 11369 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:43.680661 11371 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.734086 11371 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.734652 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:43.652601 11402 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:16:43.738982 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.742174 10052 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.745584 11381 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.746304 11381 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.746773 11381 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.753098 10123 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.753436 11459 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.675283 11485 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:16:43.757088 10124 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.762706 10048 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.765375 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:43.771464 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120380us
I20260709 13:16:43.772533 10128 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.776994 11419 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.779186 11381 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.781442 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.797998 11220 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.783051 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.784616 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.794358 11459 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:43.789618 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49867us
I20260709 13:16:43.796392 10129 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:43.806541 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 144115us
I20260709 13:16:43.808624 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.782357 11414 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.800911 11220 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:43.815310 11220 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.811535 10124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45970
I20260709 13:16:43.813431 11456 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.792048 11398 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:43.817102 10124 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:43.818930 10127 raft_consensus.cc:2368] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:43.821584 10124 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.790366 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:43.803520 10122 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:43.816864 11386 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.836040 11220 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:43.836277 10130 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.815047 11358 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.803357 11475 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:16:43.837114 10123 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:43.855953 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:43.857302 10129 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:43.845418 11386 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.863670 11488 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.823161 11480 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:16:43.880551 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:43.864938 11479 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:43.846678 11402 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.867306 11451 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.869127 10019 leader_election.cc:400] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:43.864241 11496 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:43.880105 10124 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.896907 10019 leader_election.cc:403] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.898584 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.858042 11482 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:43.925102 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56274.5us
I20260709 13:16:43.926067 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:43.879839 11386 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.927163 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:43.937249 10205 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.851867 11358 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.933809 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:43.953127 10198 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:43.928467 10124 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:43.939494 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:43.946426 11428 raft_consensus.cc:993] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:43.947273 11467 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:43.947706 11471 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.956352 10202 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:43.958384 10201 raft_consensus.cc:2368] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:43.932964 11470 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:43.973114 10198 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
W20260709 13:16:43.934885 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115103us
I20260709 13:16:43.911288 11497 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:43.886809 11419 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:16:43.957831 11472 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.910429 11402 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.967937 11467 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.886991 11488 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:43.999367 11467 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:43.980346 11470 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:44.017794 11502 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:43.934864 11482 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.023242 10865 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:44.034523 10865 leader_election.cc:403] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:44.008723 11409 raft_consensus.cc:1081] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer d51ad94710c348889d417f7f00032948 to start an election
I20260709 13:16:43.981063 11471 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:43.984951 11389 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:16:44.024590 11381 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.023375 11470 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:44.063490 11362 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.046679 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:16:44.060386 11498 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:43.984360 11497 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:44.061866 11499 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:16:44.056882 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 57985.2us
I20260709 13:16:44.042109 11490 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.038926 11414 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.026479 11502 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:44.080572 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:44.087968 10864 leader_election.cc:400] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:44.088647 10864 leader_election.cc:403] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:16:44.091073 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113578us
I20260709 13:16:44.094089 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.095027 10203 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.104847 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.109964 10049 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:44.100589 11398 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:44.104187 11452 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.088044 11502 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.133836 11509 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.142282 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:44.144215 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.145360 10047 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:44.154347 10203 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:44.116190 11451 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.158366 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:44.134723 11509 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.166266 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:44.163439 11398 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:44.134967 11471 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:44.202785 11500 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
I20260709 13:16:44.208027 11477 raft_consensus.cc:993] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:44.213460 11358 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.215207 11488 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.216657 11482 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.225275 11467 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.227768 10203 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:16:44.228348 11441 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.540s	user 0.000s	sys 0.012s
I20260709 13:16:44.261269 11441 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.225281 11509 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.262068 10051 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:44.229614 11497 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.264868 11441 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.258286 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:44.265839 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.261684 11496 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.266697 10055 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:44.243690 11472 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:44.267226 11496 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.257911 10198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "d51ad94710c348889d417f7f00032948"
 from {username='slave'} at 127.0.0.1:57108
I20260709 13:16:44.270229 10198 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:44.270690 10198 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.272063 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:44.272895 10892 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:44.252779 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:44.273530 11488 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:44.277452 10900 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:44.199692 11462 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:44.237922 11407 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:44.277544 11472 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.279644 11462 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:44.289379 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:44.282152 10198 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.251073 11471 raft_consensus.cc:1081] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:44.282447 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:44.304065 10198 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.316740 11398 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.278563 11362 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.344857 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.348431 10196 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.298700 11462 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:44.347258 11371 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.004s
I20260709 13:16:44.318058 11407 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:44.278373 11441 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.345501 11490 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:44.318660 11381 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:44.302194 11472 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.373032 11381 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.377373 11402 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.355257 11352 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.380393 11352 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.381378 10196 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.382747 11352 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.365418 11371 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.372022 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:44.360797 11495 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:44.361413 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:44.384797 10051 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.385164 10051 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:44.389753 11495 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.391501 11371 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.386292 11381 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.392445 11495 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:44.393319 11501 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.282s	user 0.000s	sys 0.002s
I20260709 13:16:44.387543 10051 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.403527 10051 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.355901 11488 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:16:44.432334 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:44.360190 10205 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.433588 11371 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:44.440243 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:44.441771 10129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.010s	sys 0.038s
I20260709 13:16:44.384846 11467 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.444656 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.448228 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:44.458346 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:44.489400 10203 raft_consensus.cc:2368] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:44.489315 10205 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.460224 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:44.477341 11402 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.490892 10095 leader_election.cc:400] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:44.491425 10095 leader_election.cc:403] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:44.491062 11402 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.492801 11497 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.493770 11402 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.297904 11470 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.495296 10053 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.496176 11470 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.496230 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:44.498083 10196 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:44.485555 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.502964 10204 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:16:44.511260 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52996.9us
W20260709 13:16:44.487373 10200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.017s	sys 0.034s
W20260709 13:16:44.490427 11428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.031s
I20260709 13:16:44.457523 11512 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:44.429430 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.517086 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:44.519140 10196 raft_consensus.cc:2368] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:16:44.524365 10053 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:44.514384 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:16:44.535877 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.014s	sys 0.049s
I20260709 13:16:44.426829 11441 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:44.536958 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.014s	sys 0.049s
I20260709 13:16:44.504967 11452 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:44.544902 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:44.550760 10127 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:44.517823 11428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.011s	sys 0.032s
W20260709 13:16:44.551656 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.019s	sys 0.015s
I20260709 13:16:44.502393 11513 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.556731 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:44.560137 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.019s	sys 0.015s
I20260709 13:16:44.567795 10128 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.520839 11470 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:16:44.516662 10200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.017s	sys 0.034s
W20260709 13:16:44.535064 11386 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.577s	user 0.014s	sys 0.000s
I20260709 13:16:44.520088 11460 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:44.580826 11460 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:44.580669 11386 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.581610 11460 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.516639 11362 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:44.584180 11362 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.501102 11371 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.546113 11409 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.537834 11485 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:44.545619 10197 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:44.546741 11512 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:44.602649 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51350.4us
I20260709 13:16:44.602566 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:44.559365 11472 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.617519 10126 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:44.578922 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:44.616372 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:44.507061 11407 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.624522 10900 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:44.616225 11492 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:16:44.604913 11485 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.611637 11358 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.654140 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:44.672799 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.057s	sys 0.053s
W20260709 13:16:44.673683 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.058s	sys 0.053s
I20260709 13:16:44.576686 11513 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:44.655018 11502 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.532s	user 0.017s	sys 0.000s
I20260709 13:16:44.674379 11485 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:44.618222 11409 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.727743 11514 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:44.728597 11514 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.729194 11514 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:44.747097 11220 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.049s	sys 0.066s
W20260709 13:16:44.747835 11220 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.049s	sys 0.066s
I20260709 13:16:44.680698 11358 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.764065 10128 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.714138 11462 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.755450 11496 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.710675 11502 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.778537 11511 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:44.785029 10203 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:44.794847 10126 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:44.782605 10051 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:44.807922 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:44.808449 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:44.826200 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:44.814890 10053 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:44.830634 10053 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:44.824306 10204 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.821417 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:44.833528 10128 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:44.837389 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.838151 11502 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.829996 10126 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:44.841207 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:44.842199 10126 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:44.847450 11496 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:44.848516 11496 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.779394 11521 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:44.834595 10199 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:44.835911 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.836347 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.837720 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:44.852632 10128 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:44.852134 11521 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.853453 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.853691 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:44.854251 10056 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:44.855463 11510 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.856127 11510 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.857455 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:44.859062 11510 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.856444 11521 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.812361 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:44.480264  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:44.861379 10048 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:44.870203  9951 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:44.871966 11496 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.883318 11371 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.804117 11462 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.885620 10053 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.882867 11511 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.894680 11386 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.900683 10127 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.825883 11441 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.908942 11409 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:44.911612 11523 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:16:44.913434 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:44.910616 11514 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:44.916268 10051 raft_consensus.cc:2368] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:44.917641 11514 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:44.921540 11513 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.754835 11512 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:44.903541 11511 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:44.948459 11358 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:44.958860 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:44.960011 10127 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:44.962809 10891 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.579s	user 0.014s	sys 0.000s
I20260709 13:16:44.963420 10891 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:44.942302 10126 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:44.956895 11497 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
W20260709 13:16:45.025913 11381 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.633s	user 0.016s	sys 0.000s
I20260709 13:16:45.009052 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.030365 10126 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:45.009792 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.032388 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.033231 10127 raft_consensus.cc:2368] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:45.033428 10126 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:45.038789 11517 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.044461 10053 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.044054 11460 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.051100 10127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45970
I20260709 13:16:45.026743 11381 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.084431 10020 leader_election.cc:400] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
W20260709 13:16:45.126030 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.002s	sys 0.014s
W20260709 13:16:45.128808 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.002s	sys 0.014s
I20260709 13:16:45.130263 10128 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:45.084189 11409 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.131465 11409 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.132823 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.133597 10124 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:45.133818 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 817) took 1235 ms. Trace:
I20260709 13:16:45.134150 11409 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.135193 10020 leader_election.cc:403] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:16:45.135860 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51657.2us
I20260709 13:16:45.136055 11444 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:16:45.134367 10129 rpcz_store.cc:276] 0709 13:16:43.898895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.899483 (+   588us) service_pool.cc:224] Handling call
0709 13:16:43.938799 (+ 39316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.522911 (+584112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.522951 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.544424 (+ 21473us) log.cc:844] Serialized 26 byte log entry
0709 13:16:44.699422 (+154998us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:44.699478 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.699570 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.133556 (+433986us) raft_consensus.cc:1685] finished
0709 13:16:45.133578 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.133726 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15654,"raft.queue_time_us":72830,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":15572,"thread_start_us":606247,"threads_started":2,"wal-append.queue_time_us":559134}
I20260709 13:16:45.134963 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.084605 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:45.087198 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 469582us
W20260709 13:16:45.140686 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56225.1us
I20260709 13:16:45.088444 11497 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.089233 11467 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.089699 11510 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.092806 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.093502 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:45.157961 10126 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:45.094709 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:45.095321 10050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.000s	sys 0.002s
W20260709 13:16:45.158725 10050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.000s	sys 0.003s
I20260709 13:16:45.095835 11518 raft_consensus.cc:476] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:16:45.096246 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:45.097959 11470 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:45.098966 11432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.002s
I20260709 13:16:45.114429 10197 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.102507 11462 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.102890 11391 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.103103 11414 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.107059 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:45.119517 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:45.185456 10205 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.081137 11485 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.121572 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:45.140054 10124 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:45.143745 10127 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.142776 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.141346 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.093959 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:45.160473 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64362.2us
I20260709 13:16:45.113966 11480 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.166859 10899 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.124617 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:45.166595 11432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.000s	sys 0.002s
W20260709 13:16:45.120981 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.173064 10053 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:45.221460 10204 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:45.158915 11419 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.222569 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:45.224006 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.240705 10126 raft_consensus.cc:2368] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
W20260709 13:16:45.243810 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102538us
I20260709 13:16:45.241564 10127 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.244782 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:45.165544 11497 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.165127 11467 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:45.257812 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 115185us
I20260709 13:16:45.178845 11444 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.263860 10096 leader_election.cc:400] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:45.265689 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:45.267357 10096 leader_election.cc:403] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:45.227905 11381 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:45.178274 11462 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.270867 11220 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.269229 10053 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:45.272931 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.283102 10126 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.173894 11510 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.282666 11220 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.188148 11414 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.184734 11391 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.253114 11509 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:45.120925 11517 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.363224 11535 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.363678 11531 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:45.363873 11220 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.285115 10122 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.366466 11414 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.257117 11389 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.256834 11511 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.280509 11512 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.362757 11485 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:45.370612 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82648.6us
I20260709 13:16:45.228044 11480 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.399776 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.400130 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.519742 10049 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:45.521740 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.522535 10049 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:45.404572 10205 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.253939 10892 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:45.413255 10047 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:45.413944 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.415031 10129 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:45.528659 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.529541 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:45.416431 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 44) took 1188 ms. Trace:
I20260709 13:16:45.531281 10050 rpcz_store.cc:276] 0709 13:16:44.218261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.220132 (+  1871us) service_pool.cc:224] Handling call
0709 13:16:44.220660 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.221233 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.221282 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.247855 (+ 26573us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.159297 (+911442us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:45.159341 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.159434 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.405907 (+246473us) raft_consensus.cc:1685] finished
0709 13:16:45.405950 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.406219 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17749,"raft.queue_time_us":757,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":12127,"thread_start_us":807033,"threads_started":1,"wal-append.queue_time_us":917626}
I20260709 13:16:45.416913 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.533003 10053 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:45.278134  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:45.533670 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:45.421912 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.004s	sys 0.029s
W20260709 13:16:45.535043 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.005s	sys 0.029s
I20260709 13:16:45.534931 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2276) took 1012 ms. Trace:
I20260709 13:16:45.247825 11419 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.398692 11381 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.535506 10123 rpcz_store.cc:276] 0709 13:16:44.522924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.528526 (+  5602us) service_pool.cc:224] Handling call
0709 13:16:45.533511 (+1004985us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:45.534820 (+  1309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26172800}
I20260709 13:16:45.536084 11381 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.537798 11419 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.367774 11407 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.309700 11456 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.540261 11456 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.542450 11456 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.424301 10127 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.544413 10127 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.545024 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45970 (request call id 2282) took 1011 ms. Trace:
I20260709 13:16:45.545363 10127 rpcz_store.cc:276] 0709 13:16:44.533548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.050880 (+517332us) service_pool.cc:224] Handling call
0709 13:16:45.544912 (+494032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:45.546069 11517 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.489929 10126 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.547483 11510 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.547888 11535 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.493852 10899 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.550197 11467 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.550494 11535 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.518301 11432 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.551451 11432 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.494161 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.473796 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:45.495029 11509 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:45.280190 11398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.007s	sys 0.027s
I20260709 13:16:45.411293 11532 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.296022 11444 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.511886 11220 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.587859 11220 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.548467 11362 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:45.535524 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.1645e+06us
W20260709 13:16:45.596699 10201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.010s
I20260709 13:16:45.622089 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.624749 10124 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:45.621420 10201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.010s
I20260709 13:16:45.629520 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.629964 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2278) took 1376 ms. Trace:
W20260709 13:16:45.630563 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 308323us
I20260709 13:16:45.631441 10020 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.634951 10056 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:45.281111 11470 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.637490 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.643026 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.519469 11536 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.586061 10126 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:45.592121 10096 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:45.592806 10129 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:45.496305 11511 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:45.580021 11398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.008s	sys 0.027s
I20260709 13:16:45.519752 11389 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:45.619374 11381 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.590921 11402 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.634272 11419 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.662142 11517 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.574718 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.656050 10051 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:16:45.669735 10096 leader_election.cc:403] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:45.630384 10201 rpcz_store.cc:276] 0709 13:16:44.253496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.253648 (+   152us) service_pool.cc:224] Handling call
0709 13:16:44.254015 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.257536 (+  3521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.257571 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.445463 (+187892us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b1800104500. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a
0709 13:16:44.446842 (+  1379us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:16:44.455942 (+  9100us) log.cc:844] Serialized 26 byte log entry
0709 13:16:44.617769 (+161827us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:44.679888 (+ 62119us) log.cc:844] Serialized 14 byte log entry
0709 13:16:45.073729 (+393841us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800104500. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:16:45.629571 (+555842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.629682 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:45.629730 (+    48us) raft_consensus.cc:1685] finished
0709 13:16:45.629745 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.629896 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20916,"prepare.queue_time_us":64213,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":404784,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":8074,"spinlock_wait_cycles":54359296,"thread_start_us":716917,"threads_started":2,"wal-append.queue_time_us":325403}
I20260709 13:16:45.674957 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.596160 11371 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.663760 10020 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:45.584372 11512 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:45.687942 11548 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.598423 11460 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.596616 11485 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.705905 10131 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:45.643936 11497 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.703346 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.692680 11552 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.585752 11509 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.576613 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:45.831393 10047 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.599974 11532 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.832296 10047 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:45.688647 10055 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:45.652206 11456 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:45.619473 11451 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:16:45.645975 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.693313 11416 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.680982 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:45.772799 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 631086us
W20260709 13:16:45.763739 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 100056us
I20260709 13:16:45.773995 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1173) took 1278 ms. Trace:
I20260709 13:16:45.816370 11517 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:45.763275 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 171783us
I20260709 13:16:45.820621 10056 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:45.872923 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:45.873893 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:45.752509 11419 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.763276 11371 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.764223 11479 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.683039 11536 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.683584 11511 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:45.768970 11542 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
I20260709 13:16:45.868851 11514 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.763967 11496 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.815229 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:45.813858 11475 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:45.869184 10203 rpcz_store.cc:276] 0709 13:16:44.495623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.495833 (+   210us) service_pool.cc:224] Handling call
0709 13:16:45.773693 (+1277860us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54001fed54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:45.773877 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660847488}
I20260709 13:16:45.868793 11416 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.822002 11485 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.877519 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:45.883668 10048 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:45.764750 11462 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.758829 11545 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.764163 11480 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.822875 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.921476 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:45.821508 11460 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:45.884310 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 371822us
I20260709 13:16:45.923650 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:45.771886 11548 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.926456 10126 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:45.899879 11479 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:45.880064 10047 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.758474 11402 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.933195 10047 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:45.771236 11539 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 13:16:45.902661 11371 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:45.903513 11536 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:45.903893 11543 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:16:45.904310 11551 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:16:45.904554 11554 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.888435 11389 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.765062 11512 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:45.878329 10130 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.072s	sys 0.112s
I20260709 13:16:45.936300 11462 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.919339 11509 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.836057 11552 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:45.989090 11552 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:45.990528 11552 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:45.860889 11456 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.894208 11531 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:45.930392 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.996937 10126 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:45.998066 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.824343 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:45.909220 11391 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.908972 11502 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:46.001333 11502 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.004318 11502 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:45.894816 10053 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.008934 10047 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:45.926188 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.010582 10200 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:46.012109 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.013021 10200 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:46.013995 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.016745 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:46.019675 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:45.904690 11416 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.931996 11517 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.953814 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:45.918722 11550 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:45.932979 11516 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:45.905961 11475 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:45.934237 11533 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:45.974381 10130 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.072s	sys 0.112s
I20260709 13:16:45.985018 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:45.992157 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117142us
I20260709 13:16:45.929153 11460 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.032770 10053 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:46.022282 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:45.955286 11444 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.095412 10204 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:45.993495 11462 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.023661 11552 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.153234 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:46.114624 10205 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:45.972910 11554 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.038789 11371 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.154428 11462 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.151218 10197 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 1.
I20260709 13:16:46.156863 11544 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.158106 11558 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:46.317893 11553 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:16:45.927656 11545 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.101421 11532 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:46.171234 11562 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:16:46.148226 11398 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:46.180500 11456 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:46.182655 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 821) took 1585 ms. Trace:
W20260709 13:16:46.184144 11549 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.339s	user 0.003s	sys 0.006s
I20260709 13:16:46.100183 11381 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.114490 11516 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:46.148483 11533 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.170555 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 784) took 2024 ms. Trace:
I20260709 13:16:46.039099 11479 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.357653 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 25) took 1638 ms. Trace:
I20260709 13:16:46.358932 11561 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:46.360551 10128 rpcz_store.cc:276] 0709 13:16:44.597218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.532161 (+934943us) service_pool.cc:224] Handling call
0709 13:16:45.532479 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:45.566090 (+ 33611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.566149 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:45.566861 (+   712us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.927223 (+360362us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.035014 (+107791us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.089982 (+ 54968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.090122 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.182401 (+ 92279us) raft_consensus.cc:1685] finished
0709 13:16:46.182425 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.182579 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87537,"prepare.queue_time_us":25620,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":104311,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":18425,"thread_start_us":342210,"threads_started":2,"wal-append.queue_time_us":352302}
I20260709 13:16:46.368664 11516 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.374023 10127 rpcz_store.cc:276] 0709 13:16:44.719680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.584568 (+864888us) service_pool.cc:224] Handling call
0709 13:16:45.585036 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:45.597054 (+ 12018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.597089 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:45.635050 (+ 37961us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.999744 (+364694us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:45.999818 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.999956 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.357325 (+357369us) raft_consensus.cc:1685] finished
0709 13:16:46.357357 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.357558 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67722,"raft.queue_time_us":95202,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":18876,"thread_start_us":299466,"threads_started":1,"wal-append.queue_time_us":409797}
I20260709 13:16:46.318032 11558 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.384461 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.384806 11533 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.312638 11371 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:46.377075 10125 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.124s	user 0.015s	sys 0.002s
I20260709 13:16:46.369905 11456 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.172262 11531 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.367291 11398 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.387693 10863 leader_election.cc:400] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:46.411057 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2293) took 1187 ms. Trace:
I20260709 13:16:46.421800 10128 rpcz_store.cc:276] 0709 13:16:45.223847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.382998 (+1159151us) service_pool.cc:224] Handling call
0709 13:16:46.410393 (+ 27395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.410579 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.410606 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.410644 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.410671 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.410725 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.410974 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.424345 11398 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.417129 11475 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:46.416425 11402 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:46.397377 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.420256 11456 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.432119 11535 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:46.481654 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2301) took 1361 ms. Trace:
W20260709 13:16:46.435348 11407 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.897s	user 0.005s	sys 0.009s
I20260709 13:16:46.434900 11513 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:46.482170 10199 rpcz_store.cc:276] 0709 13:16:45.119835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.188663 (+ 68828us) service_pool.cc:224] Handling call
0709 13:16:45.189073 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.557366 (+368293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.557423 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.557520 (+    97us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.557576 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.481350 (+923774us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:46.481406 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.481578 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846154496}
I20260709 13:16:46.492513 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.493989 10199 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:46.438558 11419 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:46.387984 11561 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.408835 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2294) took 1168 ms. Trace:
W20260709 13:16:46.457616 10049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.016s	sys 0.039s
W20260709 13:16:46.451638 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.016s	sys 0.048s
I20260709 13:16:46.412335 11554 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.507756 11407 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:46.363024 11544 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.386567 11485 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.428699 10125 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:46.572247 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2298) took 1466 ms. Trace:
I20260709 13:16:46.496611 11533 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.462939 11470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.018s	sys 0.021s
I20260709 13:16:46.414098 11548 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.435976 11480 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:16:46.536918 11552 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.525012 10205 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:16:46.530908 10049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.017s	sys 0.039s
I20260709 13:16:46.530870 10127 rpcz_store.cc:276] 0709 13:16:45.240757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.383347 (+1142590us) service_pool.cc:224] Handling call
0709 13:16:46.408008 (+ 24661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.408261 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.408297 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.408358 (+    61us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.408493 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.408574 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.408738 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.534886 11566 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.378297 11381 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:46.540135 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 606329us
W20260709 13:16:46.536716 11496 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.010s
I20260709 13:16:46.545571 11554 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:46.527833 11444 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:46.372031 10052 rpcz_store.cc:276] 0709 13:16:44.142719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.146655 (+  3936us) service_pool.cc:224] Handling call
0709 13:16:44.392862 (+246207us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:44.393154 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.393738 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.393780 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.394339 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.129279 (+734940us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.129343 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.129450 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.029711 (+900261us) raft_consensus.cc:1685] finished
0709 13:16:46.035430 (+  5719us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.166956 (+131526us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2235,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":28263,"spinlock_wait_cycles":7612288,"thread_start_us":734417,"threads_started":1,"wal-append.queue_time_us":839115}
I20260709 13:16:46.625864 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2297) took 1348 ms. Trace:
I20260709 13:16:46.565672 11488 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:46.514979 11460 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.621796 11496 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:46.572952 10196 rpcz_store.cc:276] 0709 13:16:45.115679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.121876 (+  6197us) service_pool.cc:224] Handling call
0709 13:16:45.122245 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.621168 (+498923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.621213 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.621265 (+    52us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.621298 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.571817 (+950519us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540012aa54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:46.571911 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.572148 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146845568}
W20260709 13:16:46.523617 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 253) took 2532 ms (client timeout 1784 ms). Trace:
I20260709 13:16:46.506326 11535 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:46.524186 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:46.644218 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.537609 11561 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:46.552390 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:46.565841 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 195400us
W20260709 13:16:46.576725 11470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.019s	sys 0.021s
I20260709 13:16:46.586164 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1143) took 1064 ms. Trace:
I20260709 13:16:46.646102 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:46.646660 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.011s	sys 0.030s
W20260709 13:16:46.669919 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.011s	sys 0.030s
I20260709 13:16:46.626515 10127 rpcz_store.cc:276] 0709 13:16:45.277370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.586051 (+1308681us) service_pool.cc:224] Handling call
0709 13:16:46.625114 (+ 39063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.625370 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.625410 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.625468 (+    58us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.625513 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.625588 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.625765 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.615526 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:46.650671 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.012s
W20260709 13:16:46.672030 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.000s	sys 0.012s
I20260709 13:16:46.587458 11533 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.683166 10199 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:46.686334 10201 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.012s	sys 0.015s
W20260709 13:16:46.686878 10201 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.012s	sys 0.015s
I20260709 13:16:46.668008 10049 rpcz_store.cc:276] 0709 13:16:45.521588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.891513 (+369925us) service_pool.cc:224] Handling call
0709 13:16:45.892012 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.893879 (+  1867us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:45.918376 (+ 24497us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.585803 (+667427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.585863 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.585906 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.585971 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.586096 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24276,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":80,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":35155,"thread_start_us":664129,"threads_started":1,"wal-append.queue_time_us":735795}
I20260709 13:16:46.653728 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 819) took 2086 ms. Trace:
W20260709 13:16:46.603279 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 833) took 1492 ms (client timeout 1591 ms). Trace:
I20260709 13:16:46.651831 11534 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:46.633366 11444 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.648097 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.655833 11532 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:46.708758 10124 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:46.712556 10126 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:16:46.643519 10891 rpcz_store.cc:269] 0709 13:16:44.008930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.028221 (+ 19291us) service_pool.cc:224] Handling call
0709 13:16:46.489381 (+2461160us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b540064f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:46.514660 (+ 25279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726015360}
I20260709 13:16:46.697141 10130 rpcz_store.cc:276] 0709 13:16:44.567173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.137435 (+570262us) service_pool.cc:224] Handling call
0709 13:16:45.137782 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:45.138171 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.138212 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:45.166873 (+ 28661us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.262428 (+ 95555us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.138368 (+875940us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.650291 (+511923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.650412 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.653419 (+  3007us) raft_consensus.cc:1685] finished
0709 13:16:46.653443 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.653643 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13773,"prepare.queue_time_us":663037,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":2693,"raft.queue_time_us":381186,"raft.run_cpu_time_us":513,"raft.run_wall_time_us":24623,"thread_start_us":1323628,"threads_started":3,"wal-append.queue_time_us":178479}
I20260709 13:16:46.630935 11554 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.661352 11456 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:46.725857 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2300) took 1071 ms. Trace:
I20260709 13:16:46.681236 11485 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.797667 10127 rpcz_store.cc:276] 0709 13:16:45.654480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.679621 (+1025141us) service_pool.cc:224] Handling call
0709 13:16:46.725403 (+ 45782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.725575 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.725599 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.725635 (+    36us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.725662 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.725707 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.725793 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:46.673607 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2277) took 2150 ms (client timeout 1899 ms). Trace:
W20260709 13:16:46.801231 10122 rpcz_store.cc:269] 0709 13:16:44.523376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.529155 (+  5779us) service_pool.cc:224] Handling call
0709 13:16:46.659100 (+2129945us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:46.673450 (+ 14350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921402624}
W20260709 13:16:46.699018 11482 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.007s
W20260709 13:16:46.821645 11482 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.015s	sys 0.008s
I20260709 13:16:46.675041 11496 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.633563 11566 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.679298 11545 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.793886 11470 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:46.694293 11371 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:46.653013 11488 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.739833 11573 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.796783 10863 leader_election.cc:400] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
W20260709 13:16:46.742316 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.034s	sys 0.023s
I20260709 13:16:46.722242 11558 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:46.833653 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.005s	sys 0.017s
W20260709 13:16:46.847369 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.017s
W20260709 13:16:46.822826 11559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.037s
W20260709 13:16:46.705883 10125 rpcz_store.cc:269] 0709 13:16:45.088301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.122989 (+ 34688us) service_pool.cc:224] Handling call
0709 13:16:46.580591 (+1457602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:46.839797 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2295) took 1590 ms. Trace:
I20260709 13:16:46.840847 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:46.855077 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.003s	sys 0.021s
W20260709 13:16:46.842113 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.047s	sys 0.071s
W20260709 13:16:46.856419 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 913737us
W20260709 13:16:46.857281 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.024s	sys 0.038s
I20260709 13:16:46.821180 11485 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:46.862280 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.014s	sys 0.010s
W20260709 13:16:46.857861 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.025s	sys 0.039s
W20260709 13:16:46.864408 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.004s	sys 0.021s
W20260709 13:16:46.864646 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 550175us
I20260709 13:16:46.858211 10128 rpcz_store.cc:276] 0709 13:16:45.249040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.539523 (+1290483us) service_pool.cc:224] Handling call
0709 13:16:46.580351 (+ 40828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.580659 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:46.619334 (+ 38675us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.835293 (+215959us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.835387 (+    94us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.835463 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.835583 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.839669 (+  4086us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41026,"prepare.run_cpu_time_us":635,"prepare.run_wall_time_us":2571,"raft.queue_time_us":661,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":8968,"thread_start_us":214679,"threads_started":1}
I20260709 13:16:46.827523 11496 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.796833 11532 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:46.863373 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.014s	sys 0.011s
I20260709 13:16:46.872259 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:46.858440 11559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.010s	sys 0.039s
I20260709 13:16:46.837395 11470 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.877610 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:46.855955 11517 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:46.870796 11480 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:46.856416 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 163026us
I20260709 13:16:46.881664 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:46.888563 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2296) took 1783 ms. Trace:
I20260709 13:16:46.892256 10198 rpcz_store.cc:276] 0709 13:16:45.105206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.105568 (+   362us) service_pool.cc:224] Handling call
0709 13:16:45.105999 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.481165 (+375166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.481222 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.481284 (+    62us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.481316 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.888271 (+1406955us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54001aa4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:46.888330 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.888490 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862454656}
I20260709 13:16:46.897384 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:46.898285 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:46.899758 10019 leader_election.cc:400] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
W20260709 13:16:46.886544 11572 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:16:46.887497 11574 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:16:46.886996 11570 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:16:46.884773 11496 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.883944 11559 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.899004 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:46.878563 11576 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:46.885288 11480 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.885597 11532 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:46.884610 11470 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.882797 11517 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:46.883271 11488 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:46.882020 11444 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:46.918926 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:46.844496 11573 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:46.881369 11371 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:46.902025 10863 leader_election.cc:400] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:46.896612 11545 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:16:46.903460 11544 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.910157 11533 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.910192 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:46.926426 11512 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.023368 11517 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:46.904237 11475 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.003789 11389 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.044871 11220 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:46.970686 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71756.7us
I20260709 13:16:47.036130 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:47.028851 11371 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.040423 11566 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:47.042138 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 137175us
I20260709 13:16:47.028294 10863 leader_election.cc:403] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.061283 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:46.906000 11381 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:46.960106 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:46.912874 11535 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.062741 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1185) took 1036 ms. Trace:
I20260709 13:16:47.032804 11554 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.070737 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.069152 10205 rpcz_store.cc:276] 0709 13:16:46.026053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.585113 (+559060us) service_pool.cc:224] Handling call
0709 13:16:47.035854 (+450741us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:47.062660 (+ 26806us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8337152}
I20260709 13:16:47.072953 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.058315 11371 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:47.073757 10128 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:47.042739 11533 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:47.063879 11479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.028s	sys 0.031s
W20260709 13:16:47.067689 10169 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.906s (SENT) [suppressed 2 similar messages]
I20260709 13:16:47.066918 11488 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.067759 11444 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:47.043398 11512 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:46.960713 11559 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.067323 11389 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:47.074617 11381 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.066998 11475 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:47.067440 10122 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:47.086093 11479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.029s	sys 0.031s
I20260709 13:16:47.103660 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.106235 11485 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.077033 11582 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.971279 11576 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.084247 11573 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:47.105747 11592 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:16:47.096051 11444 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:47.118053 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.118257 11514 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.118152 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 794) took 1856 ms. Trace:
I20260709 13:16:47.119712 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2278) took 2593 ms. Trace:
I20260709 13:16:47.121198 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:47.122905 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2290) took 1942 ms. Trace:
W20260709 13:16:47.111063 11568 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:16:47.144477 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.143976 10129 rpcz_store.cc:276] 0709 13:16:45.180261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.712122 (+531861us) service_pool.cc:224] Handling call
0709 13:16:45.712527 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.955617 (+243090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.955657 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.955708 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:45.955735 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.122576 (+1166841us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001c8254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:47.122633 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.122820 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558476032}
I20260709 13:16:47.101771 11480 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:47.127692 11591 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:16:47.066509 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 177770us
W20260709 13:16:47.158685 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 91029.7us
I20260709 13:16:47.092695 11533 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.141917 10131 rpcz_store.cc:276] 0709 13:16:44.525867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.538094 (+ 12227us) service_pool.cc:224] Handling call
0709 13:16:44.538576 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.817271 (+1278695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.119380 (+1302109us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:47.119613 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2675872640}
I20260709 13:16:47.141726 10051 rpcz_store.cc:276] 0709 13:16:45.261790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.262355 (+   565us) service_pool.cc:224] Handling call
0709 13:16:45.309826 (+ 47471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.752387 (+442561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.117829 (+1365442us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540010a9d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:47.118070 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795189888}
I20260709 13:16:47.154868 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.162801 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:47.163980 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47687.4us
I20260709 13:16:47.118719 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.135514 11544 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.175007 10049 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:47.137926 11597 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.176751 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.180090 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.180301 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.182503 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.136919 11593 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.137089 11459 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:47.140051 11444 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.189039 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.193585 10047 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.242758 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:47.137610 11596 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.149972 11514 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.147080 11521 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:47.163754 11485 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.140373 11579 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:47.148371 11535 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:47.192469 10050 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:47.236596 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.129014 11381 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:47.316833 11586 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:16:47.317374 11381 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.186486 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:47.239226 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.243520 10204 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:47.193267 11416 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.156s	user 0.008s	sys 0.012s
I20260709 13:16:47.271587 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2292) took 2138 ms. Trace:
I20260709 13:16:47.378258 11416 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.169422 11407 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.379530 11407 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.194834 11518 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:47.244047 11470 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.139815 11582 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:47.376394 11588 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:16:47.377053 11480 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.377658 11381 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:47.134001 11475 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.253290 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.195294 11544 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.232089 11597 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.235719 11389 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.398650 10204 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
W20260709 13:16:47.701493 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2299) took 2145 ms (client timeout 2209 ms). Trace:
I20260709 13:16:47.700491 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.378885 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2303) took 1464 ms. Trace:
W20260709 13:16:47.703053 10204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1183) took 1700 ms (client timeout 1826 ms). Trace:
W20260709 13:16:47.703523 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 517183us
W20260709 13:16:47.703842 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 326154us
I20260709 13:16:47.705687 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:47.378970 10202 rpcz_store.cc:276] 0709 13:16:45.104053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.105078 (+  1025us) service_pool.cc:224] Handling call
0709 13:16:45.105404 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.369194 (+263790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.369232 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.369281 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:45.369319 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.241927 (+1872608us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001bfb54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:47.241988 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.242174 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606114560}
I20260709 13:16:47.705681 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:47.703161 10124 rpcz_store.cc:269] 0709 13:16:45.581001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.625960 (+ 44959us) service_pool.cc:224] Handling call
0709 13:16:47.701281 (+2075321us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:47.701425 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888852736}
I20260709 13:16:47.708668 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.704505 10201 rpcz_store.cc:276] 0709 13:16:45.914860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.915081 (+   221us) service_pool.cc:224] Handling call
0709 13:16:45.915585 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.162060 (+246475us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:46.687307 (+525247us) log.cc:844] Serialized 14 byte log entry
0709 13:16:46.856714 (+169407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.856780 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.856833 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.378569 (+521736us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:47.378630 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.378810 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15205,"prepare.queue_time_us":749275,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":2225,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6587,"spinlock_wait_cycles":565841280,"thread_start_us":684183,"threads_started":2,"wal-append.queue_time_us":266953}
I20260709 13:16:47.712641 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.712831 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.713816 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:47.689401 11595 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:16:47.716261 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.717048 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:47.280637 11596 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.718236 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.309906 10122 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:47.719547 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.714531 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.714906 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.315748 11583 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.272823 11488 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:47.710780 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.723905 10056 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:47.724103 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:47.725328 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:47.729967 10047 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:47.720225 11596 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:47.303367 11546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.751s	user 0.031s	sys 0.067s
I20260709 13:16:47.563948 11504 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:47.703068 11561 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.041s	user 0.010s	sys 0.006s
W20260709 13:16:47.658795 11594 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.312s	user 0.000s	sys 0.000s
I20260709 13:16:47.735550 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.312239 11532 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:47.381038 11497 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.251470 11459 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:47.802006 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.743285 11538 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 2.247s	user 0.007s	sys 0.010s
I20260709 13:16:47.590978 11490 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.380743 11416 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.616375 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 823) took 2989 ms. Trace:
I20260709 13:16:47.309263 11485 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.732719 11479 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:47.727276 11518 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.274479 11593 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.454952 11535 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.767354 11559 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:47.768443 11432 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.319576 11579 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.778589 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:16:47.780019 11371 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.002s
I20260709 13:16:47.780786 11576 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:47.780573 11504 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.780885 11589 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:47.781992 11533 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:47.782724 11462 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.444s	user 0.010s	sys 0.006s
I20260709 13:16:47.782490 11561 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:47.744005 11546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.188s	user 0.031s	sys 0.068s
I20260709 13:16:47.726815 11536 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.733808 11534 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:47.705081 10204 rpcz_store.cc:269] 0709 13:16:46.002780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.391954 (+389174us) service_pool.cc:224] Handling call
0709 13:16:47.242586 (+850632us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:47.702982 (+460396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12694400}
I20260709 13:16:47.313243 11521 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.728052 11480 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.728202 11470 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.774490 11544 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:47.766217 11511 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.867s	user 0.019s	sys 0.002s
W20260709 13:16:47.807301 11516 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.422s	user 0.018s	sys 0.001s
I20260709 13:16:47.740718 11512 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:47.825381 11444 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.005s
I20260709 13:16:47.823416 11601 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:47.934175 11601 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:47.934706 11601 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.936295 11521 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:47.877883 11462 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.939296 11444 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.942003 11444 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.922650 10203 raft_consensus.cc:2368] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:47.912528 10050 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:47.942749 10200 raft_consensus.cc:2368] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:47.943385 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:47.922705 10049 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:47.929028 11593 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:47.741979 11583 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.806854 11497 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.878841 10123 rpcz_store.cc:276] 0709 13:16:44.626799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.536962 (+910163us) service_pool.cc:224] Handling call
0709 13:16:45.537447 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:45.619351 (+ 81904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.629213 (+  9862us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:45.708196 (+ 78983us) log.cc:844] Serialized 26 byte log entry
0709 13:16:46.179648 (+471452us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:46.179723 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.179836 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:47.606195 (+1426359us) raft_consensus.cc:1685] finished
0709 13:16:47.615952 (+  9757us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.616254 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45239,"raft.queue_time_us":90935,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":18363,"thread_start_us":335697,"threads_started":1,"wal-append.queue_time_us":807711}
I20260709 13:16:47.897768 11490 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.947460 11583 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.810751 11488 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.948369 11488 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.949753 11490 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.949753 11485 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:47.896368 11536 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.899701 11534 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.895331 11371 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:47.896194 11532 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:47.931977 11597 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.904565 11479 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:47.933319 11517 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.870s	user 0.004s	sys 0.007s
I20260709 13:16:47.909150 11416 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.944463 11582 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:48.095566 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 366766us
I20260709 13:16:47.960748 10124 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:48.241362 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:47.951517 11534 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:47.942068 11462 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:47.953562 11536 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:47.951053 11488 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:47.955641 11497 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:48.230175 11389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.018s	sys 0.032s
I20260709 13:16:47.957655 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:47.957504 11561 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:48.257328 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:47.956070 11371 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:48.228170 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 378723us
I20260709 13:16:48.273383 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:48.237382 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 530786us
I20260709 13:16:48.245422 11544 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:48.305534 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.305775 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.305929 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.306553 10204 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:47.917472 11518 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:48.235524 11416 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.287014 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.316912 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:47.961783 11511 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.301920 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:48.307637 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:47.967103 11516 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:48.285041 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.238744 11517 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.243259 11535 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.299883 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:48.320060 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:48.321269 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:48.321496 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:48.327965 11511 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.328431 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.372131 10202 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:48.334172 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:48.324469 10169 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 806f35e66c234d569cbafd8b3981518f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:48.349974 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:48.355906 11579 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:48.356549 11504 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:48.359313 11514 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.083s	user 0.015s	sys 0.000s
I20260709 13:16:48.360250 11516 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:48.361912 11601 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.367250 11517 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.311563 11532 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.370144 10900 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:48.330073 11479 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.325683 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:48.374243 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 74619.7us
I20260709 13:16:48.416074 11601 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.378641 10897 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:48.373019 10893 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
W20260709 13:16:48.857659 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 51) took 1539 ms (client timeout 1690 ms). Trace:
W20260709 13:16:48.540422 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.858228 10049 rpcz_store.cc:269] 0709 13:16:47.318305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.328346 (+ 10041us) service_pool.cc:224] Handling call
0709 13:16:47.713665 (+385319us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:48.857403 (+1143738us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:48.857542 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493568000}
I20260709 13:16:48.858958 11578 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:48.569566 10047 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.022s	sys 0.024s
I20260709 13:16:48.859658 11578 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:48.860514 10047 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.023s	sys 0.024s
I20260709 13:16:48.635105 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 847) took 1566 ms. Trace:
I20260709 13:16:48.312284 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 845) took 1248 ms. Trace:
I20260709 13:16:48.862775 11578 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.863309 10129 rpcz_store.cc:276] 0709 13:16:47.063983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.731909 (+667926us) service_pool.cc:224] Handling call
0709 13:16:47.744843 (+ 12934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.745010 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:48.228429 (+483419us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.311637 (+ 83208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.311722 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:48.311806 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.311952 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.312185 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":551101,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":65,"raft.queue_time_us":147,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":8509,"thread_start_us":503004,"threads_started":2,"wal-append.queue_time_us":160393}
I20260709 13:16:48.865461 11578 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.865814 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:48.677182 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.040s	sys 0.052s
I20260709 13:16:48.687914 10128 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:48.867339 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.040s	sys 0.052s
I20260709 13:16:48.867678 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 852) took 1731 ms. Trace:
W20260709 13:16:48.868033 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.14393e+06us
I20260709 13:16:48.868108 10128 rpcz_store.cc:276] 0709 13:16:47.136202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.308534 (+1172332us) service_pool.cc:224] Handling call
0709 13:16:48.687592 (+379058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.867397 (+179805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.867620 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.868979 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:48.869217 10049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:48.869848 10049 raft_consensus.cc:606] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:48.869832 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:48.870352 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:48.689953 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:48.872110 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 182216us
I20260709 13:16:48.873535 11579 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:48.691870 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 851) took 1576 ms. Trace:
I20260709 13:16:48.875245 11511 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.875903 11608 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:48.876159 11511 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.876600 11608 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:48.877235 11608 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:48.726315 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.013s	sys 0.004s
W20260709 13:16:48.879567 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.178s	user 0.013s	sys 0.004s
I20260709 13:16:48.881203 11601 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:48.884562 11601 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:48.885151 11601 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:48.886121 11381 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.861651 10122 rpcz_store.cc:276] 0709 13:16:47.068374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.733577 (+665203us) service_pool.cc:224] Handling call
0709 13:16:47.802114 (+ 68537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.815686 (+ 13572us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:48.238027 (+422341us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.634723 (+396696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.634799 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:48.634841 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.634910 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.635042 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":501719,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":145,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":710161,"threads_started":2,"wal-append.queue_time_us":156169}
I20260709 13:16:48.886831 11381 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.888495 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:48.890172 11381 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:48.740581 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2307) took 2811 ms (client timeout 1848 ms). Trace:
I20260709 13:16:48.875028 11496 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:48.740643 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 854) took 1498 ms. Trace:
I20260709 13:16:48.892757 10124 rpcz_store.cc:276] 0709 13:16:47.242026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.710765 (+468739us) service_pool.cc:224] Handling call
0709 13:16:48.739729 (+1028964us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54002d0d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:48.740561 (+   832us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568159104}
I20260709 13:16:48.870877 10128 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:48.741349 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:48.895223 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 650743us
W20260709 13:16:48.891717 10203 rpcz_store.cc:269] 0709 13:16:45.929271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.929765 (+   494us) service_pool.cc:224] Handling call
0709 13:16:47.120983 (+1191218us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:48.740265 (+1619282us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54001bf154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:48.740464 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797458560}
I20260709 13:16:48.896381 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:48.742909 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.84227e+06us
I20260709 13:16:48.898732 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:48.758626  9951 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 108) took 4776 ms. Trace:
I20260709 13:16:48.769181 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:48.900074 10126 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:48.778719 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:48.781064 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:48.903021 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 122049us
I20260709 13:16:48.878804 11511 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.301862 10125 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:48.926422 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:48.927539 10129 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:48.786366 10892 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.879s	user 0.018s	sys 0.000s
W20260709 13:16:48.706296 11475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.002s	sys 0.006s
I20260709 13:16:48.378139 11488 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.053505 11488 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:48.689823 11414 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:48.410924 11416 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.651087 11480 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:48.383507 11514 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.062291 11514 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.748968 11517 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.064275 11517 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.417465 11534 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.070238 11534 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.371424 11518 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:49.072449 10127 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:16:48.794575 11459 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.793408 11470 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.828343 11606 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.087972 11606 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.090685 11606 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.828881 11544 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:48.831960 10200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1177) took 2852 ms (client timeout 1802 ms). Trace:
I20260709 13:16:49.091535 11544 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:49.091599 10200 rpcz_store.cc:269] 0709 13:16:45.979573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.013700 (+ 34127us) service_pool.cc:224] Handling call
0709 13:16:48.831654 (+2817954us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54003598d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:48.831858 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4433509120}
I20260709 13:16:49.093992 11544 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.837098 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.837761 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.838989 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.096000 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.097294 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.840111 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.098397 10199 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:48.841737 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.843981 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.847898 10202 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:48.850867 11613 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:48.851686 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.102662 11613 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.102957 10131 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.104751 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.106040 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:48.866286 11578 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.107309 11613 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.108897 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.110282 11480 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:48.875458 11389 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:48.739946 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.006s	sys 0.008s
I20260709 13:16:48.781212 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.110909 10196 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:49.111085 11389 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.111477 10048 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:48.908092 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.111979 11389 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.113359 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.110921 11480 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.909333 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:48.915035 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:48.905176 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:48.924393 10055 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:49.115080 10054 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:16:49.115379 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 49) took 1813 ms (client timeout 2189 ms). Trace:
I20260709 13:16:48.918851 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.116516 10051 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:49.116648 10122 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.117455 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.109675 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:48.939659 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:49.119415 10124 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.121315 10196 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.874469 11579 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.125265 11579 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:48.892066 11496 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:48.927018 11381 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.126850 11381 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:49.127324 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.214s	user 0.007s	sys 0.009s
I20260709 13:16:49.128489 11381 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.136583 11496 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:49.112099 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1162) took 1808 ms (client timeout 1522 ms). Trace:
W20260709 13:16:49.138180 10048 rpcz_store.cc:269] 0709 13:16:47.304216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.307674 (+  3458us) service_pool.cc:224] Handling call
0709 13:16:48.781005 (+1473331us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:49.112025 (+331020us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42025728}
I20260709 13:16:49.139827 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.141633 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:49.115801 10055 rpcz_store.cc:269] 0709 13:16:47.301971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.307822 (+  5851us) service_pool.cc:224] Handling call
0709 13:16:48.778477 (+1470655us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:49.115307 (+336830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45673344}
I20260709 13:16:49.144636 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.145648 10055 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:48.972371 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.147987 10203 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:49.116080 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.151574 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:49.152732 10054 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:48.924998 11504 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:48.899788  9951 rpcz_store.cc:276] 0709 13:16:44.008713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.024544 (+ 15831us) service_pool.cc:224] Handling call
0709 13:16:48.758516 (+4733972us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:45.261010 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:45.362298 (+101288us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:45.362340 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:16:45.362360 (+    20us) tablet.cc:662] Decoding operations
0709 13:16:45.362770 (+   410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:45.362855 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:45.362886 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:45.363107 (+   221us) tablet.cc:701] Row locks acquired
0709 13:16:45.363170 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:16:45.363351 (+   181us) write_op.cc:270] Start()
0709 13:16:45.363486 (+   135us) write_op.cc:276] Timestamp: P: 1783603005363292 usec, L: 0
0709 13:16:45.363507 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:16:45.551363 (+187856us) log.cc:844] Serialized 533 byte log entry
0709 13:16:47.125712 (+1574349us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:16:48.322604 (+1196892us) op_driver.cc:464] REPLICATION: finished
0709 13:16:48.401928 (+ 79324us) write_op.cc:301] APPLY: starting
0709 13:16:48.403182 (+  1254us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:48.408739 (+  5557us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:48.409189 (+   450us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:48.412702 (+  3513us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:48.412894 (+   192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:48.412935 (+    41us) write_op.cc:312] APPLY: finished
0709 13:16:48.413991 (+  1056us) log.cc:844] Serialized 23 byte log entry
0709 13:16:48.414181 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:48.414267 (+    86us) write_op.cc:454] Released schema lock
0709 13:16:48.414306 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:48.414683 (+   377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:48.414710 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":383781,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":68365,"apply.run_cpu_time_us":2417,"apply.run_wall_time_us":16360,"num_ops":1,"prepare.queue_time_us":349616,"prepare.run_cpu_time_us":104323,"prepare.run_wall_time_us":2636588,"raft.queue_time_us":481646,"raft.run_cpu_time_us":14205,"raft.run_wall_time_us":61040,"replication_time_us":2958902,"spinlock_wait_cycles":12127104,"thread_start_us":2889765,"threads_started":4,"wal-append.queue_time_us":2263985}]]}
I20260709 13:16:48.933386 10892 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:48.941929 11511 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:49.015425 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:48.934553 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 836) took 1875 ms (client timeout 2019 ms). Trace:
I20260709 13:16:48.964259 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:48.931463 11561 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:16:49.018493 11516 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 13:16:49.024947 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54249.6us
I20260709 13:16:49.025933 11584 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:49.021381 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52680.6us
I20260709 13:16:49.031191 11601 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.168022 11504 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.190224 10048 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:49.035907 11608 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:48.875389 10123 rpcz_store.cc:276] 0709 13:16:47.115776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.946743 (+830967us) service_pool.cc:224] Handling call
0709 13:16:47.966932 (+ 20189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.040695 (+ 73763us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:48.690418 (+649723us) log.cc:844] Serialized 14 byte log entry
0709 13:16:48.691472 (+  1054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.691533 (+    61us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:48.691592 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.691671 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.691813 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":234869,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":131,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":457888,"threads_started":2,"wal-append.queue_time_us":1015}
I20260709 13:16:49.042706 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 53) took 1666 ms. Trace:
W20260709 13:16:49.046370 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3629253389999576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260709 13:16:49.048148 10129 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:16:49.050529 11475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.273s	user 0.003s	sys 0.007s
I20260709 13:16:48.607187 11407 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:49.055684 11414 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.058485 11488 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.075989 11517 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.241559 10205 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:49.078500 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 850) took 1978 ms. Trace:
I20260709 13:16:49.093823 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.244756 10200 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:48.836207 10197 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:49.101254 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.246718 10050 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:49.247567 11414 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.248500 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:48.731748 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2301) took 3025 ms. Trace:
W20260709 13:16:49.115504 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 210378us
I20260709 13:16:49.249552 10130 rpcz_store.cc:276] 0709 13:16:45.706023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.794584 (+1088561us) service_pool.cc:224] Handling call
0709 13:16:46.795192 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.838896 (+ 43704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.838930 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:46.838973 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:46.839001 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.731389 (+1892388us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b540015b154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:48.731454 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.731648 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99888000}
I20260709 13:16:49.116874 11480 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.118819 10201 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.254138 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.254797 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:49.255366 11488 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:49.256366 11488 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:49.256847 11488 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:49.256567 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.257710 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.258015 11488 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:48.932930 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 103968us
I20260709 13:16:49.199514 11470 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:49.193987 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 291) took 1404 ms. Trace:
I20260709 13:16:49.147055 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.266772 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.266829 10892 rpcz_store.cc:276] 0709 13:16:47.772090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.778150 (+  6060us) service_pool.cc:224] Handling call
0709 13:16:49.176350 (+1398200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:49.207657 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.281603 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.288693 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:49.290475 10055 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:49.291289 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.280265 11470 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:49.293480 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 395809us
I20260709 13:16:49.213912 11389 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.199673 11584 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.176628 11613 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.195356 11511 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:49.174536 10125 rpcz_store.cc:269] 0709 13:16:47.058797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.062555 (+  3758us) service_pool.cc:224] Handling call
0709 13:16:47.957418 (+894863us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:48.930665 (+973247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4618112}
I20260709 13:16:49.190538 10052 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.226114 10049 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:49.210544 11601 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.226918 11504 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.349665 10895 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:49.231544 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2594253390000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18020000000000003s for a response
W20260709 13:16:49.355208 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2378253389999827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:16:49.355615 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1163253390000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 13:16:49.358845 11496 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.361477 11389 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.361920 11496 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.365967 10204 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:49.311713 11407 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:16:49.239728 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.231369 10047 rpcz_store.cc:276] 0709 13:16:47.376788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.730912 (+354124us) service_pool.cc:224] Handling call
0709 13:16:47.731407 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.731722 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:16:48.861028 (+1129306us) log.cc:844] Serialized 14 byte log entry
0709 13:16:49.042269 (+181241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:49.042339 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:49.042389 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.042473 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.042633 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":627312,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":116,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"thread_start_us":1308307,"threads_started":2}
I20260709 13:16:49.368454 11470 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.101635 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.109455 11578 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:48.929656 11620 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.369866 11620 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.370368 11620 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.358342 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.371151 11511 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.372363 11578 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.372831 11578 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.250419 10054 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:49.373220 11578 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.373603 10054 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.256398 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.375541 10894 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:49.376299 10198 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:49.260669 10050 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.247915 10127 rpcz_store.cc:276] 0709 13:16:47.101307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.816812 (+715505us) service_pool.cc:224] Handling call
0709 13:16:47.846526 (+ 29714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.073150 (+1226624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.078052 (+  4902us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400389ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:49.078390 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2818384640}
I20260709 13:16:49.118754 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.378484 10051 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:49.263319 11608 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.268105  9953 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 216) took 2128 ms. Trace:
I20260709 13:16:49.379537  9953 rpcz_store.cc:276] 0709 13:16:47.139584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.139771 (+   187us) service_pool.cc:224] Handling call
0709 13:16:48.818168 (+1678397us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:16:49.268004 (+449836us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":2624153344}
I20260709 13:16:49.269621 10124 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.269621 10201 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:49.280601 11488 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.278827 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.313326 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:49.339178 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 455914us
W20260709 13:16:49.347785 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 43510.2us
I20260709 13:16:49.382098 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:49.349134 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 52) took 1973 ms. Trace:
I20260709 13:16:49.385583 11496 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.384763 11480 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:49.385915 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 402129us
I20260709 13:16:49.385295 10051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:49.388502 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.388674 10051 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.393151 11584 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.384274 10053 rpcz_store.cc:276] 0709 13:16:47.376372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.720564 (+344192us) service_pool.cc:224] Handling call
0709 13:16:47.720900 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.733700 (+ 12800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.733729 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.734260 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.128051 (+1393791us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:49.337193 (+209142us) log.cc:844] Serialized 14 byte log entry
0709 13:16:49.348734 (+ 11541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.348813 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:49.348864 (+    51us) raft_consensus.cc:1685] finished
0709 13:16:49.348879 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:49.349033 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41281,"prepare.queue_time_us":209093,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":86,"raft.queue_time_us":91800,"raft.run_cpu_time_us":493,"raft.run_wall_time_us":13961,"thread_start_us":1439201,"threads_started":2,"wal-append.queue_time_us":547791}
I20260709 13:16:49.399086 11620 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.405817 10020 leader_election.cc:400] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:49.404709 11620 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.407373 11517 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:49.407945 11517 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:49.408900 11517 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.409070 11479 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:49.411386 10051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87da05694091470284f66a8181207e2f"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:49.412045 10051 raft_consensus.cc:606] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.406935 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.417295 11628 raft_consensus.cc:2764] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:49.418845 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:49.385221 10095 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:49.420989 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:49.421494 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108237us
I20260709 13:16:49.423404 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.418763 10052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "db052bef7536453fa2d7f5a00e314a18"
 from {username='slave'} at 127.0.0.1:49976
W20260709 13:16:49.427402 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 512366us
I20260709 13:16:49.428311 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.431568 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.432794 10128 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.434383 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:49.435151 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.436163 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.431524 10052 raft_consensus.cc:606] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Received request to transfer leadership to TS db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:49.437230 10052 raft_consensus.cc:614] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:49.437470 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.439864 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.439893 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:49.441273 10130 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:49.440310 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.442508 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:49.443470 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.444787 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.445595 10201 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:49.446249 11496 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:49.446647 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.447520 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.431752 11470 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:49.448100 11579 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:49.448849 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.449648 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.450261 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.427577 11628 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:49.451356 11623 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.453176 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.436136 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.455474 10131 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:49.455511 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.444598 10127 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.456672 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:49.451395 10205 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:49.457386 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.527737 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.622119 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:49.618572 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 196526us
I20260709 13:16:49.604359 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:49.630192 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.631011 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.545203 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:49.628772 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 100494us
I20260709 13:16:49.632076 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:49.429414 11407 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.697870 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:49.599709 11625 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.261s	user 0.000s	sys 0.005s
I20260709 13:16:49.631253 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.668915 11623 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.438498 11630 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:49.719485 11630 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.720032 11630 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.720610 11623 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.683424 11638 ts_tablet_manager.cc:933] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.478715 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:49.683730 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.722229 11628 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:49.713609 11561 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:49.664968 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.657364 10201 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:49.733731 11462 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.733476 11578 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.734728 11462 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.713347 10893 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:49.735230 11462 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.736866 11578 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:49.429595 11584 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.738473 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 259921us
I20260709 13:16:49.438285 11626 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.301s
I20260709 13:16:49.722862 11470 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.668092 11579 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:49.633908 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.724474 10128 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.726653 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.601665 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.751493 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.727934 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.480811 10900 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:49.754045 11416 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:49.527045 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.754586 11416 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:49.755057 11416 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.755172 10899 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.435871 11544 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:49.756866 11544 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.757349 11544 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.758121 11579 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.714893 11504 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.758337 11462 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.755517 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:49.892056 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136635us
I20260709 13:16:49.892297 11641 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:49.893388 11462 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.480064 11496 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:49.775527 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.894109 11496 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.740718 11627 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.738082 11584 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.896467 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:49.897078 11584 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.897361 10047 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:49.459481 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:49.899533 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.753106 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:49.528874 11639 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.136s	user 0.003s	sys 0.000s
I20260709 13:16:49.900604 10125 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:16:49.792064 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119069us
I20260709 13:16:49.764101 11470 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:49.903587 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.905162 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.906421 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.796659 11638 tablet_copy_client.cc:323] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:16:49.907570 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.908313 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:49.908936 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:49.909864 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.910310 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.753746 10895 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:16:49.803215 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002799e0 after lost signal to thread 8928
I20260709 13:16:49.656891 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:49.912746 10921 debug-util.cc:398] Leaking SignalData structure 0x7b0800230260 after lost signal to thread 9988
W20260709 13:16:49.660966 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 201827us
W20260709 13:16:49.913209 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002300c0 after lost signal to thread 10002
I20260709 13:16:49.913000 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:49.913590 11462 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.429353 11389 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.676791 10122 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.764676 10126 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.857120 10200 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:49.859102 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.858170 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.859442 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.858173 11579 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:49.861497 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.668735 10124 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:49.914678 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002ebf80 after lost signal to thread 10080
I20260709 13:16:49.922230 10865 leader_election.cc:400] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3.
I20260709 13:16:49.915335 11601 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 13:16:49.932837 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002efa60 after lost signal to thread 10155
I20260709 13:16:49.919298 11479 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.933670 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:49.934531 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002efe60 after lost signal to thread 10229
I20260709 13:16:49.934360 11462 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.926098 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.923862 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.935985 10203 raft_consensus.cc:2368] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:49.867914 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.899582 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.869338 10130 raft_consensus.cc:2368] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:49.937013 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.480157 11381 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:49.918978 11608 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:16:49.759681 10202 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:49.938486 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.939232 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1208) took 1087 ms. Trace:
I20260709 13:16:49.913126 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.869789 10198 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:49.940150 10196 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:49.890107 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.940717 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.916976 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.941362 10198 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.942140 11579 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:49.891088 11561 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50710 (request call id 7) took 6192 ms. Trace:
I20260709 13:16:49.942282 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:49.943048 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08000408e0 after lost signal to thread 10924
I20260709 13:16:49.887648 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.900435 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.909047 11517 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.543998 10050 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:49.939728 10202 rpcz_store.cc:276] 0709 13:16:48.852355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.100865 (+248510us) service_pool.cc:224] Handling call
0709 13:16:49.938992 (+838127us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540035a2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:49.939161 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178376064}
W20260709 13:16:49.840713 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 174684us
I20260709 13:16:49.942909 11561 rpcz_store.cc:276] 0709 13:16:43.698660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.702433 (+  3773us) service_pool.cc:224] Handling call
0709 13:16:43.729684 (+ 27251us) log.cc:844] Serialized 499 byte log entry
0709 13:16:49.890979 (+6161295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46635,"raft.queue_time_us":530646,"raft.run_cpu_time_us":1276,"raft.run_wall_time_us":82172,"thread_start_us":779858,"threads_started":2,"wal-append.queue_time_us":180024,"wal-append.run_cpu_time_us":4025,"wal-append.run_wall_time_us":1229789}
I20260709 13:16:49.945433 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.945673 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.946656 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:49.924386 11470 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:16:49.947299 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.947367 11584 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:49.948712 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.949692 11516 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:49.950197 11516 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:49.950603 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.774781 11504 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:49.761098 11628 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.864521 11544 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:49.902858 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:49.880123 11416 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:49.935866 10051 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
I20260709 13:16:49.918617 11613 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.939978 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:49.892805 11641 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:49.740909 10095 leader_election.cc:400] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:49.976707 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:49.864188 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.991660 10095 leader_election.cc:403] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.991848 11561 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:49.950855 11630 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:49.962064 10203 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:49.935026 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:49.989481 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86715.8us
I20260709 13:16:49.914770 11514 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:49.984412 10096 connection.cc:256] RPC call timeout handler was delayed by -1.4213857260001532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:16:49.996042 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:50.009567 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:49.993187 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 252412us
I20260709 13:16:49.950619 11516 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.992667 11579 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:16:50.015183 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.022810 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.023034 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.015034 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.016947 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:50.015484 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.016692 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.013935 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.012080 10204 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.017875 10047 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.021508 11647 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:49.993721 11584 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.021337 11623 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:16:50.023211 11644 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.017771 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.024484 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:49.996649 11532 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:49.993103 11641 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:49.991616 11389 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.032510 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.025415 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.033962 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.047232 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.047989 10129 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
I20260709 13:16:49.937203 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.034293 10050 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:50.031783 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.032960 11488 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.022295 11504 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:50.058240 11504 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.057865 11488 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.059429 11504 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.035527 11623 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:50.062143 11623 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:16:50.041251 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 125137us
I20260709 13:16:50.063694 11623 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.064267 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.065152 11623 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.065722 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.066684 10130 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.037067 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.036006 10051 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:49.993906 11613 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:50.047923 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:50.067941 10049 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:50.065739 10124 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:16:50.069422 11579 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.018342 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:49.951922 11517 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.070406 10053 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:16:50.070830 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.032853 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.071167 10054 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.032692 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.072235 11517 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.072458 10048 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:50.032285 11475 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.050261 11544 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:50.074030 11544 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:50.074335 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:50.051054 10047 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:50.045400 11381 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:50.075490 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.075811 11381 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.076248 11381 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.076412 11606 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:50.076922 11606 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.077180 11511 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:50.077998 11511 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:49.972954 11496 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.052789 11462 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.078435 11511 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.036892 11644 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:50.043499 11532 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:50.079016 11644 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.079674 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:50.080010 11532 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.031937 11647 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.081166 11532 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.081279 11647 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.081986 11578 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:50.053923 10056 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.043166 11416 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.082716 11578 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.067829 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.055751 11584 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.042261 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.084501 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:50.085172 11462 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:50.053809 11389 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.043081 11561 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:50.086648 11561 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.069564 11488 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.070393 11579 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.070405 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.088219 11579 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.088660 10126 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.077322 11606 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.088068 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet 806f35e66c234d569cbafd8b3981518f from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:16:50.087993 11561 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.090332 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet 806f35e66c234d569cbafd8b3981518f from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f
I20260709 13:16:50.085724 11462 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.049062 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.093870 11475 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.093737 11462 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.083267 11578 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.096990 11626 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:50.036429 11630 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.100863 11626 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.105455 10123 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.110730 11496 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:50.113601 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.109905 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:50.115455 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66486.1us
I20260709 13:16:50.115610 10203 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:50.063730 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.116389 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.117185 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.117566 10204 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.117985 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:50.118752 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76583.3us
W20260709 13:16:50.119695 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56075.6us
I20260709 13:16:49.930032 11627 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.120645 11496 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:50.120846 11470 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.122287 11630 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.122300 11544 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.067298 10196 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:50.139998 10130 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:50.162501 10129 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.147908 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.163477 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.158926 10054 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:50.159921 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.164377 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:50.165493 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.166800 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.171605 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.172581 10049 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:50.171696 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:50.160915 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.176090 10124 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
I20260709 13:16:50.176645 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:50.176901 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.186452 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.190806 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.191174 10204 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:50.191537 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.192487 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.187047 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.187644 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.195218 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.196621 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.188247 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.126636 11606 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.195863 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:50.149067 11470 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:50.159247 11630 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.200376 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.183753 11513 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.207737 10129 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.201443 10020 leader_election.cc:400] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:50.202462 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.183331 11488 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:50.203307 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:50.196347 11649 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:16:50.200107 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.203753 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:50.205050 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:50.161361 11496 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:50.209703 10020 leader_election.cc:403] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:16:50.158898 11626 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.143860 11516 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.209959 10202 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:16:50.251441 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50174.1us
I20260709 13:16:50.207718 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:50.208420 11630 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.272840 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.273098 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.273928 10124 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.199400 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.279183 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.235321 11488 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:50.187678 11517 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.236068 11579 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.198820 11632 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.237380 10047 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:50.237466 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.238250 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.245591 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.240365 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.240489 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.250945 10214 tablet_copy_source_session.cc:215] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:50.248085 10201 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:50.249909 10196 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:50.251233 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.265288 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.160606 11624 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:16:50.205973 11470 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.204164 11628 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.196555 11629 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:50.284981 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89010.7us
I20260709 13:16:50.175926 11627 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.239517 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.303097 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:50.253257 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:50.340140 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87002.6us
I20260709 13:16:50.340989 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:50.250376 11626 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.342482 11626 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.326781 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.331372 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.347767 10131 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.297145 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:50.350087 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74966us
I20260709 13:16:50.333981 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.334803 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.336333 10127 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.351430 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.299391 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:50.353845 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:16:50.406802 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53047.6us
I20260709 13:16:50.408965 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:50.301169 11632 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:50.288655 11511 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.349221 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.392019 11496 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.293696 11608 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:50.412895 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60768.6us
I20260709 13:16:50.438544 10128 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:16:50.413432 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.295805 11650 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:50.388823 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:50.397181 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.338879 11620 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.339509 11629 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.423391 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.423103 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.451934 10130 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
W20260709 13:16:50.393658 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 156354us
I20260709 13:16:50.447355 10124 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.456086 11579 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.421561 11632 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:50.428561 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.420254 11628 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.476166 10129 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:50.476284 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 142416us
I20260709 13:16:50.443905 11656 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.438795 10122 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:50.429195 11416 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.475636 11629 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.485023 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:50.481719 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130429us
I20260709 13:16:50.442962 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:50.508088 11579 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.508535 11511 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.476413 11608 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.489662 11656 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:50.414743 11647 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:50.494028 11655 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:16:50.494828 11627 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:50.496109 11606 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.499490 11665 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:50.482477 11514 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.501428 11578 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.490974 11416 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:50.503501 11641 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.504594 10205 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:16:50.516750 11513 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.432583 11381 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.522825 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.493148 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.511013 11475 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.483325 11628 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:50.539265 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 325722us
I20260709 13:16:50.518849 11584 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.557690 10204 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:50.557823 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:50.545616 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55495.7us
I20260709 13:16:50.477553 11488 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.561374 11668 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:50.505740 10125 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:16:50.528295 11656 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.543879 11650 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:50.501433 11389 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.556290 11514 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:50.601209 11532 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.000s
I20260709 13:16:50.584619 10202 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.586448 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:50.624028 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.625715 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:50.578752 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56031.8us
W20260709 13:16:50.579912 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 405319us
W20260709 13:16:50.588686 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95619.4us
I20260709 13:16:50.568197 11416 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.593132 10056 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:16:50.593173 10124 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.593309 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.594967 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.654947 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.655721 10053 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.595703 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.596340 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:50.595505 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.659880 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.597676 10047 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.583638 11628 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.589905 11517 raft_consensus.cc:993] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:50.625344 11579 raft_consensus.cc:2764] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:50.599085 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.604209 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.605182 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.527742 11511 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 13:16:50.608040 11662 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:16:50.610371 11389 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.610973 10199 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.611168 11641 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.612378 11381 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.612903 11647 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.614146 11630 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.614401 11629 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.689247 11630 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.690294 10201 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:50.690548 10196 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:50.620234 11578 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.620151 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.694020 10054 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.599299 11668 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:50.620842 11514 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.671434 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:50.696507 11650 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:50.696743 10893 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:50.697126 11650 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.697332 11629 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.618305 11632 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:50.625990 11644 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.546s	user 0.005s	sys 0.009s
I20260709 13:16:50.601363 11667 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:50.671422 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:50.657987 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.661793 11479 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:50.713296 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 300953us
I20260709 13:16:50.713696 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.651364 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.663903 10197 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.603726 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.716328 10128 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.666785 10130 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:50.669399 11665 raft_consensus.cc:1081] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:16:50.670372 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.719911 10131 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:50.670964 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:50.721062 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50190.2us
I20260709 13:16:50.682119 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.621959 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.683776 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.676196 11657 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:16:50.619616 10048 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.735440 10899 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:50.735786 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.736315 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.737144 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.672274 11511 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:50.650638 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:50.694684 11516 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.703652 11389 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:16:50.748584 10054 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:50.675910 11544 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.752868 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.753955 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:50.708752 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.658691 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.705410 11644 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:50.755332 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 96720.9us
I20260709 13:16:50.671608 11672 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.714644 10124 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.756848 11672 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.757951 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.758782 10124 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.760169 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.760969 10197 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.762511 11601 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.756861 10203 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.763456 11504 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:50.764559 11504 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:50.764946 11504 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.715238 10053 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:50.666356 11470 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:50.715608 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64361us
I20260709 13:16:50.722189 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:50.779562 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57453us
I20260709 13:16:50.780208 10049 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.669798 11666 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:16:50.743661 11511 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:50.731164 11620 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.744572 10129 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.698592 11414 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:16:50.717227 11479 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:50.792289 11479 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:50.792507 11462 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:50.701159 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.720647 11496 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.655383 11626 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.756170 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.772976 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:50.774399 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65743.5us
I20260709 13:16:50.772420 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.793025 11462 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.786453 11514 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.796144 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:50.796991 10203 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:50.797654 11626 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.779450 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.794194 11630 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:16:50.798715 10126 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.799638 11672 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.800094 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.800463 11470 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:50.801402 11496 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:50.801887 11496 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.802090 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:50.752000 11613 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.004s
I20260709 13:16:50.780731 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:50.723839 10127 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:50.782812 11513 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:50.781023 11631 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:50.812965 11631 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:50.813678 11631 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.815392 10127 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:50.816900 10127 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:50.730458 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 126390us
I20260709 13:16:50.782754 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:50.820261 11638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806f35e66c234d569cbafd8b3981518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:50.821141 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.823756 11620 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.824612 11578 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.825457 10127 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:50.785710 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:50.826913 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.827860 10130 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:50.763028 11601 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.829735 11601 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:50.832312 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49602.2us
I20260709 13:16:50.832978 11578 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:50.833743 11578 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:50.834846 11578 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.836560 11632 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.839519 11511 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.841225 11511 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:50.765108 11644 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:50.791234 10124 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:50.843987 11513 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.845569 11513 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.803093 11613 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.851046 11579 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:50.802452 11654 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.358s	user 0.003s	sys 0.000s
I20260709 13:16:50.796838 10129 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:50.724380 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:50.860608 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 136285us
I20260709 13:16:50.802343 11496 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:50.797991 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:50.804895 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48819.6us
I20260709 13:16:50.808020 11656 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.810343 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.786352 11627 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.823750 11626 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:16:50.824245 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43640us
I20260709 13:16:50.869652 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.870990 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.872738 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:50.864070 10128 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:50.864223 10019 leader_election.cc:400] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3.
I20260709 13:16:50.834365 11679 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:50.783278 10053 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.865792 10049 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:50.797597 11514 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:50.874756 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.877435 10049 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:50.880309 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 70048.4us
I20260709 13:16:50.885470 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:50.887472 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:50.888972 11667 raft_consensus.cc:2764] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:16:50.875104 11672 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.875912 11407 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:50.888610 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67648.2us
I20260709 13:16:50.889664 11480 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.889990 11641 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.765956 11504 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.900604 10130 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:50.900316 11665 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:50.904826 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.787310 11628 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.874639 11678 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:50.868646 11608 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:50.891396 11381 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:50.980134 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93222.6us
I20260709 13:16:50.885456 11666 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:16:50.904338 11488 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.991798 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73140.9us
I20260709 13:16:50.912969 11627 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:51.004132 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99391.5us
I20260709 13:16:50.838522 11668 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:50.982026 11514 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:51.004225 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 131526us
I20260709 13:16:50.890097 11606 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:16:51.017246 11480 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:51.002224 11389 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:16:51.004890 11381 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.977312 11641 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:51.006611 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:50.920514 11679 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:50.920733 10197 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:51.032279 10054 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:51.035851 10053 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:50.921554 11630 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:16:51.040805 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:51.008077 11470 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:50.991146 11624 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:16:51.050386 10198 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:51.004956 11647 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:16:51.038077 11488 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:50.989228 11667 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:16:51.026276 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:50.999186 11629 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:16:51.095335 10894 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:51.096094 11629 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:51.096594 11629 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:51.097493 11629 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.098748 11629 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:51.042840 11681 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.002s
I20260709 13:16:51.076262 11606 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:16:51.063062 10053 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:51.100273 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:51.060710 11644 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:51.065235 11626 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:51.067647 11627 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 13:16:51.053352 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46857.7us
I20260709 13:16:51.108417 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.055557 11480 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:51.109948 11480 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:51.110944 11480 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.112392 11480 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.113263 11665 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:51.113824 11665 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.077854 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:51.066859 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 161749us
I20260709 13:16:51.090593 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:51.045791 11381 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.053963 11641 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.128631 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.131680 11414 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:51.132718 10123 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:51.137882 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.076565 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:51.139987 11414 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:51.140560 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78283.9us
I20260709 13:16:51.140815 11414 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:51.141649 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:51.141816 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:51.142320 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:51.118870 11608 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:51.143362 11608 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:51.143561 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:51.143396 11414 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.140393 11683 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:51.144384 11608 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.145327 11475 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:51.145565 11683 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:51.145779 11475 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.144747 11414 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:51.148093 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.104470 11644 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:51.133584 10125 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.002s
I20260709 13:16:51.146332 11475 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:51.152645 10123 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:51.152170 11679 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.107390 11620 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:51.154904 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:51.165810 10124 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:51.166723 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.159680 10893 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:51.169412 11647 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:51.170208 11647 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:51.167289 11514 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:51.171056 11514 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.154536 11414 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.159812 10128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45970
I20260709 13:16:51.145990 11683 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:51.176714 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.161826 10127 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:51.161767 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:51.163605 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:51.164202 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:51.158162 11480 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.153802 10892 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:51.145869 11608 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.156498 11644 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:51.148139 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 145193us
I20260709 13:16:51.130038 11630 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:16:51.175386 10128 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:51.205533 10128 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.129266 11650 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.159035 11656 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.196317 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:51.213069 11544 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:51.213500 11544 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:51.213879 11544 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.199745 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:51.165735 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:51.191025 11683 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.223484 11578 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:51.224551 11578 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:51.176331 11624 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.245065 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:51.165351 11627 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:16:51.248894 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:51.200969 11488 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.205173 11511 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.254213 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.273156 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "db052bef7536453fa2d7f5a00e314a18"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:16:51.273801 10202 raft_consensus.cc:606] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Received request to transfer leadership to TS db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:51.273898 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.176355 11601 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.291218 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.202086 11692 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.298101 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:51.300650 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.195955 11496 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:51.307614 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.255182 11628 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.252311 11578 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.202701 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:51.317042 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:51.318228 10900 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:51.330838 10049 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:16:51.195560 11647 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.210614 11470 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260709 13:16:51.303247 11606 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:51.283847 11511 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.210022 11480 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:16:51.271968 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:51.328670 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.339783  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:51.341964 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.255720 11631 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.351536 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:51.371202 11606 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:51.352957 10127 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:51.272773 11544 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.319907 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:51.387068 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67103.6us
I20260709 13:16:51.367095  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:51.388479 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.368467 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.320979 11517 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:16:51.361721 11511 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.339723 11667 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:16:51.393150 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.360219 11578 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:51.391196 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 57919.4us
I20260709 13:16:51.391402 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:51.389755 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet a0c93dea5060480eb0618f168078f4e6: Illegal state: Replica e9a849ac7eda426abfca97cf8b15da0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:51.406956 11514 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.425609 10128 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.430753 10128 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:51.437281 11697 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:16:51.385612 11620 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.373059 11692 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:16:51.443926 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.366225 11698 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:51.385768 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 194119us
I20260709 13:16:51.394061 11470 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:51.402472 11628 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:51.439035 11416 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.779s	user 0.008s	sys 0.004s
W20260709 13:16:51.481704 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164773us
I20260709 13:16:51.440081 11601 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.441061 11496 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.440623 11650 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:51.412696 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 160946us
I20260709 13:16:51.454301 11624 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:51.657021 11694 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:16:51.680718 11381 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.004s
I20260709 13:16:51.682453 11638 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:60866 (request call id 304) took 2397 ms. Trace:
I20260709 13:16:51.683390 11579 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:51.683007 11680 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:51.684691 11680 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:51.683916 11579 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.680513 11691 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:51.684986 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 320) took 1170 ms. Trace:
I20260709 13:16:51.683995 11544 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.485335 11416 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:51.727741 11699 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 13:16:51.728297 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:51.747844 11644 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.004s
I20260709 13:16:51.485091 11606 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.756722 11680 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:51.712113 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:51.761128 11475 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
W20260709 13:16:51.760978 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58460.5us
I20260709 13:16:51.714452 11631 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:51.686301 11683 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:51.753974 11561 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:51.769096 11475 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.793205 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.793497 11683 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:51.761937 11698 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:51.682554 11381 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.779347 11680 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:51.769778 11488 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:51.750988 11644 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.683035 11638 rpcz_store.cc:276] 0709 13:16:49.284715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.287272 (+  2557us) service_pool.cc:224] Handling call
0709 13:16:49.729037 (+441765us) ts_tablet_manager.cc:934] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
0709 13:16:50.818523 (+1089486us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:51.420734 (+602211us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:51.420887 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:51.681164 (+260277us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:51.681747 (+   583us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:51.682222 (+   475us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:51.682328 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":375,"tablet-copy.queue_time_us":301038,"thread_start_us":162923,"threads_started":2}
I20260709 13:16:51.787452 11561 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:51.795809 11475 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:51.794552 11683 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:51.835727 11416 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:51.837276 11641 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.000s
I20260709 13:16:51.839643 11641 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.843070 11641 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:51.845006 11561 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:51.847506 11561 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:51.797961 11698 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:51.854660 11638 tablet_copy_client.cc:806] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:51.861491 11698 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:51.888613 11704 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:16:51.883831 11644 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:51.899052 11620 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:51.892009 11638 tablet_copy_client.cc:670] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
W20260709 13:16:51.908520 11665 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.794s	user 0.016s	sys 0.000s
I20260709 13:16:51.909243 11665 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:51.917656 11665 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:51.971967 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 580380us
I20260709 13:16:51.974378 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.439694 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:51.980643 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:51.981689 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.984359 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:51.985831 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 501273us
I20260709 13:16:51.987757 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:51.990578 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.987799 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.991745 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:51.990465 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:52.009464 10053 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.020181 10053 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.026783 10053 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:52.056675 10899 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.030220 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 871) took 1176 ms. Trace:
I20260709 13:16:52.059405 10049 rpcz_store.cc:276] 0709 13:16:50.853347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.887784 (+ 34437us) service_pool.cc:224] Handling call
0709 13:16:50.889554 (+  1770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:51.331664 (+442110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.029207 (+697543us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400198f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.029931 (+   724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007084928}
I20260709 13:16:52.031701 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 37) took 1965 ms. Trace:
I20260709 13:16:52.061332 10122 rpcz_store.cc:276] 0709 13:16:50.065644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.067493 (+  1849us) service_pool.cc:224] Handling call
0709 13:16:52.024015 (+1956522us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:52.030906 (+  6891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784511872}
I20260709 13:16:52.066053 10056 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:52.040580 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.007s	sys 0.006s
W20260709 13:16:52.074416 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.007s	sys 0.006s
I20260709 13:16:52.074649 11381 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:52.075249 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 901968us
I20260709 13:16:52.037516 10047 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.080446 11389 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:16:52.082388 10047 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.022264 11718 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.083595 11407 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.083945 11718 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.084314 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.042239 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2317) took 2600 ms. Trace:
I20260709 13:16:52.042337 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1229) took 1829 ms. Trace:
I20260709 13:16:52.042342 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1230) took 1828 ms. Trace:
I20260709 13:16:52.091073 10201 rpcz_store.cc:276] 0709 13:16:50.213377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.337318 (+123941us) service_pool.cc:224] Handling call
0709 13:16:50.337697 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.690886 (+353189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.041544 (+1350658us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540012a554. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.042086 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810141056}
I20260709 13:16:52.086697 10199 rpcz_store.cc:276] 0709 13:16:49.441659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.441773 (+   114us) service_pool.cc:224] Handling call
0709 13:16:49.442083 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.677451 (+1235368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.041396 (+1363945us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.041898 (+   502us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687615232}
I20260709 13:16:52.086818 10196 rpcz_store.cc:276] 0709 13:16:50.213011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.307819 (+ 94808us) service_pool.cc:224] Handling call
0709 13:16:50.308241 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.691579 (+383338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.041419 (+1349840us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400217ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.042064 (+   645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878197632}
I20260709 13:16:52.084268 11407 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:52.097246 11407 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.098206 11416 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:52.098598 11479 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.305s	user 0.012s	sys 0.008s
W20260709 13:16:52.044402 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 256591us
I20260709 13:16:51.708387 10895 rpcz_store.cc:276] 0709 13:16:50.514496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.519304 (+  4808us) service_pool.cc:224] Handling call
0709 13:16:51.404593 (+885289us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:51.454703 (+ 50110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:51.644530 (+189827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:51.647466 (+  2936us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:51.670144 (+ 22678us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:51.684513 (+ 14369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.684657 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.684851 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10407808}
I20260709 13:16:52.054049 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 870) took 1283 ms. Trace:
I20260709 13:16:52.055958 10051 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.034245 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 843) took 1553 ms. Trace:
I20260709 13:16:52.099831 11496 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:52.099653 11470 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.107309 10054 rpcz_store.cc:276] 0709 13:16:50.769885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.772648 (+  2763us) service_pool.cc:224] Handling call
0709 13:16:52.053247 (+1280599us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400391054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:52.053691 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544934656}
I20260709 13:16:52.109061 11470 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.100224 11407 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:52.118120 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95033us
I20260709 13:16:52.123862 11606 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.125252 11606 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.123996 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.134456 11416 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:16:52.139952 10125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.000s	sys 0.013s
I20260709 13:16:52.140502 11665 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:52.140547 10125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.000s	sys 0.013s
I20260709 13:16:52.145392 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 882) took 1653 ms. Trace:
I20260709 13:16:52.147303 11692 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.148386 11692 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.138641 11416 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.103422 11479 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.130561 11698 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.162005 11687 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.164532 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.165390 11687 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:52.166208 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:52.146122 10125 rpcz_store.cc:276] 0709 13:16:50.491733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.504752 (+ 13019us) service_pool.cc:224] Handling call
0709 13:16:50.505290 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.505585 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:16:51.163632 (+658047us) log.cc:844] Serialized 14 byte log entry
0709 13:16:52.142356 (+978724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.142523 (+   167us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:16:52.142686 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.142824 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.145231 (+  2407us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11446,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":11927,"thread_start_us":934734,"threads_started":2,"wal-append.queue_time_us":191158}
I20260709 13:16:52.171167 11650 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.171584 10050 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:52.177762 10056 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:16:52.181749 11479 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.203418 11627 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:52.183478 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.204017 11627 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:52.172116 11650 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.172515 11470 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.184322 11620 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:52.107456 10055 rpcz_store.cc:276] 0709 13:16:50.480768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.481162 (+   394us) service_pool.cc:224] Handling call
0709 13:16:50.515856 (+ 34694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.516133 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:50.589963 (+ 73830us) log.cc:844] Serialized 14 byte log entry
0709 13:16:50.889063 (+299100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:50.889168 (+   105us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:50.889238 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.889332 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.033655 (+1144323us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b5400087350. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:52.033971 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136719,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":2026,"raft.queue_time_us":9702,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":9798,"spinlock_wait_cycles":68716800,"thread_start_us":352625,"threads_started":2,"wal-append.queue_time_us":392126,"wal-append.run_cpu_time_us":1837,"wal-append.run_wall_time_us":1150603}
W20260709 13:16:52.187301 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.025s	sys 0.005s
I20260709 13:16:52.188869 10122 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:52.209224 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.228924 10202 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.229033 11692 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.210304 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.211479 10204 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.232882 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.212834 10203 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:52.195426 11578 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.198557 10126 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:16:52.215298 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.001s	sys 0.034s
I20260709 13:16:52.200068 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:52.216426 11414 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.018s	user 0.015s	sys 0.008s
I20260709 13:16:52.202513 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:52.207453 10201 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:52.208535 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.193192 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:52.235206 10125 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:16:52.236212 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.001s	sys 0.034s
I20260709 13:16:52.236003 11578 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.178668 10056 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
W20260709 13:16:52.241400 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.026s	sys 0.006s
I20260709 13:16:52.223477 11723 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
W20260709 13:16:52.204188 11608 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.008s	user 0.016s	sys 0.001s
I20260709 13:16:52.253322 10056 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:16:52.261270 10128 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:16:52.246718 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet 2219e1af873d4a00966a8c5975eb7987: Illegal state: Replica 01bfaf1ad56a4deea886bbc64dec466c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:52.263290 11641 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.264201 11641 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.267831 11620 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:52.268527 11620 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.222865 11606 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.207326 11650 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.259874 10056 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:52.238204 11687 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.266898 11641 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.269008 11620 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:52.271360 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 874812us
I20260709 13:16:52.274641 11416 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:16:52.281646 11416 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.281838 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 854) took 1673 ms. Trace:
I20260709 13:16:52.282255 11416 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.238729 11414 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.268865 11488 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:16:52.271829 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78886.6us
I20260709 13:16:52.282555 10050 rpcz_store.cc:276] 0709 13:16:50.608123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.738176 (+130053us) service_pool.cc:224] Handling call
0709 13:16:50.738513 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.172253 (+1433740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.281364 (+109111us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540010b8d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.281735 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3286572416}
I20260709 13:16:52.295670 10199 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:52.298486 10202 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:16:52.304529 11663 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.744s	user 0.023s	sys 0.040s
I20260709 13:16:52.305558 10895 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:52.307273 11381 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.307752 10201 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.308475 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:52.309332 10047 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.205585 11627 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.308187 11381 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.289746 11606 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.328888 10201 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.328572 11719 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:52.332345 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 876) took 1023 ms. Trace:
I20260709 13:16:52.333479 11719 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.334159 10131 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:52.339694 11688 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:52.340287 11688 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:52.324672 10200 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.323315 11698 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:52.328595 11718 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.325067 11470 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.345499 10202 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.330885 11627 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:52.333818 10053 rpcz_store.cc:276] 0709 13:16:51.309651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.278326 (+968675us) service_pool.cc:224] Handling call
0709 13:16:52.327371 (+ 49045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.331875 (+  4504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.331907 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.331978 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:52.332016 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.332093 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.332272 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:52.322480 11687 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:52.345742 11579 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.011s
I20260709 13:16:52.358853  9954 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.367501 11416 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.370395 11416 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:52.407814 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet 2188d855354542c984f249b94175c660: Illegal state: Replica e9a849ac7eda426abfca97cf8b15da0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:52.381376 11407 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:52.339735 11606 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.381232 11626 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:16:52.378675 11511 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.937s	user 0.015s	sys 0.006s
I20260709 13:16:52.436327 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.408056 11470 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.390949  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:52.446882 11407 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.347210 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:52.412092 11381 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.394366 11672 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.382884 11690 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:52.473740 11690 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.474958 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.475483 11690 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.477479 11690 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.490628 11705 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:52.491385 11705 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:52.492949 11705 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.496109 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 877) took 1186 ms. Trace:
I20260709 13:16:52.496802 10049 rpcz_store.cc:276] 0709 13:16:51.310037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.070267 (+760230us) service_pool.cc:224] Handling call
0709 13:16:52.153654 (+ 83387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.256269 (+102615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.256324 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.268067 (+ 11743us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.324332 (+ 56265us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:52.324385 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.324508 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:52.495775 (+171267us) raft_consensus.cc:1685] finished
0709 13:16:52.495807 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.496027 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73853,"raft.queue_time_us":74023,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":25123,"thread_start_us":51871,"threads_started":1,"wal-append.queue_time_us":148083}
I20260709 13:16:52.414510 11584 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:52.397980 11668 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:52.499321 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 205064us
I20260709 13:16:52.500914 11689 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:52.501312 11666 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:52.396247 11613 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.502799 11718 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:52.503419 11718 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.503680 11719 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.415733 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 875) took 1110 ms. Trace:
I20260709 13:16:52.505050 11718 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.509161 11666 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:52.511698 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:52.403280 11684 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.346289 11578 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.512531 10895 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.512467 11684 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.504518 10052 rpcz_store.cc:276] 0709 13:16:51.305319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.414151 (+1108832us) service_pool.cc:224] Handling call
0709 13:16:52.414679 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.415023 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.415064 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:52.415177 (+   113us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:52.415225 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.415304 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:52.415488 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:52.514276 11684 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.498477 11688 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.516358 11688 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.516631 11684 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.519266 11584 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:52.519392 11578 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:52.520784 11719 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:52.521499 11666 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.522289 11666 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.520867 11584 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.521761 11719 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.523523 11666 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.524299 11584 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.450198 11678 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.446239 11511 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.458423 11407 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.459301 11462 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.527209 11462 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.527312 10129 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.526450 11678 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.530653 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2358) took 1420 ms. Trace:
I20260709 13:16:52.462777 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1205) took 1202 ms. Trace:
I20260709 13:16:52.534117 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:52.535394 11407 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.536114 11698 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:52.536343 11407 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.534525 11511 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.535413 10051 rpcz_store.cc:276] 0709 13:16:51.260521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.029021 (+768500us) service_pool.cc:224] Handling call
0709 13:16:52.055276 (+ 26255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.107461 (+ 52185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.462502 (+355041us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b5400391050. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:52.462668 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58492672}
I20260709 13:16:52.534996 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:52.467432 10202 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:52.496011 11514 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:52.541719 11514 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:52.542196 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.620s (SENT) [suppressed 23 similar messages]
I20260709 13:16:52.542524 10129 rpcz_store.cc:276] 0709 13:16:51.130682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.131133 (+   451us) service_pool.cc:224] Handling call
0709 13:16:52.024416 (+893283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:52.528236 (+503820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.529197 (+   961us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:52.530244 (+  1047us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400321c50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:52.530527 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3180505344}
I20260709 13:16:52.543444 11514 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.545435 11514 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.511842 10894 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.441781 11626 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:52.525197 11613 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.533376 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:52.548270 11626 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.549939 11626 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.557155 11705 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.559520 11513 raft_consensus.cc:993] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: : Instructing follower db052bef7536453fa2d7f5a00e314a18 to start an election
I20260709 13:16:52.560423 11513 raft_consensus.cc:1081] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Signalling peer db052bef7536453fa2d7f5a00e314a18 to start an election
I20260709 13:16:52.327711 11692 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:16:52.376135 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.562645 11511 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.563241 11511 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:52.388095 11504 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:52.563968 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:52.461695 11561 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.005s
I20260709 13:16:52.564721 11511 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:52.579725 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.579909 10128 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:52.580713 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:52.587066 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.588409 10129 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.588908 10047 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:52.536839 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:52.565771 11414 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:52.434907 11579 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:52.605697 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.608170 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.436442 11687 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.581980 11692 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.580821 11407 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.612593 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.620152 11661 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.620935 11661 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.627863 11511 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.634704 11723 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:52.635689 11723 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.636425 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:52.637946 11698 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:52.638532 11698 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:52.639463 10895 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399"
dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
 from {username='slave'} at 127.0.0.1:60866
I20260709 13:16:52.639693 11698 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.640380 10895 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:52.640525 11579 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.641693 10126 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:52.618713 11672 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.640822 10895 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:52.648423 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.664651 10201 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:52.668447 10128 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:52.678107 10200 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.679905 11719 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.680124 11672 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.615595 11692 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:52.683290 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:52.684681 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.686573 11718 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.687675 11678 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.687904 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:52.689764 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:52.691628 10203 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:16:52.691830 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.693549 11517 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:52.694356 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:52.695052 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 183597us
W20260709 13:16:52.695956 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.705701 10129 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:52.712339 10201 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:16:52.648854 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 319103us
I20260709 13:16:52.716892 10897 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.734375 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.737030 10201 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:52.715847 10125 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.738809 10056 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:52.666129 11620 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.749339 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:52.755674 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66105.4us
I20260709 13:16:52.758399 11723 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.758781 10200 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.759601 10052 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:52.748881 11687 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:52.758914 11701 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:16:52.762465 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:52.762810 10202 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.764022 10056 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:52.765288 11678 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.766088 11692 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.767314 11702 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:52.771731 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:52.773155 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:52.774672 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:52.777611 11620 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.777992 10201 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:52.739291 11666 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:52.779292 11514 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.780194 11514 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.780920 11514 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:52.782404 11689 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:52.782814 11689 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.732793 11661 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:52.783979 11689 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.786598 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:52.787717 11723 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:52.789037 10900 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.789541 11678 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:52.790066 11678 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.791143 11692 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.791487 11606 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:52.791983 11692 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.792696 11470 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.792217 11678 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.735440 11517 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:52.796684 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:52.803835 10129 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:52.805536 11692 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:52.806113 11678 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:52.798601 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:52.825922 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:52.836825 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80273.3us
I20260709 13:16:52.841331 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:52.850695 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.910115 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.910822 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.911556 10196 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:52.912030 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.912120 10202 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:16:52.925601 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:52.926066 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 383912us
I20260709 13:16:52.920387 10202 pending_rounds.cc:85] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 1
I20260709 13:16:52.927078 10202 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:52.927484 10202 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:52.927819 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74210.9us
I20260709 13:16:52.805446 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:52.933599 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79154.7us
I20260709 13:16:52.825080 10895 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.941929 10203 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:52.944300 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.945461 10198 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:52.941844 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.947964 10896 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:52.948108 10197 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:52.949681 11517 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:52.955407 10201 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:52.960448 11672 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.961364 11672 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:52.918200 10897 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:52.923353 11684 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.842154 11470 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:16:52.962957 11488 raft_consensus.cc:2764] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:16:52.976897 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:52.982043 10203 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:16:52.983896 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:52.967206 11416 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.004s
I20260709 13:16:52.988302 11513 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.988969 11632 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.989527 11561 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.991241 11671 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:16:52.995476 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54389.4us
W20260709 13:16:53.002673 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.970782 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:53.016785 11668 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.518s	user 0.025s	sys 0.003s
I20260709 13:16:53.017370 10894 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:16:53.020186 11488 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.021296 11488 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:53.030292 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.980175 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.036300 10200 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:53.036755 11672 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.037560 11657 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:53.037659 11414 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:53.038288 11657 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.011301 11613 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.040829 11657 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.041838 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.043133 10203 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:16:53.046464 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.047230 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.047544 11705 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:16:53.052644 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.040597 11488 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.067025 11584 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.069133 11584 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.072856 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.073480 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.073100 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.075449 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.075788 11705 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.076388 11705 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:16:53.077010 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.067189 11671 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:16:53.078042 11671 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.079526 11671 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.080648 10895 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.079684 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.081496 11671 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:53.083492 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.083897 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.084009 10197 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:53.084560 10201 raft_consensus.cc:2368] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:53.084980 11511 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.085740 11511 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:53.088285 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.095983 11672 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:53.096766 11672 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:53.099436 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.101306 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.102001 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.112844 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.102866 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.103570 11672 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:53.103947 11732 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.190s	user 0.003s	sys 0.000s
I20260709 13:16:53.047277 11561 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.105511 11626 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:53.107311 11692 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:53.109059 11666 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:53.073634 11496 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.006524 11684 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.105355 11689 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:53.122014 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.023248 11513 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.123175 10198 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.029078 11632 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:53.118306 11504 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.506s	user 0.003s	sys 0.008s
I20260709 13:16:53.144960 11661 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.131882 11626 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.145530 11632 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.145799 11626 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.132807 11561 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.148558 11626 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.150123 11626 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.151290 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.152117 10129 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:16:53.145555 11661 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.112943 10199 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:53.126695 11666 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.129343 11496 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.143947 11584 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.139952 10125 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
W20260709 13:16:53.175462 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.187817 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.188228 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.130237 11672 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.189074 10128 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:53.131704 11511 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.142272 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:53.205554 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 207669us
I20260709 13:16:53.213783 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.214679 10131 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.224617 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.225797 10202 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:16:53.217559 11718 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.005s
I20260709 13:16:53.241230 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.243762 10203 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 4. Ops: []
I20260709 13:16:53.245663 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.246587 10198 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.242564 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.252852 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.256809 10204 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:53.257789 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.258535 10202 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:53.226845 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.168604 11705 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.204417 11672 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.228704 11584 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.262079 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.225948 11678 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:16:53.265894 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 207422us
I20260709 13:16:53.240530 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.263909 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.274603 10202 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:53.279728 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.283382 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.286077 10203 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:53.289498 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.256088 11511 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:16:53.279533 11705 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.324138 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.299829 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.300259 11684 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.301584 11511 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:53.282308 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.303824 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.304845 11688 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:53.339540 10205 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:16:53.339740 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68091us
I20260709 13:16:53.340435 11688 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.340952 11688 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.341637 11613 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:53.342308 11613 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.341747 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.342818 11613 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.305583 11718 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.313225 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:53.290621 11606 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:16:53.316753 10894 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:53.317741 10198 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:16:53.320557 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.363170 11656 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.256525 11666 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.368188 11656 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.335294 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.345602 11470 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:53.369503 11470 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:53.370074 11470 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.371654 11470 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.373713 11470 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.343392 11678 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.375753 11678 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.378130 11678 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.306043 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.379643 11666 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.269220 11496 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.380707 11496 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.277516 11584 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.384187 11656 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:53.351704 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 393405us
I20260709 13:16:53.350036 11718 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.392339 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:53.353327 11661 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.342106 11688 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.394037 10897 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:53.394729 11688 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.354518 11517 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:53.396452 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:53.351280 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.350992 11668 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:16:53.360263 11606 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:53.399559 11672 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.399869 11606 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.360996 11692 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.402228 11692 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.402390 11606 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.403818 11692 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.364365 11513 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.405995 11692 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.330535 11511 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.365257 10048 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
W20260709 13:16:53.329288 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44708.3us
I20260709 13:16:53.384917 11584 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.392307 11718 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.353660 11723 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:53.400071 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:53.356729 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53035.9us
I20260709 13:16:53.400965 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:53.352705 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 419063us
I20260709 13:16:53.401374 10202 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:16:53.408856 11511 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.410378 11517 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:16:53.410774 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.416172 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.416579 10199 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:16:53.417167 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67467us
W20260709 13:16:53.423372 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.411082 11479 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.432413 10900 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:53.435925 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.438609 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.436393 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.446933 11488 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:53.447530 11488 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:53.441941 11626 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.437139 11661 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.449195 11414 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:53.437404 11511 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.436213 11684 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.441941 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.416946 11656 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.441228 10197 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:53.441066 11479 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.415719 11517 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.445443 11698 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:53.435925 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.448678 11488 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.438300 11638 tablet_copy_client.cc:538] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:16:53.457098 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.461819 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.470237 11666 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.471371 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.467070 10197 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.473240 10122 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:53.449851 11626 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.415907 11584 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:16:53.473490 11723 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:53.473263 11479 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.478830 11718 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.503736 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:53.498745 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.505479 10051 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:16:53.505707 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1198) took 2248 ms (client timeout 2025 ms). Trace:
W20260709 13:16:53.506352 10048 rpcz_store.cc:269] 0709 13:16:51.257435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.439338 (+181903us) service_pool.cc:224] Handling call
0709 13:16:53.505390 (+2066052us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002e7f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:53.505603 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386313088}
I20260709 13:16:53.507895 10050 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:53.508217 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:53.489032 11723 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.509166 11723 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.486209 11690 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.511318 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.511415 10197 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.512844 11684 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:53.513935 11684 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.515316 11626 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.515412 11630 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:53.491990 11687 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.517059 11687 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.518235 10196 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:16:53.517481 11517 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.491989 11416 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:53.485932 11666 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.501758 11748 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.521256 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.535465 11416 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.536116 11416 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.509196 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.536942 10052 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:53.539394 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.536289 11666 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.541706 11666 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.513458 10197 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.522802 11532 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:53.543635 11532 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:53.544109 11532 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.525070 11684 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.511597 11690 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.536209 10051 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:53.528515 11613 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.529906 10204 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:16:53.533264 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.576802 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58443.3us
W20260709 13:16:53.578495 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.552417 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.579720 10054 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:53.580722 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.581535 11462 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.553182 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.582490 10049 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:53.583007 11630 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.553519 11504 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:53.584177 11504 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.582281 11462 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.585884 11613 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.586401 11462 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:53.591835 11517 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.553565 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.593730 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.554248 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.595026 10048 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:53.554728 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.556068 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.557395 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.558820 10051 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:53.517504 11687 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:53.560489 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.600922 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.601365 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.581979 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.561120 11684 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:53.561321 11678 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:53.562862 11690 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.566099 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:53.561350 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.570086 10197 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:53.575647 11740 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:53.575770 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77110.5us
I20260709 13:16:53.583752 11630 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:16:53.596007 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84787.4us
I20260709 13:16:53.592113 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.627494 10049 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.605873 10047 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:53.606199 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.606518 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.649381 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.649909 11638 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:16:53.650100 10051 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.606791 11626 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.651850 11626 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:53.602185 10054 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:53.585306 11504 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.609664 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.607534 11678 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.654350 10129 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:53.607741 11656 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:53.610432 11672 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:53.655567 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89557.8us
W20260709 13:16:53.656519 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 152869us
I20260709 13:16:53.610453 11684 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.654313 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.661363 10127 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:53.661924 10054 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.655915 11672 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.612147 11532 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.661389 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.666416 10051 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:53.620922 10050 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:53.640311 11671 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:53.640797 11630 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.643739 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.644312 11723 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.644974 10095 leader_election.cc:400] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:53.671375 10095 leader_election.cc:403] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:16:53.645612 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.671566 10049 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
W20260709 13:16:53.672305 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.646442  9954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 112) took 1434 ms. Trace:
I20260709 13:16:53.673827 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.663025 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.676221 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.677196 10051 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:53.678582 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:53.679016 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 217304us
I20260709 13:16:53.679416 10049 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.679982 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.682399 10049 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:53.681831 11416 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:53.664388 11672 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.684508 11416 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.685191 11723 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.685566 10055 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:53.686275 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.687016 10047 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.687232 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.688046 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.688971 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.690588 10052 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:53.655606 11656 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.664610 11740 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:16:53.720070 11656 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.720714 11740 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.691570 11671 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:53.691828 10054 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:53.692523 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.722532 10056 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:53.673193  9954 rpcz_store.cc:276] 0709 13:16:52.212039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.212432 (+   393us) service_pool.cc:224] Handling call
0709 13:16:53.646340 (+1433908us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:52.955803 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:52.956646 (+   843us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:52.956721 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:16:52.956758 (+    37us) tablet.cc:662] Decoding operations
0709 13:16:52.957216 (+   458us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:52.957340 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:52.957371 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:16:52.957920 (+   549us) tablet.cc:701] Row locks acquired
0709 13:16:52.957991 (+    71us) write_op.cc:260] PREPARE: finished
0709 13:16:52.958247 (+   256us) write_op.cc:270] Start()
0709 13:16:52.958513 (+   266us) write_op.cc:276] Timestamp: P: 1783603012958166 usec, L: 0
0709 13:16:52.958564 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:16:52.986613 (+ 28049us) log.cc:844] Serialized 533 byte log entry
0709 13:16:53.088807 (+102194us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:16:53.613399 (+524592us) op_driver.cc:464] REPLICATION: finished
0709 13:16:53.630459 (+ 17060us) write_op.cc:301] APPLY: starting
0709 13:16:53.630611 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:53.630792 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:53.630847 (+    55us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:53.631405 (+   558us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:53.631795 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:16:53.631889 (+    94us) write_op.cc:312] APPLY: finished
0709 13:16:53.632349 (+   460us) log.cc:844] Serialized 23 byte log entry
0709 13:16:53.632618 (+   269us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:53.632754 (+   136us) write_op.cc:454] Released schema lock
0709 13:16:53.632836 (+    82us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":16685,"num_ops":1,"prepare.queue_time_us":386669,"prepare.run_cpu_time_us":59916,"prepare.run_wall_time_us":312873,"raft.queue_time_us":250887,"raft.run_cpu_time_us":9813,"raft.run_wall_time_us":16912,"replication_time_us":654639,"spinlock_wait_cycles":38266624,"thread_start_us":654439,"threads_started":4,"wal-append.queue_time_us":372874}]]}
I20260709 13:16:53.693694 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.693501 11678 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.721010 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.692935 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:53.693370 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48666.8us
I20260709 13:16:53.664691 11532 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.671891 11690 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:53.663021 11684 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.654361 11504 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.665241 11496 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.706446 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:53.707012 11688 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Leader election lost for term 3. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:53.709949 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:53.708206 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.714951 10047 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.725725 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.726583 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.735998 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.737560 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:53.738576 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.727025 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47188.3us
I20260709 13:16:53.727895 11561 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:53.729580 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.729718 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.740662 11532 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.742473 10055 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.742980 10197 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:53.740464 11511 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.743728 11511 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.744203 11511 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.731423 11672 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:16:53.736752 11671 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:53.736929 11496 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.750064 11496 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:53.716370 11687 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:53.751747 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.752590 10203 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:53.752707 11687 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.743207 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.753588 11630 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.753886 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.754923 10196 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:53.753626 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.755741 11462 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.755872 10204 raft_consensus.cc:2376] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3. Re-sending same reply.
I20260709 13:16:53.756474 11462 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.756996 11462 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.757092 11561 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:53.757733 11561 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.721372 11740 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.763789 11584 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:16:53.740718 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.760674 11678 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:53.770262 11690 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:53.770958 11690 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:53.772114 11690 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.741655 10056 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:53.730804 10050 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.730918 11488 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:53.732791 11414 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:53.787338 11414 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.787709 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:53.788786 10897 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:53.788471 11414 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.789906 11414 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.790180 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:53.791054 10900 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:53.791486 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.792407 10200 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:53.793133 11740 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.794413 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.749908 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:53.795105 10202 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.794716 11740 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.775419 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:53.779767 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.785599 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.783089 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.750917 11496 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.741276 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.795723 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.804570 10205 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:53.805794 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.807472 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.795454 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.808874 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.809303 10203 raft_consensus.cc:2376] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3. Re-sending same reply.
I20260709 13:16:53.809520 10204 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:53.810395 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.792228 11702 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:53.811890 10053 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.811982 10201 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:53.804323 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.815024 10196 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:53.816040 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:53.821893 10049 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.822345 10202 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:53.826256 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:53.828101 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87788.2us
I20260709 13:16:53.829991 10201 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:53.831003 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:53.831900 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:53.833274 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 64734us
I20260709 13:16:53.824685 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.841974 10201 pending_rounds.cc:85] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 1
I20260709 13:16:53.853742 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.856098 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.856772 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:53.858403 10201 pending_rounds.cc:107] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:53.861954 10201 raft_consensus.cc:2889] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:53.862545 10205 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:53.863683 10131 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:53.854476 10129 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:53.856554 10204 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:53.844627 10051 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:53.868485 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.868950 10205 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:53.869370 10122 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.854511 11496 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.870568 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.871476 10126 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:53.871781 10055 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:53.872867 10128 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:16:53.851387 11749 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.855386 11672 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.797821 10048 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:16:53.855445 11698 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.874833 10048 pending_rounds.cc:85] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 1
I20260709 13:16:53.875092 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.795190 11671 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.811172 11702 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.879871 10047 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:53.876935 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:53.869383 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:53.877439 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52820.4us
I20260709 13:16:53.882414 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.884639 10125 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:53.885186 10127 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.802819 11488 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:53.885710  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:53.881938 10130 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:53.888250 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:53.892765 10049 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:53.880740 10048 pending_rounds.cc:107] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:16:53.899258 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:53.899654 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.900223 10897 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:53.900709 10123 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:53.901621 11513 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:53.857089 11584 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.883867 11687 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:53.899307 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:53.875551 11718 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.874524 11678 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.910663 10900 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:53.802623 11738 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 13:16:53.918032 10050 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:53.917459  9953 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:53.918483 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.919628 10050 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:16:53.919796 10124 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:53.920038 11748 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.920653 11748 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.921173 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.921857 10124 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:53.923182 11748 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.923403 11749 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.924083 11749 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.919801  9953 catalog_manager.cc:5697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.925787  9953 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:16:53.926529 11749 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.878371 10052 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2. Re-sending same reply.
I20260709 13:16:53.906152 11584 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.928059 11584 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:53.930572 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.884433 11702 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:53.931087 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:53.931016  9953 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:53.933954 11630 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:53.934497 11630 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.880261 10197 raft_consensus.cc:2376] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3. Re-sending same reply.
I20260709 13:16:53.935197 11630 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:53.893155 11626 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.886060 11705 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:53.922880 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.893514 11688 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.891556 11692 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.958968 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:53.899071 10048 raft_consensus.cc:2889] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:53.951840 11606 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:53.952659 11758 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:53.944892 11687 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:53.956473 11656 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.865051 11657 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:16:53.874579 11672 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.883893 11561 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:53.969034 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:53.964453 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.938767  9953 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:53.972908 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67179.9us
I20260709 13:16:53.973306 10127 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:53.990160 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:53.976156 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:16:53.977602 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 72762.7us
I20260709 13:16:53.875149 11698 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:53.969938 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:53.980542 11671 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:53.999341 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.982728 10053 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:53.983338 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.984263 11488 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.003928 11561 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.003638 11511 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.004220 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.005023 10130 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:54.004912 11760 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:54.008303 11698 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.006317 11511 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:54.013909 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 131667us
I20260709 13:16:54.014379 10124 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:16:53.985997 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.015298 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:54.015717 10126 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:53.964612 11532 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.973488 11678 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.023358 11532 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.018900 11517 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.023900 11532 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.023587 11687 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.024153 11517 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.025517 11678 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:53.996495 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.028290 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.984805 11656 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:54.006348 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.962389 11688 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.016940 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.009510 11638 tablet_bootstrap.cc:700] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent opening tablet: real 0.153s	user 0.003s	sys 0.000s
I20260709 13:16:54.018321 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:53.993825 11414 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:16:53.978924 11513 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:54.020116 11749 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:53.979539 11738 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:54.028373 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.059266 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.029489 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:53.996954 11705 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:53.974009 11606 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:54.031466 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55379.2us
I20260709 13:16:53.939900 11462 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.045142 11684 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:53.994706 11723 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:53.964497 11751 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:16:53.964695 11718 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:54.040234 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50147.6us
I20260709 13:16:54.044437 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.006850 11760 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:16:54.045223 11761 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:54.080745 10129 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:54.045604 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:54.059522 10124 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:54.029158 10122 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:16:54.062412 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.031188 10202 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:16:54.068930 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.073079 10131 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:54.071659 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:54.045379 10126 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:54.073954 10127 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.081223 10128 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:54.075623 10203 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:54.103250 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:54.103535 10125 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:16:54.105858 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.131763 10126 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:54.139312 10125 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:16:54.141182 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.141868 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.133203 10050 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:16:54.142567 10130 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:54.139477 10127 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:54.143790 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.142014 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:54.075281 11723 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:54.149538 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.154109 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.075035 11606 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:54.156153 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.082422 11414 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:54.081739 11684 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.075014 11462 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.085959 11761 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.169478 10127 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:54.077613 11764 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:54.154968 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.155242 10126 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:54.050786 11513 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:54.180701 11765 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:16:54.166143 10128 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:16:54.195883 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.197046 10123 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:54.199018 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:54.201411 11740 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:54.201964 11740 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.202389 11740 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.204548 11630 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.209484 10128 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:54.210799 10126 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:54.218066 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.212468 11666 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:54.219149 11666 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.220621 11666 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.216797 11702 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.127285 11416 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.231395 10203 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:54.234231 10130 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:54.134274 11561 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:54.077394 11738 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.250214 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.252166 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.253110 10129 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.256053 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.257354 10130 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 2: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:54.157464 11723 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.162065 11414 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:54.226476 11764 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.225889 11761 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:54.153599 11749 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:16:54.292902 10129 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:54.285071 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.295437 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:54.303170 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.244498 11488 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:16:54.307509 11630 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.321238 11532 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.308581 11462 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:54.305004 11626 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.219421 11705 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.284982 11692 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:54.242976 11416 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.276904 11764 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.283052 11751 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:54.318519 11496 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:54.313774 11517 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:54.353279 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 265862us
I20260709 13:16:54.354053 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.355473 10124 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:54.332410 11511 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:54.362216 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:54.339886 11760 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.344177 11462 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:54.374986 11462 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.376730 11740 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.347919 11672 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:54.371843 11718 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:54.373059 11414 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.347714 11751 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:54.380884 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:54.394124 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.442601 10129 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:54.381631 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 320025us
W20260709 13:16:54.443805 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 57757.4us
I20260709 13:16:54.367277 11688 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:54.478176 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:54.449747 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.453545 11769 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.453541 11718 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:16:54.454638 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73839.3us
I20260709 13:16:54.483673 10124 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:54.484071 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:54.423039 11690 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:54.460567 11626 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:54.354688 11684 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.467698 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.487323 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.395831 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:54.488196 10126 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.489333 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.490129 10130 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.470171 11416 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:54.493772 11416 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:54.494568 11416 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.384831 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:54.496069 11416 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:54.498450 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:54.472077 11740 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.499799 11626 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:54.500228 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:54.500286 11626 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.500972 11626 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.501245 11687 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:54.470340  9954 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:54.505517 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50009.5us
I20260709 13:16:54.501835 11687 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:54.510825 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.520431 10196 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:16:54.522325 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.441350 11678 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.523387 10204 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:54.525168 11698 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:16:54.525614 10196 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:54.506734  9954 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 5, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 5 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.513003 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:54.533411 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:54.534493 10198 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
W20260709 13:16:54.520056 11772 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.005s
I20260709 13:16:54.527568  9954 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 2, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.368232 11517 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.143s
I20260709 13:16:54.545233 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:54.546147 10050 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:54.547194 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:54.547829 10047 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.548355 10202 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:16:54.515867 11702 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.542042 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:54.575510 10205 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:54.541131 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.378789 11760 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.537978 11414 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:54.522301 11532 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:54.578002 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 274947us
I20260709 13:16:54.673307 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:54.562161 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.520134 11561 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:16:54.551900 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:54.576431 10129 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:54.621613 10047 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:16:54.564446 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64273.4us
W20260709 13:16:54.578831 11584 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.003s
I20260709 13:16:54.686210 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:54.701269 10129 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:54.694512 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:54.706070 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 222081us
I20260709 13:16:54.706914 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:54.707255 10095 leader_election.cc:400] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:54.579281 10050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:54.707634 10095 leader_election.cc:403] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:16:54.708124 10050 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:54.708658 10050 raft_consensus.cc:614] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:54.544287 11630 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.147s
W20260709 13:16:54.714161 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.609062 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.724756 10205 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:54.554839 11678 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.696115 11738 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:54.764279 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89119.8us
I20260709 13:16:54.765635 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.709467 11584 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.711079 11511 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:16:54.613893 11749 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:16:54.710390 10197 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:54.712705 11532 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:54.772064 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:54.537252 11769 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:54.700158 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 255647us
W20260709 13:16:54.779440 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72594.9us
I20260709 13:16:54.791312 10201 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:54.751976 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:54.781348 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.722074 11690 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.722456 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.726104 11692 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:16:54.789276 11532 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.778940 11657 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.782930 11511 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:54.819681 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.816416 10126 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:16:54.855942 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.789582 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:54.857642 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:54.858858 10050 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:54.867739 11690 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.869127 11751 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:54.816953 11517 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:54.871332 11657 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.874572 11657 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:54.876222 11678 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:54.827477 10055 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:16:54.829600 11768 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.464s	user 0.002s	sys 0.000s
I20260709 13:16:54.775599 11718 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.853495 11532 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:54.851938 11698 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.882241 11511 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:16:54.918275 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115910us
I20260709 13:16:54.919762 11666 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.914245 11517 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:54.922161 11517 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:54.773502 11687 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:54.933683 11775 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:54.941910 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:54.942968 10051 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:16:54.922191 11779 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:16:55.018559 10130 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:54.963268 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:55.007962 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:55.001418 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 924) took 1088 ms. Trace:
W20260709 13:16:54.956120 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 249016us
I20260709 13:16:54.952080 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:54.915781 11738 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.037912 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.927392 11488 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:54.930068 11758 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.063774 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:55.065461 10053 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:55.060458 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:55.067183 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 129718us
I20260709 13:16:55.043594 10204 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:55.061697 10122 rpcz_store.cc:276] 0709 13:16:53.912606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.113174 (+200568us) service_pool.cc:224] Handling call
0709 13:16:54.155599 (+ 42425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.172158 (+ 16559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.172211 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.239075 (+ 66864us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.665543 (+426468us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:54.889618 (+224075us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.001074 (+111456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.001159 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.001203 (+    44us) raft_consensus.cc:1685] finished
0709 13:16:55.001215 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.001350 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176256,"prepare.queue_time_us":360682,"raft.queue_time_us":793,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":20731,"thread_start_us":537741,"threads_started":2,"wal-append.queue_time_us":521362}
I20260709 13:16:55.072073 10202 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:55.084486 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.062481 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:55.074323 10198 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:16:55.034858 11761 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.004s
I20260709 13:16:55.017154 11606 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:16:55.002735 11749 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.926259 11698 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.087852 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:54.920404 11666 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.092974 10126 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:54.957065 11671 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.953336 11760 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:55.102586 10124 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 13:16:55.117957 10124 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:16:54.954145 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:54.992851 11740 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.953984 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:55.076535 11771 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.440s	user 0.001s	sys 0.001s
I20260709 13:16:54.953416 11777 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:16:55.031306 11496 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:55.109652 10198 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:16:55.032295 10048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:54.951545 11751 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.147s
W20260709 13:16:55.121280 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.952426 11656 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260709 13:16:55.125413 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:55.104749 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:55.075290 11738 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.077853 11488 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:55.172865 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 391627us
I20260709 13:16:55.179534 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:55.098034 11761 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.181690 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:55.156333 10048 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.134078 11584 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.110024 11749 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.089342 11718 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.165580 11797 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.122068 11740 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.121640 11671 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:55.186208 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.002s	sys 0.023s
I20260709 13:16:55.090673 11758 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.183984 11496 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:55.200994 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.002s	sys 0.024s
I20260709 13:16:55.201923 11797 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:16:55.195089 11776 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.378s	user 0.000s	sys 0.002s
I20260709 13:16:55.123509 11789 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
W20260709 13:16:55.224923 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 712038us
I20260709 13:16:55.186591 11777 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.198732 11626 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.697s	user 0.009s	sys 0.007s
I20260709 13:16:55.189778 10048 raft_consensus.cc:614] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:55.203744 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:55.196650 11774 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 13:16:55.231244 11789 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:55.231456 10196 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:55.231750 11789 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:55.232743 11785 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 13:16:55.232846 11789 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.233726 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 926) took 1284 ms. Trace:
W20260709 13:16:55.235230 11783 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:16:55.234175 10131 rpcz_store.cc:276] 0709 13:16:53.949562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.104686 (+155124us) service_pool.cc:224] Handling call
0709 13:16:54.105081 (+   395us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:54.105518 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.105547 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:54.148898 (+ 43351us) log.cc:844] Serialized 52 byte log entry
0709 13:16:54.393205 (+244307us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:54.723864 (+330659us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.946417 (+222553us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b18000c24a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:16:55.079965 (+133548us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.080243 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.080365 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.233417 (+153052us) raft_consensus.cc:1685] finished
0709 13:16:55.233445 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.233623 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137455,"prepare.queue_time_us":489997,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":5326,"raft.queue_time_us":336076,"raft.run_cpu_time_us":715,"raft.run_wall_time_us":34538,"spinlock_wait_cycles":10577792,"thread_start_us":506872,"threads_started":3,"wal-append.queue_time_us":531555}
I20260709 13:16:55.235618 11803 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.236958 11798 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:16:55.237380 11786 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:55.237586 11798 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:55.238077 11798 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.238109 11805 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.239010 11805 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:55.240401 11773 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
I20260709 13:16:55.242306 11805 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:55.243798 11749 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.244848 11749 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:55.245311 11749 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:16:55.245820 11782 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:16:55.248497 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.188611 11769 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:55.228300 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46703.3us
I20260709 13:16:55.235623 11666 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:55.253796 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.246313 11749 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.226423 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.260144 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:16:55.260648 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.261868 10205 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:55.206555 11496 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:55.269337 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:55.234400 11789 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:55.268544 11723 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.991s	user 0.006s	sys 0.009s
I20260709 13:16:55.229980 11751 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:55.332000 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 203234us
W20260709 13:16:55.335597 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66385.8us
I20260709 13:16:55.268114 11656 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:55.336723 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:55.270110 11666 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.277259 11496 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:55.344542 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 219183us
I20260709 13:16:55.345851 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:55.347318 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 120984us
I20260709 13:16:55.286379 11805 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:55.351058 10096 leader_election.cc:400] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
W20260709 13:16:55.353662 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.351761 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.354593 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 929) took 1218 ms. Trace:
I20260709 13:16:55.257870 11626 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.358657 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:55.355093 10127 rpcz_store.cc:276] 0709 13:16:54.135954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.193542 (+ 57588us) service_pool.cc:224] Handling call
0709 13:16:54.193931 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.242398 (+ 48467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.242445 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.263207 (+ 20762us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.529448 (+266241us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:54.529504 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.529583 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.354336 (+824753us) raft_consensus.cc:1685] finished
0709 13:16:55.354365 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.354519 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50572,"raft.queue_time_us":69647,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":38749,"thread_start_us":223206,"threads_started":1,"wal-append.queue_time_us":445894}
I20260709 13:16:55.361176 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 932) took 1163 ms. Trace:
I20260709 13:16:55.337633 11688 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:55.337337 11613 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.339196 11803 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.361526 10128 rpcz_store.cc:276] 0709 13:16:54.198130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.217256 (+ 19126us) service_pool.cc:224] Handling call
0709 13:16:54.229260 (+ 12004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.264207 (+ 34947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:54.264252 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.283553 (+ 19301us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.574156 (+290603us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:54.872771 (+298615us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:16:54.926810 (+ 54039us) log.cc:844] Serialized 14 byte log entry
0709 13:16:55.337431 (+410621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.337528 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.337576 (+    48us) raft_consensus.cc:1685] finished
0709 13:16:55.360951 (+ 23375us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.361105 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210132,"prepare.queue_time_us":357094,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":71,"raft.queue_time_us":222107,"raft.run_cpu_time_us":2765,"raft.run_wall_time_us":606830,"spinlock_wait_cycles":148313856,"thread_start_us":804266,"threads_started":3,"wal-append.queue_time_us":501172}
I20260709 13:16:55.345319 11698 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.341936 11656 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:55.372538 10053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:55.374125 10053 raft_consensus.cc:606] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.374609 10053 raft_consensus.cc:614] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:55.356756 11769 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.355789 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.284003 11790 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.265s
I20260709 13:16:55.381343 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:55.252629 11777 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.342844 11723 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.387131 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.388015 11807 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:55.388506 11804 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.391366 11801 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.392071 11738 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.392851 11626 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:55.393174 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.014s
I20260709 13:16:55.378566 11672 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:16:55.379660 11416 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.354710 11666 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.370577 11805 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:55.440754 11805 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.370077 11656 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:55.412786 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.000s	sys 0.015s
I20260709 13:16:55.402036 11803 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:16:55.411857 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53253.2us
I20260709 13:16:55.374006 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:55.404827 10197 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.591s	user 0.006s	sys 0.007s
I20260709 13:16:55.420881 11671 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:55.421378 11764 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.065s	user 0.008s	sys 0.006s
I20260709 13:16:55.422516 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:55.474428 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87387.4us
W20260709 13:16:55.474797 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52420.9us
I20260709 13:16:55.423405 11770 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.423871 11738 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.430867 11793 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.430207 11784 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:55.449878 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.454553 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102926us
I20260709 13:16:55.373171 11613 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.467420 11811 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:55.468611 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:55.450631 11626 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:55.470155 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:55.445243 11414 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.001s
I20260709 13:16:55.461764 10197 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:16:55.471030 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:55.447356 10123 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:55.479341 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:55.484676 11630 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:55.506917 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.394876 11769 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.443425 11761 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.459996 11684 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.523213 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.529592 10900 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:16:55.509303 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.540935 11488 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:55.541486 11802 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:16:55.472921 10055 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
W20260709 13:16:55.543423 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64356.3us
I20260709 13:16:55.574126 11613 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:55.560808 11626 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:55.576735 11671 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:55.530527 11770 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.542466 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1279) took 1031 ms. Trace:
I20260709 13:16:55.537175 11803 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:55.551005 11661 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:55.551576 10048 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:55.564286 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:55.564565 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1283) took 1040 ms. Trace:
I20260709 13:16:55.590061 10201 rpcz_store.cc:276] 0709 13:16:54.524151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.524348 (+   197us) service_pool.cc:224] Handling call
0709 13:16:55.564214 (+1039866us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400260cd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:55.564448 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486779648}
I20260709 13:16:55.542613 10895 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.489369 11764 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.521328 11738 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.527678 11793 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.549127 11784 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.560760 11807 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:55.566666 11511 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.003s
I20260709 13:16:55.594863 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:55.595856 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.597903 11672 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:55.598376 11803 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.579413 10197 rpcz_store.cc:276] 0709 13:16:54.511007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.514811 (+  3804us) service_pool.cc:224] Handling call
0709 13:16:55.536769 (+1021958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:55.563284 11810 raft_consensus.cc:476] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 13:16:55.535440 11811 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.576766 11626 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.602231 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.752928 10054 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.628686 10199 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:16:55.754298 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:55.756105 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.756597 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 212764us
I20260709 13:16:55.758598 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:55.758919 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.581010 11758 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.758232 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:16:55.762980 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 239864us
I20260709 13:16:55.763486 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:55.716698 11532 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.789s	user 0.015s	sys 0.000s
W20260709 13:16:55.684460 11800 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:16:55.586391 11661 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:55.774073 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.682353 10896 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:55.778009 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 118300us
I20260709 13:16:55.785770 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:55.748705 11718 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.655510 11687 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:55.783852 11825 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:55.786710 10051 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:55.757361 11678 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:55.769728 11690 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.602615 11630 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:55.570410 11684 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.737136 11738 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.752573 11807 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:55.750100 11770 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:55.749289 11815 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:16:55.682363 11808 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.715219 11517 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.832548 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 69222.8us
I20260709 13:16:55.770195 11751 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:55.769549 11416 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:55.783592 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.826996 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:55.791747 11661 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:55.761835 11511 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.793922 11810 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:55.785120 11532 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.918439 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.853026 11807 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:55.924403 10050 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:16:55.857600 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.857666 11777 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:55.933288 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 160921us
I20260709 13:16:55.925700 10051 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:55.828358 11687 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.924798 10047 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:16:55.801702 11718 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.848162 11630 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.912920 11751 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:55.931849 11818 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:16:55.931967 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:55.899948 11769 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:55.943691 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:55.836133 11690 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.921412 11817 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.847384 11678 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:55.838470 11825 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.939798 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.949131 10053 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:55.909984 11517 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.923753 10892 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:55.953796 11764 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:55.962947 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:56.010494 10049 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:55.948256 11828 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:55.968976 11718 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.015470 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:55.962313 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:55.935817 11827 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:55.942049 11815 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:56.020613 10049 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:16:56.000056 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 226086us
I20260709 13:16:56.023504 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 911) took 1339 ms. Trace:
I20260709 13:16:56.023944 10049 rpcz_store.cc:276] 0709 13:16:54.684421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.694431 (+ 10010us) service_pool.cc:224] Handling call
0709 13:16:55.962715 (+1268284us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:56.023412 (+ 60697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11803392}
I20260709 13:16:56.024407 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:55.951515 11751 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:55.949774 11770 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.966005 11789 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.633s	user 0.013s	sys 0.000s
I20260709 13:16:55.918561 11808 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.021064 11815 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.059274 11815 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:55.944103 11672 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:56.005718 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62068.9us
I20260709 13:16:55.955657 11805 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.969594 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1276) took 1459 ms. Trace:
I20260709 13:16:56.070456 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.070154 10199 rpcz_store.cc:276] 0709 13:16:54.509972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.522611 (+ 12639us) service_pool.cc:224] Handling call
0709 13:16:54.543081 (+ 20470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.754188 (+1211107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.969259 (+215071us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:55.969508 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495571200}
I20260709 13:16:55.972918 10055 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:55.940088 11656 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:55.957391 11684 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:55.990384 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.993444 11769 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.956142 10895 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:55.948755 11777 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:16:55.999979 11810 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:55.946708 11416 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.959201 11816 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.336s	user 0.002s	sys 0.000s
W20260709 13:16:56.031368 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69184.6us
I20260709 13:16:56.022645 11828 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:16:55.972646 11801 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.005s
W20260709 13:16:56.060104 11798 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.821s	user 0.014s	sys 0.003s
I20260709 13:16:56.096449 11798 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.098553 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 936) took 1149 ms. Trace:
I20260709 13:16:55.988816 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:56.099481 10051 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:56.099594 11811 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.100591 11825 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.101307 11784 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.030279 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:56.102733 11687 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:56.103415 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73214.4us
I20260709 13:16:56.103276 11672 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.104925 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 906) took 1564 ms. Trace:
I20260709 13:16:56.105163 11808 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.105821 11705 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:56.060727 11718 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:55.966151 11738 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:56.105386 10056 rpcz_store.cc:276] 0709 13:16:54.540603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.540824 (+   221us) service_pool.cc:224] Handling call
0709 13:16:54.541297 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.408386 (+867089us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b18002a7300. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa277ffcbfb 00007fa27800764b
0709 13:16:55.424957 (+ 16571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.424986 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.433702 (+  8716us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.910117 (+476415us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:55.910161 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.910251 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.104551 (+194300us) raft_consensus.cc:1685] finished
0709 13:16:56.104581 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.104846 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":217368,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":11932,"spinlock_wait_cycles":14953344,"thread_start_us":637253,"threads_started":2,"wal-append.queue_time_us":530710}
W20260709 13:16:56.149868 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.142792 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72489.4us
I20260709 13:16:56.066576 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.032205 11827 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:16:56.010406 11817 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:56.080406 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.128959 11789 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.078243 11829 raft_consensus.cc:2764] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:16:56.077960 11805 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.082289 11769 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:56.096951 11826 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:56.076334 10054 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:56.177778 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:56.117218 11496 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.756s	user 0.015s	sys 0.000s
W20260709 13:16:56.179240 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 112766us
I20260709 13:16:56.067059 11657 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:16:56.128528 11793 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.102790 11656 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:56.092728 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.113286 11801 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:56.157737 11738 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.188740 11837 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:56.171286 10051 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:56.212095 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:56.212718 10896 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:16:56.195832 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.072731 11770 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:56.215411 11678 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.215871 11770 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.110607 11810 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:56.193187 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 98049.8us
I20260709 13:16:56.205597 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:56.211280 11738 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.202224 11496 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.159646 11511 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.183967 11784 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:56.190202 11798 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.218277 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.230968 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.232009 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.212785 10128 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.218967 10127 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:16:56.213454 11769 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.235458 10122 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:56.200008 11684 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.094648 11416 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.204181 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:56.238060 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.241933 10131 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:56.230336 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:16:56.244120 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86806.9us
I20260709 13:16:56.204105 11690 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.257279 10125 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:56.257740 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:56.258488 10129 raft_consensus.cc:2376] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2. Re-sending same reply.
I20260709 13:16:56.229530 11678 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:56.221241 11808 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.217952 11657 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:56.230731 11833 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:56.201261 11789 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.230189 11784 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:56.108186 10123 rpcz_store.cc:276] 0709 13:16:54.949577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.953651 (+  4074us) service_pool.cc:224] Handling call
0709 13:16:56.098262 (+1144611us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54001f3454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:56.098449 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584361216}
I20260709 13:16:56.250545 11718 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:56.338133 10131 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:56.237097 11793 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:56.339572 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.237368 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 914) took 1290 ms. Trace:
I20260709 13:16:56.369179 11817 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.367835 10052 rpcz_store.cc:276] 0709 13:16:54.939470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.951182 (+ 11712us) service_pool.cc:224] Handling call
0709 13:16:55.103936 (+152754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:55.165196 (+ 61260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:55.169921 (+  4725us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.257304 (+ 87383us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.482882 (+225578us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:55.523350 (+ 40468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.532845 (+  9495us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.202521 (+669676us) raft_consensus.cc:1685] finished
0709 13:16:56.202575 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.230137 (+ 27562us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1796,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":32279,"thread_start_us":99006,"threads_started":1,"wal-append.queue_time_us":187582}
I20260709 13:16:56.343561 10199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:16:56.352977 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.370955 10199 raft_consensus.cc:606] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:56.341835 11784 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.371593 10199 raft_consensus.cc:614] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:56.283802 11690 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:56.294178 10050 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:56.292528 11705 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:56.229869 11828 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.220000 11810 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.287344 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:56.440426 10130 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:56.426177 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 194285us
I20260709 13:16:56.358219 11830 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:56.354580 11718 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:56.294898 11798 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:16:56.344049 11678 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:56.362389 11793 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.436127 10131 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:16:56.272096 11684 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.362854 11805 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.339998 11808 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.356096 11833 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.460644 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.486171 10126 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:56.442997 11838 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.425213 11690 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:16:56.471688 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:16:56.442798 11764 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.464080 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.468943 10130 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:16:56.491707 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.458491 11718 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.470356 11793 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:56.483299 11799 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.225s	user 0.006s	sys 0.000s
I20260709 13:16:56.469835 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:56.503929 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.555754 10130 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:56.468746 11830 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:16:56.551959 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:56.500290 11690 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:56.562505 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:56.358739 11827 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.506505 11805 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.508301 11496 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:56.559286 11809 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:16:56.470603 11657 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.552606 10123 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:56.505342 11764 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.600790 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:56.602973 10131 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:56.565855 11844 raft_consensus.cc:2764] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:16:56.579389 11789 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.613613 11761 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.614547 11761 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.617571 11761 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.618393 11761 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.503947 11838 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.627048 11833 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:56.608978 11416 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.628125 11416 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.628579 11416 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.630996 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:56.590297 11805 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:56.645865 11661 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.003s
I20260709 13:16:56.581015 11678 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.648613 11764 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.649091 11789 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:56.649367 11764 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.625190 11840 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.567708 11705 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:56.663719 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:56.667598 11843 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:16:56.646473 11661 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:56.653203 11848 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.002s
W20260709 13:16:56.685101 11834 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:16:56.649884 11764 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.618971 11761 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.690994 11838 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.699748 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:56.712441 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:16:56.760058 11839 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:16:56.673220 11840 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:56.751581 11838 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.683146 11830 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.814265 11723 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.854421 11797 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:56.816861 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 90) took 1461 ms (client timeout 1620 ms). Trace:
I20260709 13:16:56.819159 11740 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.756330 11808 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:56.827807 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 292) took 1357 ms. Trace:
W20260709 13:16:56.830241 11807 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.888s	user 0.008s	sys 0.008s
I20260709 13:16:56.833668 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2397) took 1475 ms. Trace:
I20260709 13:16:56.888342 10125 rpcz_store.cc:276] 0709 13:16:55.358347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.358566 (+   219us) service_pool.cc:224] Handling call
0709 13:16:56.833199 (+1474633us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54001c6e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:56.833558 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2000022272}
I20260709 13:16:56.835278 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:56.890098 10900 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:16:56.837677 11738 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.618s	user 0.015s	sys 0.008s
I20260709 13:16:56.759991 10131 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:16:56.814656 11751 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.748s	user 0.021s	sys 0.004s
I20260709 13:16:56.860095 11488 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:56.863373 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 375640us
W20260709 13:16:56.862900 10048 rpcz_store.cc:269] 0709 13:16:55.355204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.355409 (+   205us) service_pool.cc:224] Handling call
0709 13:16:56.816405 (+1460996us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b54001b7e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:56.816726 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207414016}
I20260709 13:16:56.878916 11661 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:56.879750 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.878541 11797 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:56.911350 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:57.166136 10122 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:57.219303 10131 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:57.220041 10049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:57.220863 10049 raft_consensus.cc:606] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:57.221026 10122 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:57.221366 10049 raft_consensus.cc:614] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:57.222045 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3416443490000347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:16:57.136353 10021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.006s	sys 0.037s
W20260709 13:16:57.280577 10021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.009s	sys 0.038s
W20260709 13:16:57.222731 11638 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.579s	user 0.019s	sys 0.017s
I20260709 13:16:56.888065 10896 rpcz_store.cc:276] 0709 13:16:55.470413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.482443 (+ 12030us) service_pool.cc:224] Handling call
0709 13:16:55.528218 (+ 45775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.213355 (+685137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.827422 (+614067us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:56.827683 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":290086784}
I20260709 13:16:56.946686 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:56.920564 11738 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.224537 11751 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.226310 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 930) took 1570 ms. Trace:
I20260709 13:16:57.227128 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 934) took 1225 ms. Trace:
I20260709 13:16:57.299887 10050 rpcz_store.cc:276] 0709 13:16:56.001385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.015079 (+ 13694us) service_pool.cc:224] Handling call
0709 13:16:57.226879 (+1211800us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400135954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:57.227021 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638853888}
I20260709 13:16:56.911470 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 355) took 1108 ms. Trace:
W20260709 13:16:56.986809 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.041s	sys 0.037s
I20260709 13:16:57.228289 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:57.217775 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:57.302542 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.046s	sys 0.046s
I20260709 13:16:57.302748 10129 raft_consensus.cc:2376] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4. Re-sending same reply.
I20260709 13:16:57.298804 10056 rpcz_store.cc:276] 0709 13:16:55.656316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.154358 (+498042us) service_pool.cc:224] Handling call
0709 13:16:56.154793 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.177493 (+ 22700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:56.177542 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:56.178290 (+   748us) log.cc:844] Serialized 26 byte log entry
0709 13:16:56.360524 (+182234us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:56.360575 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:56.360652 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:57.226007 (+865355us) raft_consensus.cc:1685] finished
0709 13:16:57.226036 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.226229 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":24507,"thread_start_us":142363,"threads_started":1,"wal-append.queue_time_us":302073}
I20260709 13:16:57.229218 11488 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.230520 10130 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:16:57.231437 11815 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.154s	user 0.016s	sys 0.011s
W20260709 13:16:57.203470 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.013s	sys 0.022s
I20260709 13:16:57.231892 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 59) took 1023 ms. Trace:
I20260709 13:16:57.328950 10127 rpcz_store.cc:276] 0709 13:16:56.208533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.208715 (+   182us) service_pool.cc:224] Handling call
0709 13:16:56.209189 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.235866 (+ 26677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.231582 (+995716us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:57.231816 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21530496}
I20260709 13:16:56.858832 11838 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.234931 11761 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.235741 11749 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:16:57.078338 11784 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 13:16:57.239259 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 920) took 1658 ms. Trace:
I20260709 13:16:57.341974 10053 rpcz_store.cc:276] 0709 13:16:55.580485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.582326 (+  1841us) service_pool.cc:224] Handling call
0709 13:16:57.238961 (+1656635us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:57.239176 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447651200}
I20260709 13:16:57.249900 11630 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:57.252635 11690 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.000s
I20260709 13:16:57.253145 11532 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.254632 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:57.255898 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:57.346696 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 64) took 1002 ms. Trace:
I20260709 13:16:57.347198 10125 rpcz_store.cc:276] 0709 13:16:56.344183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.912186 (+568003us) service_pool.cc:224] Handling call
0709 13:16:57.255592 (+343406us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:57.346600 (+ 91008us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3788032}
I20260709 13:16:57.258021 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2410) took 2062 ms. Trace:
W20260709 13:16:57.267580 11820 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.333s	user 0.011s	sys 0.000s
I20260709 13:16:57.268553 11661 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.270329 11513 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.273545 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.279256 11517 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.333863 11416 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:57.288874 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.335664 11740 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:16:57.293376 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.023s	sys 0.026s
W20260709 13:16:57.229411 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 928) took 1611 ms (client timeout 1828 ms). Trace:
W20260709 13:16:57.370733 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.023s	sys 0.026s
I20260709 13:16:57.301376 10900 rpcz_store.cc:276] 0709 13:16:55.782462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.782694 (+   232us) service_pool.cc:224] Handling call
0709 13:16:56.834953 (+1052259us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:56.890718 (+ 55765us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4508544}
W20260709 13:16:57.371553 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 819657us
W20260709 13:16:57.370846 10047 rpcz_store.cc:269] 0709 13:16:55.617928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.758578 (+140650us) service_pool.cc:224] Handling call
0709 13:16:57.229176 (+1470598us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400198f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:57.229342 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379052928}
I20260709 13:16:57.321310 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:57.373715 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52488.9us
W20260709 13:16:57.320461 11758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.005s	sys 0.050s
I20260709 13:16:57.251138 11666 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:57.298025 11688 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.333158 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.014s	sys 0.022s
W20260709 13:16:57.299615 11791 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 2.120s	user 0.005s	sys 0.012s
I20260709 13:16:57.381450 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:57.348522 10205 rpcz_store.cc:276] 0709 13:16:55.195243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:55.195453 (+   210us) service_pool.cc:224] Handling call
0709 13:16:55.195840 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:55.262555 (+ 66715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.235091 (+1972536us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b540012aa54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:57.257930 (+ 22839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150577536}
I20260709 13:16:57.328581 11837 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.331250 11672 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.330053 11808 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.910779 11807 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:57.293083 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.000s	sys 0.029s
W20260709 13:16:57.421494 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.000s	sys 0.029s
I20260709 13:16:57.375962 10047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "db052bef7536453fa2d7f5a00e314a18"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:16:57.344760 11847 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.344229 11761 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:57.333715 11815 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.405716 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.393082 11835 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.407153 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.398478 11805 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:57.392141 11770 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.021s	sys 0.015s
I20260709 13:16:57.346485 11784 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.407966 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:57.423733 10047 raft_consensus.cc:606] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Received request to transfer leadership to TS db052bef7536453fa2d7f5a00e314a18
I20260709 13:16:57.425491 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.380128 11671 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.426820 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:57.418959 11764 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.427620 10050 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:57.426905 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:57.400295 11791 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 2.228s	user 0.005s	sys 0.013s
I20260709 13:16:57.429600 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:57.382354 11684 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.358371 11690 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.396337 11630 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.406666 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.433121 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.434499 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.436669 10205 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
W20260709 13:16:57.375190 11758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.005s	sys 0.051s
I20260709 13:16:57.438468 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.396770 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 294) took 1145 ms. Trace:
I20260709 13:16:57.408030 11688 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:16:57.401026 11606 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.446133 10124 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:57.377044 11513 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:57.394011 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.13328e+06us
I20260709 13:16:57.508162 10049 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:57.377761 11740 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.507974 10201 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:57.427495 11810 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:57.440181 11761 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:57.559106 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 223961us
I20260709 13:16:57.559866 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:57.444010 11770 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.022s	sys 0.016s
I20260709 13:16:57.565217 10196 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:16:57.416079 11837 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.369463 11661 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.554625 11817 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.579968 10051 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:16:57.580531 10053 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:16:57.613858 11785 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.575729 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.572991 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.656649 10199 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:57.657214 10056 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:57.551635 10047 raft_consensus.cc:614] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:57.659344 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.660754 10047 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:57.450356 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:57.576089 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.605552 11606 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.446039 11835 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.608948 11513 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.609855 11488 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:57.610002 11718 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.050s	user 0.012s	sys 0.000s
I20260709 13:16:57.610363 11807 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.548868 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.580783 10895 rpcz_store.cc:276] 0709 13:16:56.248624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.251331 (+  2707us) service_pool.cc:224] Handling call
0709 13:16:57.254347 (+1003016us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:57.393554 (+139207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27800320}
I20260709 13:16:57.612062 11690 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.548388 11858 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.554035 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 936) took 1537 ms. Trace:
I20260709 13:16:57.431408 11808 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.561690 11666 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:57.437245 11830 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.145s
W20260709 13:16:57.623482 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 242423us
I20260709 13:16:57.435672 11672 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.568321 11702 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:57.437341 11847 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.630486 10203 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:16:57.656955 11751 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.639868 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:57.641001 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 266541us
I20260709 13:16:57.575601 11416 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.571086 11859 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:57.575445 11855 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.553970 11671 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.684056 11764 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.086s
I20260709 13:16:57.571333 11684 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.600106 10052 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:57.608938 11815 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:57.663196 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:57.665494 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:16:57.670173 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.013s	sys 0.023s
I20260709 13:16:57.573410 11630 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.669474 10049 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:57.682323 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 941) took 1452 ms. Trace:
I20260709 13:16:57.668634 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:57.647516 11797 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.633769 10123 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:57.723860 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:16:57.803395 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 134833us
I20260709 13:16:57.645289 10200 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:57.728480 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:57.657902 11793 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.094s	user 0.014s	sys 0.003s
I20260709 13:16:57.685364 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.740365 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.742292 10054 rpcz_store.cc:276] 0709 13:16:56.018314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.241919 (+223605us) service_pool.cc:224] Handling call
0709 13:16:56.279497 (+ 37578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.279879 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:57.216190 (+936311us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.553594 (+337404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.553657 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:57.553728 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.553806 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.553963 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133162,"prepare.queue_time_us":284069,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":80,"raft.queue_time_us":140547,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":14179,"spinlock_wait_cycles":29846784,"thread_start_us":579683,"threads_started":2,"wal-append.queue_time_us":381794}
I20260709 13:16:57.692616 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 935) took 1686 ms. Trace:
I20260709 13:16:57.693511  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:16:57.731063 11638 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:16:57.684355 10197 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:16:57.764670 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:57.635455 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 231183us
I20260709 13:16:57.665570 11840 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.728891 11838 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:57.757408 10048 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:57.682955 11810 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:57.755904 11789 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.106s	user 0.012s	sys 0.003s
I20260709 13:16:57.752429 11871 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.720738 11718 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.757741 11738 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.581306 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.911957 11784 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.792716 11855 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.923738 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:16:57.789364 10863 connection.cc:256] RPC call timeout handler was delayed by -1.1893657079999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 13:16:57.795989 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.015s	sys 0.024s
I20260709 13:16:57.797746 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.684652 11817 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.804307 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:57.941066 10900 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:16:57.936273 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:16:57.799451 10124 rpcz_store.cc:276] 0709 13:16:56.230347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.230634 (+   287us) service_pool.cc:224] Handling call
0709 13:16:57.681957 (+1451323us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54002d1254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:57.682212 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2791967488}
W20260709 13:16:57.805616 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 142614us
I20260709 13:16:57.786721 11751 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:57.793622 11852 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.371s	user 0.001s	sys 0.001s
I20260709 13:16:57.685360 11785 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.811225 11657 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:57.808940 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.921054 11690 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:57.789000 11416 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.792050 10202 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:57.927248 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.917172 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:57.753019 11858 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.870621 10204 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:57.938697 10201 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:57.767648 11880 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:57.945691 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:57.763533 11702 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.845134 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.933996 11784 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:57.953110 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 148892us
I20260709 13:16:57.912356 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.925526 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.971514 11838 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:57.988827 11882 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:16:58.014199 11606 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:58.014744 11879 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.016063 11790 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:57.994647 11828 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:16:58.017109 11690 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.940862 11871 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.811283 11770 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:57.991489 10056 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:58.022014 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:57.991608  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:57.934033 11808 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:58.026073 10047 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:16:58.028805 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44054.5us
I20260709 13:16:58.031915 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:57.989362 10051 rpcz_store.cc:276] 0709 13:16:56.005853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.214716 (+208863us) service_pool.cc:224] Handling call
0709 13:16:56.215089 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.651716 (+1436627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.692163 (+ 40447us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001630d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:57.692503 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3132769024}
I20260709 13:16:57.983306 11661 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:57.947554 11688 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:57.987026 11789 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:57.794689 11859 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.834028 11684 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:57.943854 11793 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:57.982111 11751 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:58.519441 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:58.527516 10196 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:58.525995 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:58.626689 11690 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.127291 11847 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.115253 11837 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:58.097923 11784 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:58.614930 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:57.996291 11758 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:58.624634 11871 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:58.524252 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.127184 11798 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.024987 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:58.570387 11879 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:58.111308 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:58.524252 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:58.054801 10205 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:58.033114 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:58.202723 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.035799 11657 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:58.053040 11761 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.655953 11718 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:58.621480 11630 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:58.519225 11876 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.465s	user 0.000s	sys 0.001s
I20260709 13:16:58.546020 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1289) took 1099 ms. Trace:
W20260709 13:16:58.548347 10864 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.631s (SENT) [suppressed 17 similar messages]
W20260709 13:16:58.520272 11886 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 13:16:58.033665 11880 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:58.399029 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.534588 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:58.687494 10052 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.694255 11847 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:16:58.368973 11810 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:58.366184 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:57.987891 11488 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.566701 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:58.603403 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.399288 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:16:58.652913 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.008s	sys 0.002s
I20260709 13:16:58.619313 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:58.527256 11838 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.684921 11764 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:16:58.516665 10131 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:58.388206 11672 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:16:58.314724 11863 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.512s	user 0.000s	sys 0.002s
W20260709 13:16:58.614667 11877 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.498s	user 0.001s	sys 0.000s
I20260709 13:16:58.671701 11858 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.671975 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:58.751695 11847 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.675309 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:58.501027 11671 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:58.678014 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 152085us
I20260709 13:16:58.860082 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:58.680810 10054 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:16:58.118074 11865 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.361s	user 0.001s	sys 0.001s
I20260709 13:16:58.682965 10049 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
W20260709 13:16:58.653178 11853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.008s
I20260709 13:16:58.688386 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.693615 10201 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:58.113619 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:58.693825 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1252) took 1073 ms. Trace:
W20260709 13:16:58.660459 11849 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.001s
I20260709 13:16:58.114255 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:58.718528 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.570875 11738 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.119s
I20260709 13:16:58.130446 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:16:58.687668 11749 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.346s	user 0.009s	sys 0.008s
I20260709 13:16:58.345685 11835 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:58.730728 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 182422us
I20260709 13:16:58.724913 10204 rpcz_store.cc:276] 0709 13:16:57.405219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.405429 (+   210us) service_pool.cc:224] Handling call
0709 13:16:58.504664 (+1099235us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400359dd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:58.504860 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009251200}
I20260709 13:16:58.959777 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:16:58.960376 10204 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:58.735664 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:58.572060 11828 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:58.957028 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:58.964654 10201 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:58.685387 11883 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.535759 10130 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:58.201174 11857 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.536s
W20260709 13:16:58.518488 11878 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.486s	user 0.000s	sys 0.000s
I20260709 13:16:58.740212 10128 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:58.694315 11817 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.688247 11702 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:16:58.085054 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53368.7us
I20260709 13:16:58.688130 11758 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:58.754002 10047 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:58.754522 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:58.755074 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.008s	sys 0.003s
I20260709 13:16:58.755719 10051 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:58.753153 10125 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:58.131170 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:58.737861 11416 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.079384 11758 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:59.080988 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1257) took 1458 ms (client timeout 1522 ms). Trace:
W20260709 13:16:59.081606 10056 rpcz_store.cc:269] 0709 13:16:57.622330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.665096 (+ 42766us) service_pool.cc:224] Handling call
0709 13:16:59.080720 (+1415624us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400199954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.080901 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432117504}
W20260709 13:16:59.083161 11893 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.184s	user 0.000s	sys 0.005s
I20260709 13:16:59.048118 11897 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:59.047370 11882 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.085480 11770 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.002s
I20260709 13:16:59.086127 11770 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:58.687407 11798 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:59.070432 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 962722us
W20260709 13:16:58.775218 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 112204us
I20260709 13:16:59.087534 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:59.088341 11798 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:16:58.829063 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.089036 11798 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.089681 11798 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:16:59.090173 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.776445 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:59.091538 10898 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 1.
I20260709 13:16:58.373354 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:59.093215 10893 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:16:59.093745 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 124) took 1060 ms. Trace:
I20260709 13:16:59.093916 11828 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.740046 11889 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.748944 11751 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.094149 10893 rpcz_store.cc:276] 0709 13:16:58.033890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.277526 (+243636us) service_pool.cc:224] Handling call
0709 13:16:59.093647 (+816121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.094581 11828 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:16:58.894469 10053 rpcz_store.cc:276] 0709 13:16:57.620804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.053033 (+432229us) service_pool.cc:224] Handling call
0709 13:16:58.693735 (+640702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.097985 11889 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:59.098711 11828 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.099473 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.100543 10053 raft_consensus.cc:2410] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:59.101027 11889 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.101213 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1249) took 1535 ms. Trace:
I20260709 13:16:59.101753 10053 rpcz_store.cc:276] 0709 13:16:57.567466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.099066 (+1531600us) service_pool.cc:224] Handling call
0709 13:16:59.101128 (+  2062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:58.926630 10124 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:58.772509 11785 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:58.752074 11887 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:16:59.052690 11892 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:58.952313 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.961053 10205 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:59.105851 10122 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.105996 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2412) took 1510 ms. Trace:
I20260709 13:16:59.105295 11887 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.106516 10205 rpcz_store.cc:276] 0709 13:16:57.596198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.718496 (+1122298us) service_pool.cc:224] Handling call
0709 13:16:59.105902 (+387406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:58.961714 11606 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:16:58.892645 11853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.000s	sys 0.010s
I20260709 13:16:58.859252 11847 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:16:59.110656 11828 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.111388 11887 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.112079 11416 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.112705 11416 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:58.925963 11849 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.009s	sys 0.001s
I20260709 13:16:58.967384 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2427) took 1161 ms. Trace:
I20260709 13:16:59.107586 10892 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.115564 10201 rpcz_store.cc:276] 0709 13:16:57.806182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.894115 (+1087933us) service_pool.cc:224] Handling call
0709 13:16:58.967312 (+ 73197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.117002 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:59.116551 11606 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:59.117796 10201 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.118451 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2421) took 1350 ms. Trace:
W20260709 13:16:58.970376 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 571138us
I20260709 13:16:59.118836 10201 rpcz_store.cc:276] 0709 13:16:57.767886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.116604 (+1348718us) service_pool.cc:224] Handling call
0709 13:16:59.118358 (+  1754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.120057 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:58.975657 11874 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.935029 11858 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:16:58.979712 11885 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.612s	user 0.000s	sys 0.001s
I20260709 13:16:58.774839 11488 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:59.047891 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:59.109361 11903 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 13:16:59.150424 10130 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:16:59.107687 11867 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.339s	user 0.000s	sys 0.010s
I20260709 13:16:59.151913 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:58.953596 10048 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:16:58.993290 11883 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:58.961928 11767 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.021s	sys 0.026s
I20260709 13:16:58.969349 11657 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.054150 10055 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:16:59.048518 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.057425 10054 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:16:59.070027 10126 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.072403 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1254) took 1451 ms. Trace:
I20260709 13:16:59.074967 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1243) took 1512 ms. Trace:
I20260709 13:16:59.075517 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:59.076894 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2429) took 1131 ms. Trace:
I20260709 13:16:59.085671 11882 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:58.831604 10196 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.095779 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:59.076241 10204 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:58.863957 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:59.103711 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.047379 11857 consensus_queue.cc:1243] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:16:58.772151 11838 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.106989 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.105352 10053 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:16:59.105252 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.052140 11880 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.107903 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:59.050362 11879 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.110473 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:16:59.101027 10899 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:59.122689 10203 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:59.124645 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 76) took 1708 ms. Trace:
I20260709 13:16:59.151845 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:58.967628 11806 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.160902 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1265) took 1473 ms. Trace:
I20260709 13:16:59.122454 11858 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.090863 11798 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:59.089720 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:59.298086 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 149220us
W20260709 13:16:59.371085 10204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2423) took 1585 ms (client timeout 2102 ms). Trace:
I20260709 13:16:59.371825 10128 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 1.
W20260709 13:16:59.371745 10204 rpcz_store.cc:269] 0709 13:16:57.785453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.959518 (+1174065us) service_pool.cc:224] Handling call
0709 13:16:59.371009 (+411491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.373080 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.373796 10128 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:16:59.374903 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.376479 11882 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.397307 10128 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.377664 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.399830 10126 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
I20260709 13:16:59.401417 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.157994 11785 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.380743 10047 rpcz_store.cc:276] 0709 13:16:57.621326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.566254 (+944928us) service_pool.cc:224] Handling call
0709 13:16:59.072339 (+506085us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.382032 10124 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.382748 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1246) took 1818 ms. Trace:
I20260709 13:16:59.383179 10200 rpcz_store.cc:276] 0709 13:16:57.945230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.991191 (+ 45961us) service_pool.cc:224] Handling call
0709 13:16:59.076789 (+1085598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.384406 11901 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:59.527084 10129 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:59.150537 11874 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:59.162947 11767 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.021s	sys 0.027s
I20260709 13:16:59.159907 11416 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.378428 10051 rpcz_store.cc:276] 0709 13:16:57.562784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.526930 (+964146us) service_pool.cc:224] Handling call
0709 13:16:59.074888 (+547958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.408092 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.586810 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.588080 10126 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:59.423235 10201 rpcz_store.cc:276] 0709 13:16:57.412107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.119874 (+1707767us) service_pool.cc:224] Handling call
0709 13:16:59.120249 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.120464 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.120501 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.120549 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:59.120592 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.120648 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.120790 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.435834 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:59.456664 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85340.2us
I20260709 13:16:59.566941 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:59.589485 10096 connection.cc:256] RPC call timeout handler was delayed by -1.000742425999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 13:16:59.567324 10054 rpcz_store.cc:276] 0709 13:16:57.563892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.545718 (+981826us) service_pool.cc:224] Handling call
0709 13:16:59.382651 (+836933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.142278 11784 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.150449 11828 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:16:59.530118 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 666186us
I20260709 13:16:59.568759 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1250) took 1969 ms. Trace:
I20260709 13:16:59.569115 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.601150 10124 raft_consensus.cc:2410] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:59.601020 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:59.600585 10053 rpcz_store.cc:276] 0709 13:16:57.599575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.102634 (+1503059us) service_pool.cc:224] Handling call
0709 13:16:59.105140 (+  2506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.567962 (+462822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.568625 (+   663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.570050 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.602805 10047 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:59.603346 11738 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.153128 11887 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:59.384829 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 309389us
I20260709 13:16:59.603907 11702 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.377434 11606 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.603825 11849 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:59.571815 11858 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.581379 10196 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:16:59.614352 10196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2416) took 1890 ms (client timeout 1993 ms). Trace:
I20260709 13:16:59.557307 11859 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:16:59.614804 10196 rpcz_store.cc:269] 0709 13:16:57.724264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.671430 (+947166us) service_pool.cc:224] Handling call
0709 13:16:59.614279 (+942849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.593222 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.577513 11853 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:16:59.610806 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:59.611750 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.130975 11684 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.571187 11901 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.615154 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:59.596231 10128 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:16:59.618947 10130 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:16:59.215838 10897 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:16:59.585965 11797 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:16:59.645263 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.646422 10051 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:59.605744 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 296) took 1572 ms. Trace:
I20260709 13:16:59.499228 10048 rpcz_store.cc:276] 0709 13:16:57.687902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.055687 (+367785us) service_pool.cc:224] Handling call
0709 13:16:59.159277 (+1103590us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:59.577042 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 487384us
I20260709 13:16:59.616410 10122 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:16:59.643963 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:16:59.579329 11785 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.566347 11806 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:59.623188 10197 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.631s	user 0.020s	sys 0.003s
I20260709 13:16:59.651153 11883 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.646654 11853 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:59.641379 11901 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.632230 11859 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.640245 11810 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.674499 11738 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:16:59.614718 11770 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:16:59.645969 11797 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.611893 11828 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:16:59.685721 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 225944us
I20260709 13:16:59.759156 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:16:59.785907 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1258) took 2154 ms (client timeout 1636 ms). Trace:
I20260709 13:16:59.790323 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:16:59.658888 11892 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.733188 10056 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:16:59.795066 10124 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.780870 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 946) took 2350 ms. Trace:
I20260709 13:16:59.820030 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:59.658612 11849 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:16:59.830951 10020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.018s	sys 0.078s
W20260709 13:16:59.831828 10020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.018s	sys 0.078s
W20260709 13:16:59.832314 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 674834us
I20260709 13:16:59.832885 10124 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:16:59.820214 11738 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:59.762485 11903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.028s	sys 0.068s
I20260709 13:16:59.826378 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:59.770279 10197 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:16:59.824753 10050 rpcz_store.cc:276] 0709 13:16:57.429903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.553306 (+123403us) service_pool.cc:224] Handling call
0709 13:16:57.581646 (+ 28340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.685341 (+1103695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.685374 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.685425 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:58.685462 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.780419 (+1094957us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400369054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:16:59.780496 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.780713 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537724288}
I20260709 13:16:59.783870 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:59.818364 11805 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.713267 11488 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.758174 10200 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:16:59.684983 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:16:59.825534 11797 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.769891 11785 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.712435 11758 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.136s
I20260709 13:16:59.837972 11666 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.846494 11903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.028s	sys 0.075s
I20260709 13:16:59.844539 11738 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.900638 10124 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:16:59.900388 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.845013 11899 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:59.908782 10051 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:16:59.908450 10124 pending_rounds.cc:85] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 1
I20260709 13:16:59.909562 10124 pending_rounds.cc:107] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:16:59.769603 11855 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.910169 10124 raft_consensus.cc:2889] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:59.827405 11810 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:59.817698 10055 rpcz_store.cc:269] 0709 13:16:57.631364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.811668 (+180304us) service_pool.cc:224] Handling call
0709 13:16:59.785527 (+1973859us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b5400235ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.785790 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2136153856}
I20260709 13:16:59.712775 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:59.847886 11883 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:16:59.903072 11917 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 13:16:59.846155 11849 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:16:59.650873 10899 rpcz_store.cc:276] 0709 13:16:58.024893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.030133 (+  5240us) service_pool.cc:224] Handling call
0709 13:16:59.100222 (+1070089us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:16:59.100555 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.563261 (+462706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.597328 (+ 34067us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43941760}
I20260709 13:16:59.854508 11606 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:16:59.920462 10124 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:16:59.911492 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:59.932163 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:16:59.921913 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 311215us
I20260709 13:16:59.900560 10893 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:16:59.933815 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:16:59.934983 10050 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:16:59.913729 11899 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.937098 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:16:59.913594 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1290) took 2498 ms (client timeout 1792 ms). Trace:
I20260709 13:16:59.942505 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:59.943742 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:59.944419 10051 raft_consensus.cc:2368] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5.
I20260709 13:16:59.921720 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2413) took 2298 ms. Trace:
I20260709 13:16:59.948843 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.951546 10096 leader_election.cc:400] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5.
I20260709 13:16:59.951843 10124 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:59.952164 10096 leader_election.cc:403] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:16:59.927019 11883 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.908450 11666 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.925866 11632 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:16:59.953085 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:16:59.938071 11810 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.942837 11859 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:59.959636 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.954630 11828 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:16:59.967016 11666 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.967823 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 955) took 1031 ms. Trace:
I20260709 13:16:59.949699 11853 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.970031 10129 rpcz_store.cc:276] 0709 13:16:58.936693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.609442 (+672749us) service_pool.cc:224] Handling call
0709 13:16:59.627356 (+ 17914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.652895 (+ 25539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.652948 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.653599 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.905981 (+252382us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:16:59.906022 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.906100 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.967542 (+ 61442us) raft_consensus.cc:1685] finished
0709 13:16:59.967563 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.967749 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18433,"raft.queue_time_us":23079,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":12163,"thread_start_us":225558,"threads_started":1,"wal-append.queue_time_us":285542}
I20260709 13:16:59.972607 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:16:59.974105 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:59.974462 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2417) took 2221 ms (client timeout 1602 ms). Trace:
W20260709 13:16:59.956154 10020 connection.cc:256] RPC call timeout handler was delayed by -1.6999821889999112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:16:59.938705 11488 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:16:59.968581 11905 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:16:59.975688 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1292) took 2569 ms (client timeout 2225 ms). Trace:
I20260709 13:16:59.968190 11859 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:16:59.976136 10202 rpcz_store.cc:269] 0709 13:16:57.406143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.406405 (+   262us) service_pool.cc:224] Handling call
0709 13:16:59.975458 (+2569053us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54001ef5d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.975575 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523491584}
I20260709 13:16:59.938542 11899 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:59.977031 11859 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.977880 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:16:59.978147 10202 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:16:59.978739 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 129) took 1551 ms. Trace:
I20260709 13:16:59.979203 10898 rpcz_store.cc:276] 0709 13:16:58.427892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.092382 (+664490us) service_pool.cc:224] Handling call
0709 13:16:59.977710 (+885328us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:16:59.978663 (+   953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5072128}
I20260709 13:16:59.835870 11892 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:16:59.975028 10203 rpcz_store.cc:269] 0709 13:16:57.753503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.764309 (+ 10806us) service_pool.cc:224] Handling call
0709 13:16:59.974242 (+2209933us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.974369 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724467968}
I20260709 13:16:59.980626 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2400) took 1777 ms. Trace:
I20260709 13:16:59.981832 10203 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:16:59.981086 10127 rpcz_store.cc:276] 0709 13:16:58.203006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.203412 (+   406us) service_pool.cc:224] Handling call
0709 13:16:59.978957 (+1775545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.979409 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.979445 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:59.979493 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:59.979530 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.979595 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.980424 (+   829us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54002ef250. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.980553 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":179,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":4082315136}
I20260709 13:16:59.982297 11892 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:16:59.982582 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2430) took 2028 ms. Trace:
W20260709 13:16:59.951629 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 265915us
I20260709 13:16:59.985561 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:16:59.937563 11849 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.961004 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:16:59.979550 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1253) took 2358 ms (client timeout 2989 ms). Trace:
I20260709 13:16:59.941040 11805 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:16:59.987246 11805 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:16:59.987085 10056 rpcz_store.cc:269] 0709 13:16:57.621070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.082820 (+1461750us) service_pool.cc:224] Handling call
0709 13:16:59.979341 (+896521us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54003cfad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.979485 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783314304}
I20260709 13:16:59.988695 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.988415 11805 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.956234 11883 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.989424 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.989934 11805 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:16:59.990134 10125 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:16:59.957333 11847 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.007s
I20260709 13:16:59.991413 11847 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:16:59.957113 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:16:59.960100 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2414) took 2314 ms. Trace:
I20260709 13:16:59.997215 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:16:59.999850 10201 rpcz_store.cc:276] 0709 13:16:57.645797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.597449 (+1951652us) service_pool.cc:224] Handling call
0709 13:16:59.620546 (+ 23097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.621037 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.621080 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.626807 (+  5727us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.815442 (+188635us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:59.815491 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.815575 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.959795 (+144220us) raft_consensus.cc:1685] finished
0709 13:16:59.959825 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.960020 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6226,"raft.queue_time_us":1208,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":15529,"thread_start_us":115944,"threads_started":1,"wal-append.queue_time_us":159166}
I20260709 13:16:59.978749 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2426) took 2180 ms. Trace:
I20260709 13:17:00.001573 10124 raft_consensus.cc:2376] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6. Re-sending same reply.
I20260709 13:17:00.002126 10202 rpcz_store.cc:276] 0709 13:16:57.798312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.977363 (+2179051us) service_pool.cc:224] Handling call
0709 13:16:59.977739 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.978503 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.978673 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.980533 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.003719 11613 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:00.006412 10898 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:16:59.982887 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:00.010599 11845 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:00.045838 11845 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:00.047225 11845 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:16:59.996891 11488 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:00.049589 11488 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:00.050038 11488 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.968806 10050 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:00.051028 11838 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:00.052110 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.054685 11740 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:00.068720 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:16:59.995543 10203 rpcz_store.cc:276] 0709 13:16:57.954113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.981028 (+2026915us) service_pool.cc:224] Handling call
0709 13:16:59.981354 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.982293 (+   939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.982513 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:59.997252 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 954) took 3536 ms (client timeout 2162 ms). Trace:
I20260709 13:17:00.069780 10054 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:00.070187 10123 rpcz_store.cc:269] 0709 13:16:56.460425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.866747 (+406322us) service_pool.cc:224] Handling call
0709 13:16:59.978819 (+3112072us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002ef254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:16:59.997134 (+ 18315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700113664}
I20260709 13:16:59.998320 11638 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:59.975035 10129 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:16:59.989740 10056 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:00.073542 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.074510 10056 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:00.075863 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 360) took 2137 ms. Trace:
I20260709 13:17:00.076308 11638 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:17:00.077685 11638 ts_tablet_manager.cc:1403] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 6.428s	user 0.091s	sys 0.065s
I20260709 13:17:00.019649 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:00.099979 10892 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.984s	user 0.011s	sys 0.002s
I20260709 13:17:00.100793 10892 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:00.102257 10893 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:00.103709 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.103935 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.105820 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.109218 10053 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:00.021170 11912 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:16:59.959306 11606 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5.
W20260709 13:17:00.024740 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1259) took 2392 ms (client timeout 1943 ms). Trace:
W20260709 13:17:00.150280 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 489451us
W20260709 13:17:00.150607 10049 rpcz_store.cc:269] 0709 13:16:57.632184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.808537 (+176353us) service_pool.cc:224] Handling call
0709 13:17:00.024389 (+2215852us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b540010c2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:00.024582 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586761600}
W20260709 13:16:59.941340 10197 rpcz_store.cc:269] 0709 13:16:57.405528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.421164 (+ 15636us) service_pool.cc:224] Handling call
0709 13:16:59.904312 (+2483148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:00.000838 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.172976 10891 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:00.173784 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 366) took 1077 ms. Trace:
I20260709 13:17:00.174261 10891 rpcz_store.cc:276] 0709 13:16:59.096749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.948075 (+851326us) service_pool.cc:224] Handling call
0709 13:17:00.173683 (+225608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:59.957593 11632 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:00.176370 11632 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:00.177901 11632 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:16:59.993854 11847 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:00.026971 10897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 362) took 2070 ms (client timeout 1733 ms). Trace:
W20260709 13:17:00.179884 10897 rpcz_store.cc:269] 0709 13:16:57.957153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.982928 (+ 25775us) service_pool.cc:224] Handling call
0709 13:17:00.026645 (+2043717us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:00.026853 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147488128}
I20260709 13:17:00.181552 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.182272 10897 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:17:00.028424 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.076618 10896 rpcz_store.cc:276] 0709 13:16:57.937925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.942486 (+  4561us) service_pool.cc:224] Handling call
0709 13:17:00.074584 (+2132098us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:00.074914 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:00.075240 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:00.075286 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:00.075355 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:00.075416 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.075528 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.075726 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45090688}
I20260709 13:17:00.184173 10055 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:00.184985 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:00.028631 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:00.186301 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 128) took 1773 ms (client timeout 1943 ms). Trace:
I20260709 13:17:00.186533 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:00.029268 10052 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.256s	user 0.013s	sys 0.000s
I20260709 13:16:59.951377 10200 rpcz_store.cc:276] 0709 13:16:57.623389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.605645 (+1982256us) service_pool.cc:224] Handling call
0709 13:16:59.676163 (+ 70518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.904059 (+227896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.918196 (+ 14137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:00.032650 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 361) took 2083 ms. Trace:
I20260709 13:17:00.056371 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 958) took 1118 ms. Trace:
I20260709 13:16:59.970988 11806 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:00.058105 10125 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:16:59.947731 11770 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:00.059741 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.187990 10130 rpcz_store.cc:276] 0709 13:16:58.937954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.566569 (+628615us) service_pool.cc:224] Handling call
0709 13:17:00.056123 (+489554us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b540036b354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:00.056287 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67060096}
I20260709 13:17:00.198071 10047 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:00.034900 11899 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:00.201527 11899 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:00.060374 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.080541 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.170814 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2411) took 2608 ms. Trace:
I20260709 13:17:00.204142 10204 rpcz_store.cc:276] 0709 13:16:57.562065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.373110 (+1811045us) service_pool.cc:224] Handling call
0709 13:16:59.373593 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:59.390032 (+ 16439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:59.390074 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.572503 (+182429us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.915419 (+342916us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:16:59.915468 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.915547 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:00.170491 (+254944us) raft_consensus.cc:1685] finished
0709 13:17:00.170524 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.170720 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":294193,"threads_started":1,"wal-append.queue_time_us":570705}
I20260709 13:17:00.185575 10896 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:00.187757 10900 rpcz_store.cc:276] 0709 13:16:57.949396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.949776 (+   380us) service_pool.cc:224] Handling call
0709 13:16:57.950205 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:58.676277 (+726072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.676321 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.676612 (+   291us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:16:58.676663 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:00.032310 (+1355647us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b540064e1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:00.032366 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:00.032559 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1669062016}
W20260709 13:17:00.186756 10894 rpcz_store.cc:269] 0709 13:16:58.412742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.079823 (+667081us) service_pool.cc:224] Handling call
0709 13:17:00.028447 (+948624us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:00.186223 (+157776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36642176}
I20260709 13:17:00.197103 10052 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:00.105862 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.213218 10054 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:00.243497 11845 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:00.219545 11859 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:00.226943 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:00.226050 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:00.232451 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4110821890001262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24230000000000002s for a response
I20260709 13:17:00.260488 10898 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
W20260709 13:17:00.247550 10168 connection.cc:256] RPC call timeout handler was delayed by -1.7883642360000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
I20260709 13:17:00.248250 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.261761 10204 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:00.236418 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:00.260679 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0886349439999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 13:17:00.262593 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0148349440000857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:17:00.210963 11858 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.002s
I20260709 13:17:00.264557 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:00.267904 10202 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:00.268151 11532 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:00.268680 11532 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:00.278985 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.217869 11901 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:00.934186 10047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.006s
W20260709 13:17:00.983834 10047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.012s	sys 0.007s
I20260709 13:17:00.313376 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:00.892788 10128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.001s	sys 0.017s
I20260709 13:17:00.985569 10053 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
W20260709 13:17:00.986048 10128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.001s	sys 0.021s
I20260709 13:17:00.254779 11912 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:00.656697 11875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.008s	sys 0.015s
W20260709 13:17:00.987712 11875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.013s	sys 0.019s
I20260709 13:17:00.987568 11912 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:00.366079 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:00.990240 10056 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:00.329066 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 66161.3us
W20260709 13:17:00.992420 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 628861us
I20260709 13:17:00.992681 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:00.417410 11891 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:00.993659 11891 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:00.533725 11657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.008s	sys 0.012s
I20260709 13:17:00.995586 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:00.996381 11657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.009s	sys 0.022s
I20260709 13:17:00.996976 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 965) took 1038 ms. Trace:
I20260709 13:17:00.995692 11891 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:00.372202 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:00.980692 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.997937 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:00.833101 11861 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.014s	sys 0.015s
I20260709 13:17:01.001647 10122 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:01.001775 11861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.008s	user 0.014s	sys 0.019s
I20260709 13:17:01.002091 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.000315 11891 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:00.383991 11899 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.003739 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.388065 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.010257 10204 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:17:01.003842 11899 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.019759 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:01.021608 11488 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.971s	user 0.014s	sys 0.020s
I20260709 13:17:00.997459 10125 rpcz_store.cc:276] 0709 13:16:59.959139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.995211 (+1036072us) service_pool.cc:224] Handling call
0709 13:17:00.996892 (+  1681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.022228 11899 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:00.314411 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.023196 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.023515 11488 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.023164 10049 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:01.025811 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:00.915249 11798 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.006s	sys 0.011s
W20260709 13:17:01.004045 11738 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.082s	user 0.016s	sys 0.000s
I20260709 13:17:00.049783  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:01.027348 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.028167 10049 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:01.028334 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1287) took 3623 ms. Trace:
I20260709 13:17:01.019862 10122 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.021565 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.029278 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:01.015853 10896 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.000s
I20260709 13:17:01.029333 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 967) took 1067 ms. Trace:
I20260709 13:17:01.030851 10896 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:01.031102 10122 rpcz_store.cc:276] 0709 13:16:59.961516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.997591 (+1036075us) service_pool.cc:224] Handling call
0709 13:17:01.029249 (+ 31658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.024118 10125 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:00.356738 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.032231  9951 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 3, term changed from 2 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to <none>, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:01.033234 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.034111 10203 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.034165 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.021428 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.026960 11738 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.031497 11532 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.035795 10055 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:17:01.036247 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.037530 11738 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:01.033087 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.038993 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.039942 10125 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.040907 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.041793 10125 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:00.986992 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.044008 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.045128 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2432) took 1093 ms. Trace:
I20260709 13:17:01.045733 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.045691 10129 rpcz_store.cc:276] 0709 13:16:59.951672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.043638 (+1091966us) service_pool.cc:224] Handling call
0709 13:17:01.045031 (+  1393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35456}
I20260709 13:17:01.047652 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.049039 10129 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.050585 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.051538 10129 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:00.993698 10900 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:00.597419 10126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.004s	sys 0.049s
W20260709 13:17:01.057763 10126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.004s	sys 0.053s
I20260709 13:17:01.052348 10053 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:01.038780 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.058998 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:01.052917 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.028926 10198 rpcz_store.cc:276] 0709 13:16:57.404601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.404751 (+   150us) service_pool.cc:224] Handling call
0709 13:16:57.405146 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:57.643475 (+238329us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:58.040063 (+396588us) log.cc:844] Serialized 14 byte log entry
0709 13:16:58.426103 (+386040us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:58.426195 (+    92us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:16:58.426281 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.027901 (+2601620us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54001fed54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:01.028002 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.028236 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":979290,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":92,"raft.queue_time_us":147,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":2681,"spinlock_wait_cycles":547294464,"thread_start_us":741814,"threads_started":2,"wal-append.queue_time_us":689368,"wal-append.run_cpu_time_us":6481,"wal-append.run_wall_time_us":1346794}
I20260709 13:17:01.036974 10048 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.055212 11875 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:01.055347 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.055517 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:01.055969 11798 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.006s	sys 0.012s
I20260709 13:17:01.023315 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.044001 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.031735 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 305) took 1074 ms. Trace:
I20260709 13:17:01.019794 11793 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:01.080312 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:00.277038 11837 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:01.059195 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.059443 10049 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:01.060433 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 956) took 2123 ms. Trace:
I20260709 13:17:01.011752 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.042747 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.086295 10054 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.068843 10129 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:01.036926 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1255) took 3415 ms. Trace:
I20260709 13:17:01.070537 11879 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:01.086004 10126 rpcz_store.cc:276] 0709 13:16:58.937141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.610008 (+672867us) service_pool.cc:224] Handling call
0709 13:16:59.610398 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.610666 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:16:59.914524 (+303858us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.060053 (+1145529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.060111 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:01.060172 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.060255 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.060376 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":664518,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":1679,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":3769,"thread_start_us":1429271,"threads_started":2}
I20260709 13:17:00.315348 10893 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:00.592911 11853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.014s	sys 0.042s
I20260709 13:17:01.072091 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 71) took 1031 ms. Trace:
I20260709 13:17:01.071918 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:00.622712 11849 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.000s	sys 0.023s
W20260709 13:17:00.258693 10052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1245) took 2693 ms (client timeout 2198 ms). Trace:
I20260709 13:17:01.079520 10205 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:01.104135 11632 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.926s	user 0.006s	sys 0.010s
I20260709 13:17:01.081416 10198 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:01.085130 11837 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:00.354077 11838 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:00.313376 11859 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:00.734043 11805 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.088968 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.136560 11805 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.137806 10125 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.138579 10122 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:01.138815 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.140231 10125 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:01.140739 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 969) took 1178 ms. Trace:
I20260709 13:17:01.026448 11874 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.091801 10894 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:01.141625 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 298) took 1289 ms. Trace:
I20260709 13:17:01.076599 10127 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:01.141937 10894 rpcz_store.cc:276] 0709 13:16:59.852119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.355029 (+502910us) service_pool.cc:224] Handling call
0709 13:17:01.087177 (+732148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.141341 (+ 54164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.141557 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.142495 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2433) took 1180 ms. Trace:
I20260709 13:17:01.142925 10127 rpcz_store.cc:276] 0709 13:16:59.962178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.043638 (+1081460us) service_pool.cc:224] Handling call
0709 13:17:01.142421 (+ 98783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34432}
I20260709 13:17:01.085484 10053 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:17:01.104175 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.104562 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.071457 11875 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:01.091876 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:01.146703 10053 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:01.146857 10051 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:17:01.147269 10895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 127) took 2778 ms (client timeout 3149 ms). Trace:
I20260709 13:17:01.108249 10900 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.147877 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1262) took 3502 ms. Trace:
I20260709 13:17:01.110443 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.080118 10896 rpcz_store.cc:276] 0709 13:16:59.957038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.184599 (+227561us) service_pool.cc:224] Handling call
0709 13:17:01.031641 (+847042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.083822 10897 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:01.079790 10050 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:01.058998 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.162038 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 97) took 1327 ms. Trace:
I20260709 13:17:01.122256 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.162652 10050 rpcz_store.cc:276] 0709 13:16:59.834677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.054973 (+1220296us) service_pool.cc:224] Handling call
0709 13:17:01.161946 (+106973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.163141 10198 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.071798 11532 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:17:01.140426 11805 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.084060 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.107901 10124 rpcz_store.cc:276] 0709 13:17:00.040794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.055187 (+1014393us) service_pool.cc:224] Handling call
0709 13:17:01.072018 (+ 16831us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.164098 10203 raft_consensus.cc:2368] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5.
W20260709 13:17:01.107625 11853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.014s	sys 0.047s
I20260709 13:17:01.148346 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1261) took 3507 ms. Trace:
I20260709 13:17:00.322932 11858 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.149516 10129 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:01.148336 10051 rpcz_store.cc:276] 0709 13:16:57.645844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.146015 (+3500171us) service_pool.cc:224] Handling call
0709 13:17:01.146320 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.147542 (+  1222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.147800 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.141121 10125 rpcz_store.cc:276] 0709 13:16:59.962141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.139824 (+1177683us) service_pool.cc:224] Handling call
0709 13:17:01.140090 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.140504 (+   414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.140657 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.118155 11513 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.156702 11901 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:01.122191 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 302) took 1178 ms. Trace:
W20260709 13:17:01.157083 11883 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.167s	user 0.008s	sys 0.006s
W20260709 13:17:01.109790 11849 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.003s	sys 0.030s
I20260709 13:17:01.159900 10891 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:01.160772 10864 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.631s (SENT)
I20260709 13:17:01.207386 10196 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:01.207628 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46918.4us
W20260709 13:17:01.147918 10895 rpcz_store.cc:269] 0709 13:16:58.369006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.087081 (+718075us) service_pool.cc:224] Handling call
0709 13:17:01.091655 (+2004574us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:01.147169 (+ 55514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17018240}
I20260709 13:17:01.162127 10202 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:17:01.164899 10053 rpcz_store.cc:276] 0709 13:16:57.640774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.145887 (+3505113us) service_pool.cc:224] Handling call
0709 13:17:01.146298 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.147979 (+  1681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.148257 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.168848 10051 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.158259 11488 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.125463 10200 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:01.118042 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.172525 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.283010 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.284891 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.173646 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.170763 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.149456 11837 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.166632 11793 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:01.177913 11513 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.258721 11858 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:01.206281 11861 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.264446 11912 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:01.322458 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.268047 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.269213 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.093019 10055 rpcz_store.cc:276] 0709 13:16:57.621568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.034831 (+3413263us) service_pool.cc:224] Handling call
0709 13:17:01.035326 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.036516 (+  1190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.036842 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.252267 10050 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:01.119516 10052 rpcz_store.cc:269] 0709 13:16:57.563560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.916611 (+353051us) service_pool.cc:224] Handling call
0709 13:17:00.256624 (+2340013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.276105 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:01.278558 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 298041us
I20260709 13:17:01.294195 10056 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.295805 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.350313 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 951) took 1124 ms. Trace:
I20260709 13:17:01.297111 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.300757 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.350757 10056 rpcz_store.cc:276] 0709 13:17:00.226100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.365656 (+139556us) service_pool.cc:224] Handling call
0709 13:17:01.350206 (+984550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.351589 10053 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.352479 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1264) took 3685 ms. Trace:
I20260709 13:17:01.325637 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.266845 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.352923 10050 rpcz_store.cc:276] 0709 13:16:57.667071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.163756 (+3496685us) service_pool.cc:224] Handling call
0709 13:17:01.251980 (+ 88224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.352124 (+100144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.352399 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.340267 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.353835 10124 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:01.354816 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.356180 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:01.357893 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:01.356210 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 363587us
I20260709 13:17:01.355592 10127 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.208564 11883 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.340713 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.344730 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.362183 10125 raft_consensus.cc:2376] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5. Re-sending same reply.
I20260709 13:17:01.266616 11879 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:01.180785 10893 rpcz_store.cc:276] 0709 13:16:59.940451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.105538 (+165087us) service_pool.cc:224] Handling call
0709 13:17:01.118386 (+1012848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.349205 10202 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:01.370045 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.370635 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.370986 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:01.371793 10051 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:01.372054 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.372763 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:01.376708 10126 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.365978 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.372619 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1266) took 3658 ms. Trace:
I20260709 13:17:01.380414 10051 rpcz_store.cc:276] 0709 13:16:57.714394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.168199 (+3453805us) service_pool.cc:224] Handling call
0709 13:17:01.168728 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.372302 (+203574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.372538 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:01.363903 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 304963us
I20260709 13:17:01.383531 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:01.385643 10056 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:01.391973 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.397246 10896 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:01.398380 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:01.398890 11513 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:01.350493 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2436) took 1096 ms. Trace:
I20260709 13:17:01.399885 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:01.400321 10200 rpcz_store.cc:276] 0709 13:17:00.252823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.256156 (+  3333us) service_pool.cc:224] Handling call
0709 13:17:01.084339 (+828183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.297749 (+213410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.348612 (+ 50863us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.401803 11632 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:01.402109 10200 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:01.402261 11632 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:01.402649 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2424) took 3612 ms (client timeout 1584 ms). Trace:
I20260709 13:17:01.402765 11632 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.335994 10048 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:01.399811 11702 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:01.404603 10199 rpcz_store.cc:269] 0709 13:16:57.789893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.804372 (+ 14479us) service_pool.cc:224] Handling call
0709 13:17:01.402387 (+3598015us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.402555 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668894464}
I20260709 13:17:01.409593 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 132) took 2905 ms. Trace:
I20260709 13:17:01.395099 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:01.412335 11798 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:01.412714 11915 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:01.412899 11798 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.394163 11879 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.415256 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:01.416100 11613 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:01.413267 11915 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:01.417440 11926 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:17:01.417711 11915 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.418406 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 131) took 2937 ms. Trace:
I20260709 13:17:01.362913 11488 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:01.423564 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2436) took 1168 ms. Trace:
I20260709 13:17:01.424099 10122 rpcz_store.cc:276] 0709 13:17:00.256859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.033868 (+777009us) service_pool.cc:224] Handling call
0709 13:17:01.423316 (+389448us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540036b854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.423491 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237953792}
W20260709 13:17:01.376549 11797 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.447s	user 0.013s	sys 0.003s
I20260709 13:17:01.425467 11797 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:01.427474 10892 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 295) took 3440 ms (client timeout 2106 ms). Trace:
W20260709 13:17:01.428162 10892 rpcz_store.cc:269] 0709 13:16:57.986751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.990240 (+  3489us) service_pool.cc:224] Handling call
0709 13:17:01.427106 (+3436866us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b540050f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.427311 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367331072}
I20260709 13:17:01.429435 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2426) took 1751 ms. Trace:
I20260709 13:17:01.429773 11901 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:01.393734 11849 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:01.431337 11933 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:01.431749 11849 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:01.431857 11933 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:01.343102 11861 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.432993 11933 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.430080 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.434513 11933 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:01.434990 10130 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.435680 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2434) took 1434 ms. Trace:
I20260709 13:17:01.436040 10130 rpcz_store.cc:276] 0709 13:17:00.001607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.429704 (+1428097us) service_pool.cc:224] Handling call
0709 13:17:01.435586 (+  5882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.436740 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 297) took 1591 ms. Trace:
I20260709 13:17:01.413257 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.439142 10052 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:01.440104 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1267) took 3719 ms. Trace:
I20260709 13:17:01.440616 10052 rpcz_store.cc:276] 0709 13:16:57.721017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.438204 (+3717187us) service_pool.cc:224] Handling call
0709 13:17:01.438663 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.439719 (+  1056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.440015 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.442028 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.438984 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.442795 10131 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.442910 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.443447 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 973) took 1395 ms. Trace:
I20260709 13:17:01.433347 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.443795 10124 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:01.443939 10131 rpcz_store.cc:276] 0709 13:17:00.047688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.438797 (+1391109us) service_pool.cc:224] Handling call
0709 13:17:01.443381 (+  4584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.444459 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2443) took 1080 ms. Trace:
I20260709 13:17:01.410212 10891 rpcz_store.cc:276] 0709 13:16:58.504268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.258463 (+1754195us) service_pool.cc:224] Handling call
0709 13:17:01.111394 (+852931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.266933 (+155539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.409275 (+142342us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54006f2f50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.409498 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923520256}
I20260709 13:17:01.444945 10201 rpcz_store.cc:276] 0709 13:17:00.364476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.364803 (+   327us) service_pool.cc:224] Handling call
0709 13:17:01.433117 (+1068314us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:01.444391 (+ 11274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6562560}
I20260709 13:17:01.459321 11887 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:01.460090 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.460026 11887 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.460743 11883 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.463145 10895 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:01.463388 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.465101 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.466467 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.466583 10197 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.413453 11798 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.467826 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.468727 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 972) took 1498 ms. Trace:
I20260709 13:17:01.469126 10123 rpcz_store.cc:276] 0709 13:16:59.970023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.467498 (+1497475us) service_pool.cc:224] Handling call
0709 13:17:01.468656 (+  1158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:01.470934 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2428) took 3544 ms (client timeout 2122 ms). Trace:
W20260709 13:17:01.471477 10205 rpcz_store.cc:269] 0709 13:16:57.926346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.110122 (+1183776us) service_pool.cc:224] Handling call
0709 13:17:01.470704 (+2360582us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54002175d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.470841 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3568802560}
I20260709 13:17:01.472395 11861 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:01.472831 11887 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:01.475605 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.476665 10126 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:01.418869 10897 rpcz_store.cc:276] 0709 13:16:58.480608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.196419 (+1715811us) service_pool.cc:224] Handling call
0709 13:17:01.069604 (+873185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.167426 (+ 97822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.418139 (+250713us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b5400604850. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.418334 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1980650240}
I20260709 13:17:01.477406 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.477869 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.478288 10124 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:01.479971 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:01.437304 10898 rpcz_store.cc:276] 0709 13:16:59.845639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.267712 (+422073us) service_pool.cc:224] Handling call
0709 13:17:01.436362 (+1168650us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b5400641fa8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cfa8ebc 00007fa27cfde08c 00007fa27cfc893b 00007fa27cfc83d2 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c
0709 13:17:01.436650 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057220992}
I20260709 13:17:01.481187 10893 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:01.442972 10051 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.405452 11875 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.484498 11875 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.485168 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.479746 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.486583 10130 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:01.432996 11849 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.489156 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:01.490079  9954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 113) took 7147 ms. Trace:
I20260709 13:17:01.398898 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:01.491487 10055 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:01.441303 11815 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:01.492238 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1274) took 2333 ms. Trace:
I20260709 13:17:01.492398 11815 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:01.493009 11815 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.493615 11883 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:01.494102 11883 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:01.494541 11883 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.495730 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2415) took 3816 ms. Trace:
I20260709 13:17:01.496165 10196 rpcz_store.cc:276] 0709 13:16:57.679809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.616360 (+1936551us) service_pool.cc:224] Handling call
0709 13:16:59.616766 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.207907 (+1591141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.495437 (+287530us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:01.495654 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3657496960}
I20260709 13:17:01.497756 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.498659 10054 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:01.499912 10054 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:01.500725 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1276) took 2288 ms. Trace:
I20260709 13:17:01.501245 10054 rpcz_store.cc:276] 0709 13:16:59.212228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.499367 (+2287139us) service_pool.cc:224] Handling call
0709 13:17:01.499714 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.500252 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.500611 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.502017 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:01.402112 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:01.450193 11797 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.480255 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.503530 10125 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.485373 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.506178 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 948) took 1281 ms. Trace:
I20260709 13:17:01.507503 11883 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.508541 11798 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:01.509023 10048 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:01.503553 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:01.509899 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1272) took 2354 ms. Trace:
I20260709 13:17:01.510476 11874 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:01.511009 11874 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.511683 11874 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:01.511735 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109715us
I20260709 13:17:01.512701 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:01.513547 11901 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:01.513942 11901 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:01.514348 11901 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.516320 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.517180 10125 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.429955 10128 rpcz_store.cc:276] 0709 13:16:59.678468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:59.705194 (+ 26726us) service_pool.cc:224] Handling call
0709 13:16:59.788478 (+ 83284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:00.078826 (+290348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:00.078890 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:00.079613 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:17:00.986648 (+907035us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:00.986707 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.407202 (+420495us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b18000b9920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:01.429059 (+ 21857us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:01.429133 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.429186 (+    53us) raft_consensus.cc:1685] finished
0709 13:17:01.429202 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.429364 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27489,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":164,"spinlock_wait_cycles":716170880,"thread_start_us":875282,"threads_started":1,"wal-append.queue_time_us":920072}
I20260709 13:17:01.265492 11638 raft_consensus.cc:359] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:01.517994 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.506726 10049 rpcz_store.cc:276] 0709 13:17:00.225007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.504992 (+1279985us) service_pool.cc:224] Handling call
0709 13:17:01.505343 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.505761 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.505793 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.505840 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:01.505875 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.505940 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.506100 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.525360  9951 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.520061 10051 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.526232 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.521152 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 99) took 1685 ms. Trace:
I20260709 13:17:01.527350 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.527737 10054 rpcz_store.cc:276] 0709 13:16:59.835577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.502339 (+1666762us) service_pool.cc:224] Handling call
0709 13:17:01.502722 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.502950 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.502976 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.503024 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:01.503054 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.520818 (+ 17764us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.521057 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.530717 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.524255 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.540143 10056 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:01.540606 10126 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:01.541296 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.534320 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.538358 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:01.542881 10122 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:17:01.539443 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.543519 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.541937 10126 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:01.541738 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.544278 10051 raft_consensus.cc:2376] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:01.528394 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.545001 10056 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.544924 10122 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:17:01.549610 10122 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:01.552742 10054 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:01.554504 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.555645 10054 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:01.492683 10055 rpcz_store.cc:276] 0709 13:16:59.158888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.490480 (+2331592us) service_pool.cc:224] Handling call
0709 13:17:01.490913 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.491960 (+  1047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.492172 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:01.581182 10019 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.548s (SENT)
I20260709 13:17:01.582336 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:01.559906 10126 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:17:01.525880 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.532424 11838 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:01.546689 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.533674 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1251) took 3913 ms. Trace:
I20260709 13:17:01.569173 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.570999 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:01.603403 10131 raft_consensus.cc:2376] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4. Re-sending same reply.
I20260709 13:17:01.603910 10056 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.603427 10051 raft_consensus.cc:2376] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3. Re-sending same reply.
I20260709 13:17:01.598708 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.605948 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 963) took 1336 ms. Trace:
I20260709 13:17:01.606593 10054 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:01.606617 10051 rpcz_store.cc:276] 0709 13:17:00.269787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.605104 (+1335317us) service_pool.cc:224] Handling call
0709 13:17:01.605415 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.605614 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.605640 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.605684 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:01.605714 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.605767 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.605883 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:01.603051 10095 connection.cc:256] RPC call timeout handler was delayed by -3.3296874499999376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2989s for a response
I20260709 13:17:01.555980 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:01.510399 10048 rpcz_store.cc:276] 0709 13:16:59.155335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.484714 (+2329379us) service_pool.cc:224] Handling call
0709 13:17:01.508558 (+ 23844us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:01.508811 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.509549 (+   738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.509818 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51252096}
I20260709 13:17:01.588565 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.591641 10126 pending_rounds.cc:85] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 1
I20260709 13:17:01.639755 10126 pending_rounds.cc:107] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:17:01.640189 10126 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:01.546532 11532 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.532195 11805 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:01.562489 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.538080 11513 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:01.537245 11657 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:01.533034 11638 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:17:01.621277 10049 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:01.661489 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.662129 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.549023 11887 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:17:01.538018 11912 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:01.601761 10047 rpcz_store.cc:276] 0709 13:16:57.620488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.206616 (+2586128us) service_pool.cc:224] Handling call
0709 13:17:00.207161 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:00.208344 (+  1183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:00.208386 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:00.246133 (+ 37747us) log.cc:844] Serialized 26 byte log entry
0709 13:17:00.984511 (+738378us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:01.532940 (+548429us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.533267 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.533357 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:01.533410 (+    53us) raft_consensus.cc:1685] finished
0709 13:17:01.533422 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.533589 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50727,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":145,"thread_start_us":1092499,"threads_started":2,"wal-append.queue_time_us":864890}
I20260709 13:17:01.638698 10054 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:01.652768 11805 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.641287 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.642357 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.653048 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.546705 11891 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:17:01.490533  9954 rpcz_store.cc:276] 0709 13:16:54.355327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.363332 (+  8005us) service_pool.cc:224] Handling call
0709 13:17:01.489984 (+7126652us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:54.711669 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:54.900072 (+188403us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:54.903524 (+  3452us) write_op.cc:435] Acquired schema lock
0709 13:16:54.903560 (+    36us) tablet.cc:662] Decoding operations
0709 13:16:55.097521 (+193961us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:55.106992 (+  9471us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:55.113780 (+  6788us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:16:55.182527 (+ 68747us) tablet.cc:701] Row locks acquired
0709 13:16:55.182550 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:16:55.195483 (+ 12933us) write_op.cc:270] Start()
0709 13:16:55.204859 (+  9376us) write_op.cc:276] Timestamp: P: 1783603015192537 usec, L: 0
0709 13:16:55.204894 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:16:57.685649 (+2480755us) log.cc:844] Serialized 1293 byte log entry
0709 13:16:58.473519 (+787870us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:16:59.817606 (+1344087us) op_driver.cc:464] REPLICATION: finished
0709 13:17:00.005285 (+187679us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277fffe28 00007fa27ba3a581 00007fa27ba39d58 00007fa27ba3e9f1 00007fa27ba3e9a9 00007fa27ba3e939 00007fa27ba3e901 00007fa27ba3db2d 000000000038f354 000000000038f199
0709 13:17:00.257771 (+252486us) write_op.cc:301] APPLY: starting
0709 13:17:00.257880 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:00.258048 (+   168us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:00.258073 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:00.313956 (+ 55883us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:00.314154 (+   198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:17:00.314201 (+    47us) write_op.cc:312] APPLY: finished
0709 13:17:00.314452 (+   251us) log.cc:844] Serialized 35 byte log entry
0709 13:17:00.314670 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.314808 (+   138us) write_op.cc:454] Released schema lock
0709 13:17:00.314855 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:00.315321 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:00.315354 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":715400,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":439505,"apply.run_cpu_time_us":2738,"apply.run_wall_time_us":57532,"num_ops":3,"prepare.queue_time_us":12462,"prepare.run_cpu_time_us":85062,"prepare.run_wall_time_us":5304711,"raft.queue_time_us":654500,"raft.run_cpu_time_us":57860,"raft.run_wall_time_us":119077,"replication_time_us":4610813,"spinlock_wait_cycles":5391159936,"thread_start_us":2689608,"threads_started":3,"wal-append.queue_time_us":1340468,"wal-append.run_cpu_time_us":108902,"wal-append.run_wall_time_us":2264863}]]}
I20260709 13:17:01.666742 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.719766 10053 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.674976 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:01.663566 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.620s (SENT)
I20260709 13:17:01.637321 11859 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:01.546227 11632 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:01.657353 11926 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:17:01.547564 11797 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:01.708904 10047 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:01.709887 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 965) took 1439 ms. Trace:
I20260709 13:17:01.738109 10054 rpcz_store.cc:276] 0709 13:17:00.270316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.637964 (+1367648us) service_pool.cc:224] Handling call
0709 13:17:01.638481 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.709459 (+ 70978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.709776 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:01.740496 10054 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:17:01.741103 10054 pending_rounds.cc:85] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 1
I20260709 13:17:01.741433 10054 pending_rounds.cc:107] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:01.741786 10054 raft_consensus.cc:2889] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:01.658006 11513 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.654978 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:01.713538 10124 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:01.753129 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2437) took 1495 ms. Trace:
I20260709 13:17:01.713599 10126 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:01.714902 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:01.659907 11657 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.660341 11932 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:01.668540 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.720697 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:01.764782 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 101271us
I20260709 13:17:01.766803 11797 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:01.770881 10125 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:01.721426 10201 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:17:01.701098 11845 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.428s	user 0.012s	sys 0.008s
I20260709 13:17:01.780028 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.664057 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:01.660364 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.692682 11838 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:01.733767 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:01.790462 10131 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:01.734905 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.702750 11912 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.763602 11657 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.653940 11532 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.715869 11805 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:01.753747 10197 rpcz_store.cc:276] 0709 13:17:00.258867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:00.263641 (+  4774us) service_pool.cc:224] Handling call
0709 13:17:01.752822 (+1489181us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001bf154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.753032 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363084544}
I20260709 13:17:01.755559 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.710158 11883 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:01.758841 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.801107 10205 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.730686 11859 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:01.753129 11513 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:01.774983 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 247244us
I20260709 13:17:01.783615 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.786203 10204 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:01.780860 10199 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:01.834685 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.749044 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:01.793406 10095 connection.cc:256] RPC call timeout handler was delayed by -2.8857513289999588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 13:17:01.836352 10095 connection.cc:256] RPC call timeout handler was delayed by -2.8794513289999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:17:01.836762 10095 connection.cc:256] RPC call timeout handler was delayed by -2.7831513289997929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:17:01.837152 10095 connection.cc:256] RPC call timeout handler was delayed by -2.6427513289997933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:17:01.837560 10095 connection.cc:256] RPC call timeout handler was delayed by -2.6427513289997933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:17:01.837934 10095 connection.cc:256] RPC call timeout handler was delayed by -2.6031513289999566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 13:17:01.838328 10095 connection.cc:256] RPC call timeout handler was delayed by -2.6031513289999566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 13:17:01.838740 10095 connection.cc:256] RPC call timeout handler was delayed by -2.5068513289998009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:17:01.787034 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 186834us
W20260709 13:17:01.815171 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 252782us
W20260709 13:17:01.840660 10095 connection.cc:256] RPC call timeout handler was delayed by -2.3070513289999326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260709 13:17:01.840818 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:01.841498 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2899513289999049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
I20260709 13:17:01.842090 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:01.842100 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2530513289998453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260709 13:17:01.845575 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2530513289998453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260709 13:17:01.846051 10095 connection.cc:256] RPC call timeout handler was delayed by -2.0865513289998034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24100000000000002s for a response
W20260709 13:17:01.846460 10095 connection.cc:256] RPC call timeout handler was delayed by -1.8470461629999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 13:17:01.847481 10095 connection.cc:256] RPC call timeout handler was delayed by -1.6868461629999274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
I20260709 13:17:01.844489 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:01.847631 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.821094 11879 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.849725 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:01.849748 10095 connection.cc:256] RPC call timeout handler was delayed by -1.4214513289998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3149s for a response
W20260709 13:17:01.850478 10095 connection.cc:256] RPC call timeout handler was delayed by -1.4214513289998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3149s for a response
I20260709 13:17:01.871311 10124 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:01.871731 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:01.859977 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.796389 11912 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.873848 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.835366 11859 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:01.736334 11638 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:01.813421 11838 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:01.796880 10896 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:01.878058 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.793757 11887 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:17:01.925028 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.790886 11932 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:17:01.927364 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.835168 11936 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:01.893666 10204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2448) took 1280 ms (client timeout 1670 ms). Trace:
I20260709 13:17:01.893925 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 964) took 1623 ms. Trace:
I20260709 13:17:01.875053 11845 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:01.877346 10047 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:17:01.937458 10204 rpcz_store.cc:269] 0709 13:17:00.613611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.021241 (+407630us) service_pool.cc:224] Handling call
0709 13:17:01.893445 (+872204us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b54002df0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:01.893590 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738700160}
I20260709 13:17:01.892812 10199 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:01.940565 10130 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:01.940403 10051 rpcz_store.cc:276] 0709 13:17:00.270037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.608873 (+1338836us) service_pool.cc:224] Handling call
0709 13:17:01.609324 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.609613 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:01.792273 (+182660us) log.cc:844] Serialized 14 byte log entry
0709 13:17:01.893528 (+101255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:01.893590 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:01.893648 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.893735 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:01.893864 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42337,"prepare.queue_time_us":190019,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":9681,"thread_start_us":226818,"threads_started":2}
I20260709 13:17:01.941895 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.942829 10130 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:01.943504 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.944669 10199 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:01.860422 11738 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:17:01.960676 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 101253us
I20260709 13:17:01.934306 10200 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:01.912631 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.969911 10124 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:01.914780 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.913410 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.971088 10197 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:01.923558 10122 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:01.885900 11798 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.876703 11853 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:17:01.887418 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.982319 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:01.982861 10197 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:01.957697 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:01.958120 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:01.869431 11858 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:01.997629 10205 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:01.967381 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90235.9us
I20260709 13:17:01.970314 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:01.949740 10202 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:01.964037 11883 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:01.963991 11838 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:01.985267 11887 consensus_queue.cc:1243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:17:01.986542 11941 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:17:01.986820 10200 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:01.928079 11947 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 13:17:01.988000 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.022s	sys 0.018s
I20260709 13:17:01.991734 11793 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:02.035454 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:01.993315 11879 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:01.920447 11891 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:17:01.930431 11950 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:01.987406 11937 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:17:01.992286 11815 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:02.080248 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.022s	sys 0.018s
I20260709 13:17:02.080570 10126 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:02.072877 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 223502us
I20260709 13:17:02.082652 10205 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:02.019093 11933 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 13:17:01.980993 11845 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:01.996281 11738 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:01.965025 11936 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.294546 11912 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.366845 10204 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:02.300240 11953 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:02.370594 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:02.019852 11901 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.013s
W20260709 13:17:02.179834 10049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.022s
W20260709 13:17:02.338457 10865 leader_election.cc:336] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.061s (SENT)
W20260709 13:17:02.269836 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:02.376739 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 993217us
W20260709 13:17:02.387305 10049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.005s	sys 0.032s
W20260709 13:17:02.377310 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 415247us
I20260709 13:17:02.395190 10198 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:02.361766 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:02.396426 10124 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:02.395180 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:02.398213 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:02.398859 10122 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:02.362771 10130 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:02.377887 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 506355us
I20260709 13:17:02.387814 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:02.243523 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.275225 11887 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 13:17:02.142287 11532 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:02.141043 11632 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:02.411918 11970 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:17:02.079190 11858 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:02.417445 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:02.418078 10200 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.418778 11797 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:02.303745 11879 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:02.419553 11879 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:02.419752 11797 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:02.420985 10053 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:02.420594 11879 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.421125 11815 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:02.421891 11845 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.422441 11738 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.416697 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:02.211764 11874 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.699s	user 0.008s	sys 0.005s
I20260709 13:17:02.423729 11932 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:02.424302 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:02.424422 11874 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:02.301157 11956 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:17:02.086787 11793 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:02.426409 10129 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:02.426963 10130 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:17:02.427511 11874 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:02.187273 11837 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.851s	user 0.009s	sys 0.003s
I20260709 13:17:02.360139 10197 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.414685 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:02.415495 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77092.6us
W20260709 13:17:02.034953 11915 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.617s	user 0.001s	sys 0.019s
I20260709 13:17:02.374006 11798 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:02.399966 11958 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 13:17:02.448326 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.019459 11883 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.391685 11912 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:02.421922 11879 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.412657 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 299) took 2495 ms. Trace:
I20260709 13:17:02.090415 11944 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.366s
W20260709 13:17:02.455037 11943 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:17:02.245559 11938 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.211s
I20260709 13:17:02.400121 11853 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:17:02.401703 11901 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.442246 11837 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.458127 11798 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:02.393309 11950 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.388171 11948 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:02.460630 11632 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:02.645632 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:02.392917 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:02.464933 10894 rpcz_store.cc:276] 0709 13:16:59.904020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.143400 (+1239380us) service_pool.cc:224] Handling call
0709 13:17:01.143919 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.144388 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.144421 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.145165 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.561760 (+416595us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:01.584708 (+ 22948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.599671 (+ 14963us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.373074 (+773403us) raft_consensus.cc:1685] finished
0709 13:17:02.373108 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.399704 (+ 26596us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4277,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":173,"thread_start_us":358166,"threads_started":1,"wal-append.queue_time_us":519098}
I20260709 13:17:02.516011 11638 ts_tablet_manager.cc:1434] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 2.438s	user 0.012s	sys 0.011s
I20260709 13:17:02.647068 10054 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:02.648017 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1305) took 1105 ms. Trace:
I20260709 13:17:02.654260 10198 rpcz_store.cc:276] 0709 13:17:01.541916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.588252 (+ 46336us) service_pool.cc:224] Handling call
0709 13:17:02.647356 (+1059104us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:02.647684 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1737159936}
I20260709 13:17:02.658030 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 973) took 1301 ms. Trace:
I20260709 13:17:02.610818 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:02.615532 10051 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:17:02.672914 10051 pending_rounds.cc:85] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 1
W20260709 13:17:02.616761 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.010s
I20260709 13:17:02.673262 10050 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
W20260709 13:17:02.673754 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.007s	sys 0.011s
I20260709 13:17:02.618382 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 989) took 1103 ms. Trace:
I20260709 13:17:02.623860 10202 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:02.675422 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 955) took 2422 ms. Trace:
I20260709 13:17:02.677898 10055 rpcz_store.cc:276] 0709 13:17:00.253282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.596735 (+1343453us) service_pool.cc:224] Handling call
0709 13:17:01.597198 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.597513 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:01.864922 (+267409us) log.cc:844] Serialized 14 byte log entry
0709 13:17:02.674720 (+809798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:02.674838 (+   118us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:02.674952 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.675048 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.675348 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166289,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":52,"thread_start_us":1024634,"threads_started":2,"wal-append.queue_time_us":625108}
I20260709 13:17:02.623957 10203 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:02.625828 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:02.673354 10051 pending_rounds.cc:107] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:02.670770  9953 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 217) took 9349 ms. Trace:
I20260709 13:17:02.684007 10051 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.629042 10047 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:17:02.632711 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 217959us
I20260709 13:17:02.615533 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 974) took 1142 ms. Trace:
I20260709 13:17:02.645828 11632 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:02.600099 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 185716us
I20260709 13:17:02.604904 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 956) took 2347 ms. Trace:
W20260709 13:17:02.649478 11638 ts_tablet_manager.cc:1467] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Tablet startup took 2571ms
I20260709 13:17:02.660485  9951 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.715655  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:02.645718 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 228218us
W20260709 13:17:02.647923 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 254876us
I20260709 13:17:02.671535 10053 rpcz_store.cc:276] 0709 13:17:01.356014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.720729 (+364715us) service_pool.cc:224] Handling call
0709 13:17:01.721077 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.421463 (+700386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.657054 (+235591us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:02.657706 (+   652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609149696}
I20260709 13:17:02.676900 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 97) took 1109 ms. Trace:
I20260709 13:17:02.624380 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 952) took 2397 ms. Trace:
I20260709 13:17:02.676738 11532 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:02.418851 11858 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:02.533324 11875 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.048s	user 0.021s	sys 0.006s
I20260709 13:17:02.453049 11915 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.745990 11798 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.749974 11915 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:02.443003 11933 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:17:02.751884 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:02.753152 10894 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.692684 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:02.753989 11837 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:02.754508 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62297.5us
I20260709 13:17:02.755832 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:02.541240 11948 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:02.696022 11947 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.005s
I20260709 13:17:02.771342 11874 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:17:02.771535 11947 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.001s	sys 0.006s
I20260709 13:17:02.771968 11874 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:02.695642 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:02.698894 11513 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.823s	user 0.016s	sys 0.010s
I20260709 13:17:02.772574 11948 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:02.772991 10050 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:02.773141 11513 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:02.537961 11958 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.002s
I20260709 13:17:02.775753 10199 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:02.545709 11960 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.005s
I20260709 13:17:02.536917 11976 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.779374 11976 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:02.781314 11976 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.782014 11938 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:02.682866 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 87) took 1587 ms. Trace:
I20260709 13:17:02.778851 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 982) took 1266 ms. Trace:
I20260709 13:17:02.783360 10203 rpcz_store.cc:276] 0709 13:17:01.094934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.371299 (+276365us) service_pool.cc:224] Handling call
0709 13:17:02.682331 (+1311032us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:02.682753 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2873719040}
I20260709 13:17:02.675021 10129 rpcz_store.cc:276] 0709 13:17:01.514872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.650255 (+135383us) service_pool.cc:224] Handling call
0709 13:17:02.617748 (+967493us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:02.618095 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623843712}
I20260709 13:17:02.712136 10202 rpcz_store.cc:276] 0709 13:17:01.567896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.021142 (+453246us) service_pool.cc:224] Handling call
0709 13:17:02.021629 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.675480 (+653851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.676502 (+  1022us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:02.676799 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382815488}
I20260709 13:17:02.784555  9951 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:02.788014 11875 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:02.789474 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.705639 10056 rpcz_store.cc:276] 0709 13:17:00.256986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.604480 (+1347494us) service_pool.cc:224] Handling call
0709 13:17:01.604918 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.795864 (+190946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.795907 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.796561 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.968600 (+172039us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:01.969164 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.971376 (+  2212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.603295 (+631919us) raft_consensus.cc:1685] finished
0709 13:17:02.603591 (+   296us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.604345 (+   754us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":252840,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":16264,"thread_start_us":273792,"threads_started":2,"wal-append.queue_time_us":597430}
I20260709 13:17:02.725206 11815 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:02.762215 11901 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:02.785238 11859 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.857s	user 0.008s	sys 0.007s
I20260709 13:17:02.784451 11968 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.770690 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:02.704777 10123 rpcz_store.cc:276] 0709 13:17:01.473217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.510266 (+ 37049us) service_pool.cc:224] Handling call
0709 13:17:01.510650 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:01.510905 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:01.852550 (+341645us) log.cc:844] Serialized 14 byte log entry
0709 13:17:02.446257 (+593707us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b18000c24a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:17:02.614230 (+167973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:02.614425 (+   195us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:02.614634 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.614848 (+   214us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.615206 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176455,"prepare.queue_time_us":379708,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2966,"raft.queue_time_us":264045,"spinlock_wait_cycles":73937536,"thread_start_us":391572,"threads_started":2,"wal-append.queue_time_us":455312}
I20260709 13:17:02.797359 11837 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:02.804054 11982 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.808197 11837 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.713804 10052 rpcz_store.cc:276] 0709 13:17:00.226436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.515288 (+1288852us) service_pool.cc:224] Handling call
0709 13:17:01.515763 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.516082 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.516114 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.516708 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.722402 (+205694us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:01.732781 (+ 10379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:01.738125 (+  5344us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.623506 (+885381us) raft_consensus.cc:1685] finished
0709 13:17:02.623561 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.624071 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":191139,"threads_started":1,"wal-append.queue_time_us":612792}
I20260709 13:17:02.808450 11982 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.808812 11837 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.809082 11982 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.782953 11976 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.793205 11815 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:02.807899 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.684146  9953 rpcz_store.cc:276] 0709 13:16:53.321448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.321764 (+   316us) service_pool.cc:224] Handling call
0709 13:17:02.670343 (+9348579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:54.375220 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:54.375964 (+   744us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:54.376010 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:16:54.376035 (+    25us) tablet.cc:662] Decoding operations
0709 13:16:54.376883 (+   848us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:54.376968 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:54.376996 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:16:54.377548 (+   552us) tablet.cc:701] Row locks acquired
0709 13:16:54.377567 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:16:54.377732 (+   165us) write_op.cc:270] Start()
0709 13:16:54.377856 (+   124us) write_op.cc:276] Timestamp: P: 1783603014377685 usec, L: 0
0709 13:16:54.377879 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:16:54.472831 (+ 94952us) log.cc:844] Serialized 2074 byte log entry
0709 13:16:55.115898 (+643067us) op_driver.cc:464] REPLICATION: finished
0709 13:16:56.909138 (+1793240us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa27ba3aef0
0709 13:16:57.618580 (+709442us) write_op.cc:301] APPLY: starting
0709 13:16:57.623063 (+  4483us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:57.680161 (+ 57098us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:57.684886 (+  4725us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:58.130846 (+445960us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:58.386359 (+255513us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:16:58.430412 (+ 44053us) write_op.cc:312] APPLY: finished
0709 13:16:58.500256 (+ 69844us) log.cc:844] Serialized 47 byte log entry
0709 13:16:58.572741 (+ 72485us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:58.617078 (+ 44337us) write_op.cc:454] Released schema lock
0709 13:16:58.655638 (+ 38560us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:58.672918 (+ 17280us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:58.672954 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":459795,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":2448619,"apply.run_cpu_time_us":7239,"apply.run_wall_time_us":1076541,"num_ops":5,"prepare.queue_time_us":323342,"prepare.run_cpu_time_us":4518,"prepare.run_wall_time_us":167944,"replication_time_us":711905,"spinlock_wait_cycles":2426112,"thread_start_us":2632145,"threads_started":2}]]}
I20260709 13:17:02.812166 11953 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.811744 11815 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.772511 11874 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.812990 11953 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.804380 11849 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:02.805812 11968 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.807196 11859 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.798849 11513 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:02.812955 10054 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:02.778736 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:02.811726 11632 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.807590 11875 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:02.823557 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.824275 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:02.804396 11805 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.946s	user 0.012s	sys 0.008s
I20260709 13:17:02.839581 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:02.784089 10128 rpcz_store.cc:276] 0709 13:17:01.512727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.544162 (+ 31435us) service_pool.cc:224] Handling call
0709 13:17:01.548227 (+  4065us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:01.719616 (+171389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:01.719655 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:01.720215 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:17:01.914950 (+194735us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:01.954237 (+ 39287us) log.cc:844] Serialized 14 byte log entry
0709 13:17:02.246177 (+291940us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b1800335bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:02.393512 (+147335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.393609 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:02.777954 (+384345us) raft_consensus.cc:1685] finished
0709 13:17:02.778184 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:02.778575 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129101,"prepare.queue_time_us":42078,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":615085,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":22713,"spinlock_wait_cycles":70004096,"thread_start_us":437452,"threads_started":4,"wal-append.queue_time_us":225676}
I20260709 13:17:02.842806 11947 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:02.844249 11968 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:02.844950 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:02.839653 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 61221.1us
W20260709 13:17:02.707926 11638 ts_tablet_manager.cc:1469] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Trace:
0709 13:16:49.284715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.287272 (+  2557us) service_pool.cc:224] Handling call
0709 13:16:49.729037 (+441765us) ts_tablet_manager.cc:934] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
0709 13:16:50.818523 (+1089486us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:16:51.420734 (+602211us) tablet_metadata.cc:665] Metadata flushed
0709 13:16:51.420887 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:16:51.681164 (+260277us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:16:51.681747 (+   583us) tablet_replica.cc:206] Creating consensus instance
0709 13:16:51.682222 (+   475us) ts_tablet_manager.cc:989] Replica registered.
0709 13:16:51.682328 (+   106us) inbound_call.cc:177] Queueing success response
0709 13:16:53.649274 (+1966946us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:00.078239 (+6428965us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:00.928513 (+850274us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:01.162168 (+233655us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":375,"mutex_wait_us":1174,"raft.queue_time_us":5119,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":7178,"tablet-copy.queue_time_us":301038,"tablet-download-pool.queue_time_us":2391,"tablet-download-pool.run_cpu_time_us":1882,"tablet-download-pool.run_wall_time_us":1532265,"thread_start_us":162923,"threads_started":2}
I20260709 13:17:02.849653 11875 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:02.853008 11797 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:02.860124 10050 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:02.864315 11947 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:02.865263 11938 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.868446 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:02.871981 11968 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:02.872617 11968 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:02.873092 11968 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:02.872972 11947 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.873106 11972 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:02.877982 10051 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:02.883394 10051 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:02.885572 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1304) took 1343 ms. Trace:
I20260709 13:17:02.889544 11978 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.890123 11793 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:02.891705 11632 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:02.891024 11657 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.014s	user 0.013s	sys 0.013s
I20260709 13:17:02.896773 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1313) took 1304 ms. Trace:
I20260709 13:17:02.902581 11702 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:02.856531  9951 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 3, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:02.862561 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54845.2us
I20260709 13:17:02.935431 11657 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.936103 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:02.870211 11938 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.869521 11875 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.937103 11938 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.937398 10196 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:02.905593 10199 rpcz_store.cc:276] 0709 13:17:01.541648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.945710 (+404062us) service_pool.cc:224] Handling call
0709 13:17:01.946143 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:02.776537 (+830394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.885135 (+108598us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001bfb54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:02.885467 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1906446720}
I20260709 13:17:02.846896 11859 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:02.879433 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.900892 11901 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:02.902602 11968 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.933578  9951 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 2, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.963848 11968 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:02.964998 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85856.3us
I20260709 13:17:02.966388 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.968448 11932 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:02.969149 11932 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:02.969575 11932 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:02.971208 11936 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:02.972297 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 148278us
I20260709 13:17:02.929399 10201 rpcz_store.cc:276] 0709 13:17:01.592375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.873507 (+281132us) service_pool.cc:224] Handling call
0709 13:17:02.896292 (+1022785us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54002189d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:02.896546 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990829952}
I20260709 13:17:02.965391  9951 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.974545 11879 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:02.976243 11950 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:02.971861 10051 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:02.976328 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:02.984489 11797 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.937933 11875 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:02.926254 11978 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:03.000316 11532 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:03.000895 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:02.953639 11953 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:03.006990 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.009500 10894 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:03.010362 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.021782 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.012552 11793 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:02.869024 11805 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.025275 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1324) took 1311 ms. Trace:
I20260709 13:17:03.025772 11805 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:03.025935 10196 rpcz_store.cc:276] 0709 13:17:01.714247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.847581 (+133334us) service_pool.cc:224] Handling call
0709 13:17:03.024980 (+1177399us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002de1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:03.025205 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337594112}
I20260709 13:17:03.027058 10124 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:03.030555 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 76) took 1009 ms. Trace:
I20260709 13:17:03.031028 10124 rpcz_store.cc:276] 0709 13:17:02.021565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.417096 (+395531us) service_pool.cc:224] Handling call
0709 13:17:02.417432 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:03.027597 (+610165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.030143 (+  2546us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540023c3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:03.030439 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400974080}
W20260709 13:17:03.032742 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56695.2us
I20260709 13:17:03.037024 11879 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:03.037379 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:02.985713 11797 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:03.040616 11969 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:02.998821 11936 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.041502 10051 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.041572 11859 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:03.040616 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64476.4us
I20260709 13:17:03.048048 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:03.051815 11793 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:03.052191 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51454.2us
I20260709 13:17:03.054108 11883 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.055709 11849 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.057418 10055 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
W20260709 13:17:03.058887 10020 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.019266 11972 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:03.060379 11874 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.061023 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:03.061690 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.062175 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.063500 10123 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:03.064204 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 74) took 1056 ms. Trace:
I20260709 13:17:03.064733 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.065526 11532 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:03.066260 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.017727 11798 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.037686 11879 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:03.080659 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 88242.1us
I20260709 13:17:03.090709 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.112462 11879 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.112560 10124 raft_consensus.cc:2376] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4. Re-sending same reply.
I20260709 13:17:03.084403 11936 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.115918 11936 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:03.116752 11657 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:03.079962 11969 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:03.104143 10131 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.097872 10125 rpcz_store.cc:276] 0709 13:17:02.008033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:02.008238 (+   205us) service_pool.cc:224] Handling call
0709 13:17:02.008790 (+   552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:02.391723 (+382933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.391757 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:02.392407 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 13:17:02.997004 (+604597us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:03.001088 (+  4084us) log.cc:844] Serialized 14 byte log entry
0709 13:17:03.022391 (+ 21303us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b18000d8c40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:03.035180 (+ 12789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:03.037850 (+  2670us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.063507 (+ 25657us) raft_consensus.cc:1685] finished
0709 13:17:03.063629 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.063833 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":269026,"prepare.queue_time_us":331,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":337162,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":202,"spinlock_wait_cycles":20179200,"thread_start_us":533038,"threads_started":2,"wal-append.queue_time_us":402774}
W20260709 13:17:03.104692 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.105086 11532 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.080156 11849 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.082810 11859 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.107769 10896 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:03.109162 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 967) took 1843 ms. Trace:
I20260709 13:17:03.118903 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.111342 11972 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:03.122619 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f received from {username='slave'} at 127.0.0.1:50726
I20260709 13:17:03.147621 11859 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.148371 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f on tablet 806f35e66c234d569cbafd8b3981518f with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:03.148911 11972 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.151741 11982 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.148792 11657 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:17:03.157609 11657 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.127664 11849 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.130591 11798 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:03.077445 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45844.1us
I20260709 13:17:03.134516 10127 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:03.136556 11969 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.170403 11798 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.170986 10055 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:17:03.124605 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.170905 11815 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.172303 11946 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:03.172865 11946 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.139283 11858 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 13:17:03.173271 11946 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.173838 11815 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:03.138674 11978 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.141577 10049 rpcz_store.cc:276] 0709 13:17:01.266260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.664983 (+398723us) service_pool.cc:224] Handling call
0709 13:17:01.665580 (+   597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:02.245112 (+579532us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b48000030d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:17:02.390294 (+145182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:02.390340 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:02.390914 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:17:02.978042 (+587128us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:02.994379 (+ 16337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:02.999035 (+  4656us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:03.058467 (+ 59432us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b1800009700. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:03.108665 (+ 50198us) raft_consensus.cc:1685] finished
0709 13:17:03.108695 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:03.108932 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":225962,"raft.run_cpu_time_us":3173,"raft.run_wall_time_us":335352,"spinlock_wait_cycles":23186688,"thread_start_us":1150102,"threads_started":2,"wal-append.queue_time_us":429515}
I20260709 13:17:03.170254 11849 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.140729 10201 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:03.176308 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.177202 11978 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.177657 11989 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:03.179992 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:03.181308 11838 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:03.183601 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102546us
I20260709 13:17:03.184028 11912 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:03.184372 11838 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.184667 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:03.186724 11853 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:03.184868 11912 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.185236 11838 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.163335 11947 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:03.188411 11912 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.188850 11947 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.189733 11932 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.192723 11932 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.111296 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:03.166858 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 91908.3us
I20260709 13:17:03.166908 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:03.171258 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.175271 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50788.6us
I20260709 13:17:03.200183 10196 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:03.187585 11938 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.187372 11853 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:03.192368 11838 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.148787 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:03.193116 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.115973 11837 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.161010 11874 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.157816 11982 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.216099 11837 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:03.220201 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 71547.3us
I20260709 13:17:03.220659 11837 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:03.221124 11837 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.221717 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:03.222579 10125 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:03.223701 11797 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.226876 11874 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:03.227807 11874 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.230312 11874 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.181399 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:03.221486 11837 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.233654 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.203655 10051 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:17:03.207101 11853 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.215206 11657 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.245150 11657 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:03.231801 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50512.2us
I20260709 13:17:03.244995 11946 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.247522 11933 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:03.248957 11933 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:03.249434 11933 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.231990 11513 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:03.250613 11532 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.195714 11947 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:03.202648 11936 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:17:03.253738 11532 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.235090 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.226994 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.216053 10201 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:03.251566 11513 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.235302 10051 pending_rounds.cc:85] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:17:03.199018 10131 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:03.257229 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60273.1us
I20260709 13:17:03.269325 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.270082 11798 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.270428 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.271242 10055 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:03.271489 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:03.250619 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0487481499999376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 13:17:03.280481 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0046481499998663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
I20260709 13:17:03.281919 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.282151 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.282943 10051 pending_rounds.cc:107] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:17:03.283792 10051 raft_consensus.cc:2889] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:03.285634 10054 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:03.285914 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59050us
I20260709 13:17:03.286154 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.287078 10204 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4.
I20260709 13:17:03.302640 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:03.301542 10865 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.194s (SENT)
I20260709 13:17:03.285691 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.293493 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:03.287988 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:03.277045 11976 raft_consensus.cc:2764] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:03.310629 10199 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:03.278802 11798 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:03.320106 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:03.332566 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.336164 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:03.339020 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45886.1us
I20260709 13:17:03.338526 10196 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:03.340010 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.346890 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.347285 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.348272 10052 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:03.353012 11962 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:17:03.332868 11874 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:17:03.250633 11933 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.355962 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.343336 10053 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.366211 10054 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:03.319761 11946 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:03.369272 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67931us
I20260709 13:17:03.369426 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.370666 11879 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.371558 11991 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:17:03.374310 11993 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:17:03.375741 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.332047 11657 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:03.374480 11879 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:03.394387 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109521us
I20260709 13:17:03.360534 11883 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:03.362416 11838 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.364101 11887 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:03.287173 11513 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.401477 10050 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.406023 10055 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.407383 10047 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:03.419271 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 131426us
I20260709 13:17:03.428157 11996 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:17:03.431313 10196 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:03.410843 11883 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.455232 11858 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:03.455822 11883 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.462759 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.463346 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.464323 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.439296 10095 leader_election.cc:400] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5.
I20260709 13:17:03.440068 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.466619 10049 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:03.467768 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.468940 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.470041 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.470599 10049 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.447561 11933 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.471585 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.425954 11798 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.433028 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.451352 11938 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:03.463513 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:03.471707 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.473737 10054 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:03.475207 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.477033 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.385224 11875 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.481976 10054 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:03.483394 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.449463 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:03.454504 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:03.484826 10198 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3. Re-sending same reply.
I20260709 13:17:03.485261 11657 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:03.489532 11797 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.493561 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.494875 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:03.495316 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:03.495883 10202 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:03.495908 10196 raft_consensus.cc:2376] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5. Re-sending same reply.
W20260709 13:17:03.500697 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.500715 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 103427us
W20260709 13:17:03.503253 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48899us
I20260709 13:17:03.505128 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:03.506589 11657 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:03.507241 11657 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.509444 10054 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:03.517899 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:03.521656 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.523231 10050 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:03.528708 10047 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:03.525868 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:03.513293 11657 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.516606 11798 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.540560 10201 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:03.542861 10050 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:17:03.538015 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.548601 10196 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.548570 10198 raft_consensus.cc:2376] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4. Re-sending same reply.
I20260709 13:17:03.552160 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:03.558578 10019 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.580s (SENT)
I20260709 13:17:03.558817 10054 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:17:03.561430 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52240.8us
I20260709 13:17:03.572726 10053 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:03.574610 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 202061us
I20260709 13:17:03.580699 11837 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.581310 10047 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:17:03.583452 10053 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:03.583911 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:03.586302 10019 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.671s (SENT)
I20260709 13:17:03.556699 11875 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:03.559298 11936 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:03.589447 10051 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:03.591343 10054 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:03.591343 10050 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:03.594143 11837 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:03.597695 10019 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.057s (SENT)
W20260709 13:17:03.603909 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67750.1us
I20260709 13:17:03.590850 11887 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:03.598003 10048 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:03.571633 11976 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:17:03.609073 11976 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.609501 11976 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:03.611436 10198 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:03.599606 11936 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.613610 11936 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.599951 11912 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:03.615816 11912 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:03.618326 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.618469 11912 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:03.619563 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.621184 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:03.623144 10019 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.106s (SENT)
I20260709 13:17:03.623526 11978 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:03.623986 11978 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:03.624366 10019 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.239s (SENT)
I20260709 13:17:03.624660 11887 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:03.625135 11887 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.587960 11513 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:17:03.625662 11887 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.628377 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:03.624418 11978 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:03.629220 10052 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:03.610473 11976 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.633319 10894 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:17:03.625157 10019 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.423s (SENT)
W20260709 13:17:03.639376 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.643496 10049 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
W20260709 13:17:03.657233 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.649735 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.650326 10047 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:03.651880 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:03.648653 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.658367 10054 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.659997 10055 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:03.662935 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.663779 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.663661 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:03.661514 11879 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:03.667061 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.667655 11874 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:03.668229 11874 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:03.668648 11874 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:03.668927 10131 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:17:03.668010 10049 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.670540 10127 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.674808 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:03.676837 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48567.6us
I20260709 13:17:03.678278 11875 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:17:03.686949 11513 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:03.603263 11853 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:03.667191 11879 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:17:03.692730 10865 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.909s (SENT) [suppressed 90 similar messages]
I20260709 13:17:03.712925 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:03.719646 10049 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:03.724318 10019 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.880s (SENT)
I20260709 13:17:03.732450 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:03.721609 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:03.687626 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:03.678875 11838 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:17:03.692337 11513 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:03.738993 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 143229us
I20260709 13:17:03.741189 10196 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:03.703088 11879 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:03.744851 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52151.6us
I20260709 13:17:03.747850 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:03.746613 11912 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:03.757699 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82925.6us
W20260709 13:17:03.758276 10019 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.975s (SENT)
I20260709 13:17:03.744164 10896 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:03.794085 10865 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.758s (SENT)
W20260709 13:17:03.776194 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.744410 11513 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:03.795413 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:03.772956 10054 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:03.797223 10050 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:17:03.797178 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49457.6us
I20260709 13:17:03.753818 10894 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:17:03.795094 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:03.754786 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:03.747357 11849 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:17:03.801584 11858 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:03.815563 10019 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.000s (SENT)
I20260709 13:17:03.821883 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:03.824693 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:03.752321 11933 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:17:03.816452 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:03.807017 11798 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.850625 11938 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.859172 12010 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:03.859934 12010 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:03.860359 12010 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.838747 11805 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.867825 11805 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:03.844491 12002 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 13:17:03.873636 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58150.9us
W20260709 13:17:03.874267 10865 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.767s (SENT)
I20260709 13:17:03.851632 10052 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:03.886662 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 200921us
W20260709 13:17:03.889394 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 134280us
I20260709 13:17:03.907270 11805 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:03.906391 12011 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:03.885572 11912 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:03.931550 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84633.4us
W20260709 13:17:03.948094 10019 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.149s (SENT)
W20260709 13:17:03.914927 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90221.1us
I20260709 13:17:03.926697 11702 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:03.987385 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:03.990064 10127 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:03.928658 11976 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.923581 11853 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:03.957167 11879 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:03.957207 11875 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.946869 11887 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.088354 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:03.918694 11798 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:17:03.943650 11883 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:03.941187 11874 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:03.991863 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 111783us
W20260709 13:17:04.039952 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84065.3us
W20260709 13:17:04.052549 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 209195us
I20260709 13:17:04.040218 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:04.022668 10020 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.180s (SENT)
I20260709 13:17:04.099629 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:04.076174 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:04.076346 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:04.017964 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:04.078420 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:04.082118 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:04.013742 11702 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:03.987071 11805 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 13:17:03.989019 11837 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:17:04.059459 11858 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.103829 11793 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:04.110015 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:04.114400 11991 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:04.117014 11875 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.021584 10896 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:04.088675 12011 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:04.119796 11887 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.018258 12014 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:04.121117 10893 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:03.930341 11657 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.019214 11978 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.121789 11879 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.123482 11874 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:04.126960 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:04.176021 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.129145 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:04.108472 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:04.135406 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 117642us
W20260709 13:17:04.177328 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 154711us
W20260709 13:17:04.145469 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 234136us
I20260709 13:17:04.153306 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:04.149046 11936 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:04.152024 11991 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:04.146628 11912 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.170116 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:04.170899 11875 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:04.172005 11883 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.172984 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:04.173996 12023 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:04.151319 11513 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:04.196753 12014 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:04.223973 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.212622 11887 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:04.209304 11936 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.234875 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:04.237612 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:04.212435 11879 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.190029 12019 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:17:04.246465 12026 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:17:04.251729 12014 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.252689 11513 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:04.246351 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:04.254427 10891 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:04.255277 11883 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:04.257524 11978 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.214406 12022 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:17:04.260092 11657 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:04.259845 11859 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:04.262249 12019 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:04.262794 12019 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:04.261159 11912 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.268718 10202 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:04.223444 11837 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:17:04.221743 11933 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
W20260709 13:17:04.249421 12020 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:17:04.283782 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89259.8us
W20260709 13:17:04.285096 10020 connection.cc:256] RPC call timeout handler was delayed by -1.9959399399999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:17:04.285633 10020 connection.cc:256] RPC call timeout handler was delayed by -1.9140399400000661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:17:04.286124 10020 connection.cc:256] RPC call timeout handler was delayed by -1.7304399399999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
W20260709 13:17:04.286664 10020 connection.cc:256] RPC call timeout handler was delayed by -1.7259399399999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:17:04.287163 10020 connection.cc:256] RPC call timeout handler was delayed by -1.6899399399999311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 13:17:04.287603 10020 connection.cc:256] RPC call timeout handler was delayed by -1.6404399400000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 13:17:04.288254 10020 connection.cc:256] RPC call timeout handler was delayed by -1.5474420459997873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 13:17:04.289033 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4862420459999157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
W20260709 13:17:04.289565 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4829399400000511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 13:17:04.289921 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4349420459998328s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 13:17:04.290264 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2939399399999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.236s for a response
I20260709 13:17:04.213395 12024 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:17:04.294596 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 165558us
I20260709 13:17:04.267899 11887 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:04.212419 11858 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.210350 11702 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:04.301087 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.282622 12026 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:04.221323 11991 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.288650 11793 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:04.309280 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120061us
I20260709 13:17:04.288041 11879 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.293493 11969 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:17:04.314605 11887 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.309301 12010 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.317279 11978 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.317953 11978 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.320431 11978 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.321751 11991 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.314865 11879 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.329470 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:04.307272 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:04.337082 11805 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:04.337759 11805 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:04.359635 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53364.3us
W20260709 13:17:04.340415 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51662.9us
I20260709 13:17:04.362085 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:04.362843 10202 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.364430 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.365543 10127 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:04.369377 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:04.367748 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 223158us
I20260709 13:17:04.371367 10199 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:17:04.371465 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.372498 10125 raft_consensus.cc:2376] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5. Re-sending same reply.
I20260709 13:17:04.371529 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.326205 11702 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.326339 12026 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.388717 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.374796 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:04.393419 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.388963 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:04.401157 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.405261 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.364300 11859 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:04.414922 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.414132 10123 raft_consensus.cc:2376] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5. Re-sending same reply.
W20260709 13:17:04.418056 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.369819 11805 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:04.420379 12030 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:17:04.422165 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.422637 10127 raft_consensus.cc:2376] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7. Re-sending same reply.
I20260709 13:17:04.427639 10198 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:04.423432  9951 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 204) took 9619 ms. Trace:
I20260709 13:17:04.430614 10128 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:04.435765  9950 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 4 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.438261 10202 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:04.439344 11978 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.335057 11936 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.439786 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.439992 11978 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.367040 11793 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.444204 10131 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:04.405117 11875 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:04.436858 12010 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.438529  9950 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:04.455964 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.456708 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:04.446322 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.448657 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:04.458846 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.464274 12037 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:04.465279 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.472257 10125 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:04.474654 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:04.476362 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.478921 10129 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:04.480647 10129 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:04.481865 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.429431  9951 rpcz_store.cc:276] 0709 13:16:54.804099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.876831 (+ 72732us) service_pool.cc:224] Handling call
0709 13:16:57.801172 (+2924341us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:17:04.423356 (+6622184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:03.109400 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:03.110270 (+   870us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:03.110342 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:17:03.110369 (+    27us) tablet.cc:662] Decoding operations
0709 13:17:03.115429 (+  5060us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:03.115591 (+   162us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:03.115622 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:17:03.116483 (+   861us) tablet.cc:701] Row locks acquired
0709 13:17:03.116559 (+    76us) write_op.cc:260] PREPARE: finished
0709 13:17:03.116789 (+   230us) write_op.cc:270] Start()
0709 13:17:03.116969 (+   180us) write_op.cc:276] Timestamp: P: 1783603023116706 usec, L: 0
0709 13:17:03.116997 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:17:03.174788 (+ 57791us) log.cc:844] Serialized 2867 byte log entry
0709 13:17:03.678202 (+503414us) op_driver.cc:464] REPLICATION: finished
0709 13:17:04.022762 (+344560us) write_op.cc:301] APPLY: starting
0709 13:17:04.022922 (+   160us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:04.023214 (+   292us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:04.023405 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:04.024723 (+  1318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:04.025015 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:17:04.025063 (+    48us) write_op.cc:312] APPLY: finished
0709 13:17:04.025465 (+   402us) log.cc:844] Serialized 59 byte log entry
0709 13:17:04.025693 (+   228us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.025831 (+   138us) write_op.cc:454] Released schema lock
0709 13:17:04.025894 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:04.026324 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.026355 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2310333,"reported_tablets":18,"spinlock_wait_cycles":5515893504,"child_traces":[["op",{"apply.queue_time_us":303565,"apply.run_cpu_time_us":3644,"apply.run_wall_time_us":13637,"num_ops":7,"prepare.queue_time_us":78250,"prepare.run_cpu_time_us":31659,"prepare.run_wall_time_us":222724,"raft.queue_time_us":131919,"raft.run_cpu_time_us":17037,"raft.run_wall_time_us":205456,"replication_time_us":558929,"thread_start_us":339530,"threads_started":4,"wal-append.queue_time_us":227417}]]}
I20260709 13:17:04.488619 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:04.489492 10055 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:04.446821 11859 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.455303 11875 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.500869 10131 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:04.325893 12019 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:04.488188 12030 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:04.506166  9950 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 4 to 2, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:04.510644 12037 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:04.510612 11798 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:04.511454 12037 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:04.511730 11798 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.512589 12037 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.513695 10129 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:04.514190 12037 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.514801 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.515853 11798 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.518456  9950 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:04.527922 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 125482us
I20260709 13:17:04.513252 10131 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:17:04.450366 11938 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:17:04.483731 12036 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:04.484441 11978 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.524912 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.498027 11859 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.459568 11962 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:17:04.542634 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.543417 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.543882 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.558475 10129 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:04.544564  9950 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 2 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.546885 10124 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:04.506173 12026 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.504200 11875 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.500883 12023 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:04.522495 12035 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:04.567262 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:04.568975 12018 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:04.561772 12029 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.572722 10125 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:04.571861 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.571645  9950 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 2 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.574868 10131 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:04.576556 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:04.577322 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76468.1us
I20260709 13:17:04.578140 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:04.579000 10198 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:04.606248 10130 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:04.562034 11838 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.607944  9950 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 2 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.569649 11978 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:04.569581 12018 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.615928 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:04.616128 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:04.617273 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:04.571043 12036 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:04.620270 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74297us
I20260709 13:17:04.572417 12029 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.607146 12021 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.628283 12010 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:04.628257 12021 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.629698 12010 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:04.630316 12010 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.613911 11838 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.631237 12036 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.612994 12023 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:04.612214 11793 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.636334 12023 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:04.637940 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:04.622092 11978 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:04.639595 10055 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:04.640648 11978 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.578373 12019 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:04.622064 10126 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 1.
W20260709 13:17:04.644408 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66133.6us
I20260709 13:17:04.645032 10056 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:04.630844 12010 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.661237 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:04.662149 10126 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:04.661480 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:04.651798 10129 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:04.668391 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.668949 10050 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:04.676934 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:04.688398 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67725us
I20260709 13:17:04.688959 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:04.690260 10129 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:04.695302 10130 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:04.592689 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:04.695188 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:04.700467 10047 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:04.702875 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57401.5us
I20260709 13:17:04.709817 10052 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:17:04.710839 11858 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:04.711227 12051 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:17:04.712400 10130 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:04.719033 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:04.721993 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 98886.8us
I20260709 13:17:04.723305 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:04.725797 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:04.726564 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:04.727525 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:04.697731 11805 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:04.727567 10047 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:04.701857 11797 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:04.760994 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57245.9us
I20260709 13:17:04.759761 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:04.762104 10047 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:04.791729 12051 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:04.801738 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74319.3us
I20260709 13:17:04.753238 11805 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:04.808188 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:04.823999 10122 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:04.753883 11858 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:17:04.798213 11797 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:04.848526 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:04.817422 11849 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:04.851213 12041 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.135s
I20260709 13:17:04.804550 12022 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:17:04.856608 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:04.817744 11805 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:04.856105 11991 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.007s
W20260709 13:17:04.868191 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113573us
I20260709 13:17:04.884523 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:04.860459 11858 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:04.886632 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46393.6us
I20260709 13:17:04.891656 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:04.949259 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.030786 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99216.8us
W20260709 13:17:04.914992 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 191780us
W20260709 13:17:04.996083 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.353s (SENT)
I20260709 13:17:05.052484 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:05.028667 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 137111us
I20260709 13:17:04.887821 12063 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:04.889154 11858 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:04.890827 11849 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:05.115083 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.116950 11978 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:05.122411 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 70707.8us
I20260709 13:17:04.927711 12068 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:05.124078 12068 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.124274 11969 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:05.124631 12068 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:04.967442 12051 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:05.124083 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 128105us
I20260709 13:17:04.898794 11991 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:05.124686 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:04.898713 12014 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:05.129724 12057 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:05.123891 11858 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:05.043452 12060 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 13:17:05.091650 12036 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.169224 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:05.159869 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 91824.2us
I20260709 13:17:05.179589 10054 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:05.180056 12011 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:05.180236 12022 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:05.180565 12011 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:05.180724 10054 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:05.180974 12022 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:05.181655 12022 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.181762 12011 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.181594 10054 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.182962 10054 raft_consensus.cc:614] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:05.043092 12064 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:05.192221 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:05.193778 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 353803us
W20260709 13:17:05.206960 10126 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.000s
I20260709 13:17:05.207811 10126 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:05.166108 11991 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:05.212627 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88021.4us
I20260709 13:17:05.186437 12011 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.213730 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.220963 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:05.201295 11858 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.191890 12064 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:05.201550 12063 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:05.197448 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 626613us
I20260709 13:17:05.209542 11976 raft_consensus.cc:2764] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:05.187302 12036 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:05.259588 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 117258us
I20260709 13:17:05.263454 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:05.237051 11797 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.279845 10049 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:05.247810 12064 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.285167 10049 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:05.286826 10049 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:05.248991 11912 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.247810 11798 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:05.290951 10049 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:05.256073 11879 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:05.307607 10054 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:05.298082 12022 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.299860 11978 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:05.301921 10049 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:05.281273 12075 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:17:05.263094 12076 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:17:05.303226 11969 raft_consensus.cc:2764] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:17:05.285434 12024 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:17:05.261802 11849 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.354100 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a82842926e645c08dde847bfe182c31"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:05.355240 10202 raft_consensus.cc:606] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:05.341989 12059 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.323s	user 0.001s	sys 0.001s
I20260709 13:17:05.343197 10049 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.337121 10891 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:17:05.369277 12069 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:17:05.370841 12071 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:05.373764 12078 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:05.376005 12074 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:17:05.378474 12066 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:17:05.381212 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 125435us
W20260709 13:17:05.366195 12061 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.314s	user 0.000s	sys 0.002s
I20260709 13:17:05.309051 11798 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:05.403519 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 182674us
W20260709 13:17:05.381067 12058 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:17:05.380147 11978 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:05.359099 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 97561.3us
I20260709 13:17:05.388567 12022 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:05.422854 10019 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.205s (SENT)
I20260709 13:17:05.417136 10049 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.423501 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.380326 11938 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:05.345270 11991 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.410450 12021 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.407848 11879 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 13:17:05.433044 12010 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.739s	user 0.015s	sys 0.000s
I20260709 13:17:05.425597 11938 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:05.423094 11969 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:17:05.456172 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:05.456666 10049 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:05.422719 11978 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.462527 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:05.457296 10049 pending_rounds.cc:85] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 2
W20260709 13:17:05.468910 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46100.7us
W20260709 13:17:05.452028 11875 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.873s	user 0.010s	sys 0.010s
I20260709 13:17:05.467700 10049 pending_rounds.cc:107] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:05.470479 10049 raft_consensus.cc:2889] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:05.426934 12014 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:17:05.452636 12010 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.487310 11875 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:05.466128 12081 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:17:05.475049 12077 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 13:17:05.529618 12090 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.530361 12090 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.531412 12068 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.532276 11798 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.534330 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:05.534855 12010 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.535362 10052 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:05.536754 10051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:05.537019 11978 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.537698 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:05.510329 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44842.9us
I20260709 13:17:05.538444 10052 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.482256 11938 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.539899 10020 leader_election.cc:400] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4.
I20260709 13:17:05.540316 10020 leader_election.cc:403] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:05.474316 11879 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:05.524542 11969 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:05.526340 11702 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.124s	user 0.014s	sys 0.005s
I20260709 13:17:05.558866 12076 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.559619 12076 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.526839 12036 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.437083 12022 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:17:05.527180 12092 raft_consensus.cc:2764] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:17:05.562417 12092 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4.
I20260709 13:17:05.538509 12014 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:05.563607 12014 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:05.524083 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63223.1us
I20260709 13:17:05.539930 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:05.549744 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128891us
I20260709 13:17:05.548877 11879 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:05.550622 11938 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:05.530922 11805 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.000s
I20260709 13:17:05.556486 12064 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:05.548698 12019 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.897s	user 0.013s	sys 0.004s
I20260709 13:17:05.566768 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.593079 11805 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.597280 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:05.597610 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.598315 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1026) took 1182 ms. Trace:
W20260709 13:17:05.568325 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.599355 12063 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:05.599426 12092 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:05.601486 11805 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.601846 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:05.602599 12014 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.604413 12014 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:05.605463 10130 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:05.607617 11838 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.606875 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.566732 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.436967 11936 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.115s
I20260709 13:17:05.588057 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1044) took 1038 ms. Trace:
I20260709 13:17:05.610493 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:05.611759 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:05.598776 10123 rpcz_store.cc:276] 0709 13:17:04.415790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.596891 (+1181101us) service_pool.cc:224] Handling call
0709 13:17:05.598215 (+  1324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:05.567858 12029 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.583425 11978 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.618712 11936 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:05.619963 11936 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:05.619889 11978 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.621096 11936 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:05.602592 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.623392 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.613199 11969 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:05.627305 12094 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:05.629596 12094 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.613910 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:05.630079 12094 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.630312 10129 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 2.
I20260709 13:17:05.615345 10054 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:05.615810 12082 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:05.633598 12082 consensus_queue.cc:1243] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:05.633714 10053 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:17:05.634655 12093 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.109s	user 0.005s	sys 0.001s
I20260709 13:17:05.616751 12064 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:05.632833 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:05.588716 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 154473us
I20260709 13:17:05.583425 11879 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:05.567961 12036 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:05.564605 12068 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.622413 11936 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.644501 10053 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:05.610535 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43918.9us
I20260709 13:17:05.611927 10122 rpcz_store.cc:276] 0709 13:17:04.548633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.825569 (+276936us) service_pool.cc:224] Handling call
0709 13:17:04.860831 (+ 35262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.879181 (+ 18350us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:04.913044 (+ 33863us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.586170 (+673126us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.586753 (+   583us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:05.586829 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:05.586883 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.586957 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.587105 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52301,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":131,"raft.queue_time_us":32832,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":210,"thread_start_us":596001,"threads_started":3,"wal-append.queue_time_us":587437}
I20260709 13:17:05.631392 12029 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:17:05.638398 12082 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:05.619087 11838 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:05.573518 12016 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:05.645882 12082 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.646111 12016 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:05.646487 12082 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.622424 11978 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.647994 12016 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.651521 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:05.651968 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.627933 11912 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:05.650381 11837 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:05.654723 11657 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:05.654886 12068 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:05.655243 11657 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:05.655993 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:05.656234 11657 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.657600 11657 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.658550 12075 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:05.653539 10129 pending_rounds.cc:85] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:17:05.661955 10129 pending_rounds.cc:107] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:17:05.663024 10129 raft_consensus.cc:2889] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:05.663332 12092 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:05.644130 12036 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.676692 11838 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:05.677249 11838 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:05.678400 11838 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.699890 11838 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:05.700881 10865 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.675s (SENT)
I20260709 13:17:05.701467 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.684190 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:05.682864 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.682127 12036 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:05.681746 12076 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:05.707826 12088 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.183s	user 0.000s	sys 0.004s
I20260709 13:17:05.705128 10052 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:05.705282 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.703342 11978 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:05.706228 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:05.707244 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.691812 10051 raft_consensus.cc:606] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.717803 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:05.718498 10051 raft_consensus.cc:614] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:05.718775 10055 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:05.744447 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73666.2us
I20260709 13:17:05.746860 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:05.708087 11938 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:05.748258 10052 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:05.722606 11805 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:05.698946 12094 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.728396 11798 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.771152 11805 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:05.772390 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:05.746454 11874 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:05.774322 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:05.779808 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.762403 11853 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:05.763300 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54661.5us
I20260709 13:17:05.763957 11657 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:05.765537 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58423.7us
I20260709 13:17:05.789659 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.712821 11912 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:17:05.794579 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:05.768275 10052 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:05.795282 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:05.751758 12092 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.772928 12094 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:05.817224 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:05.782330 11969 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:05.782711 12024 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.787317 12019 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:05.782392 11874 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:05.827782 10020 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:05.838075 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:05.839684 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65531.3us
I20260709 13:17:05.772933 11805 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.771284 11798 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.813899 11938 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:05.806787 12068 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:05.865743 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:05.828505 12014 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:05.862421 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:05.825062 12075 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.867120 10055 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:05.790014 11853 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.864528 12063 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:05.858559 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:05.855526 12076 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:05.870365 12068 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:05.930975 12096 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:05.932044 12096 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:05.941787 11978 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:05.943181 11978 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:05.946835 10865 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.909s (SENT)
I20260709 13:17:05.905761 12019 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:05.849217 12024 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:06.024268 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 93561.7us
W20260709 13:17:06.035326 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.021914 12019 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:06.022094 12050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.012s	sys 0.008s
W20260709 13:17:06.055455 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:05.924014 12068 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:05.921465 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 394) took 1433 ms. Trace:
I20260709 13:17:06.023888 12076 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:06.059835 12050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.435s	user 0.013s	sys 0.008s
W20260709 13:17:06.075554 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.079319 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.078856 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 225502us
W20260709 13:17:06.074328 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 127561us
W20260709 13:17:06.080991 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 222556us
I20260709 13:17:06.054121 12019 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:06.095705 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:06.123178 11837 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.124531 11837 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.128293 11936 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:06.129778 11936 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.133848 11936 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:06.134151 12024 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:06.135604 11837 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:06.136190 11853 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:06.074855 10899 rpcz_store.cc:276] 0709 13:17:04.323141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.358238 (+ 35097us) service_pool.cc:224] Handling call
0709 13:17:04.372109 (+ 13871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:04.372802 (+   693us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:04.373189 (+   387us) log.cc:844] Serialized 14 byte log entry
0709 13:17:04.459515 (+ 86326us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:04.573215 (+113700us) log.cc:844] Serialized 26 byte log entry
0709 13:17:04.649585 (+ 76370us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:04.652623 (+  3038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:04.671458 (+ 18835us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:05.474264 (+802806us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b1800350680. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:05.707441 (+233177us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800350680. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:05.744663 (+ 37222us) raft_consensus.cc:1685] finished
0709 13:17:05.748043 (+  3380us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:05.756703 (+  8660us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8974,"prepare.queue_time_us":959,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":6009,"raft.queue_time_us":16370,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":27359,"spinlock_wait_cycles":14917248,"thread_start_us":50332,"threads_started":1,"wal-append.queue_time_us":108510}
I20260709 13:17:06.150614 10202 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:05.920033 12082 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:05.909626 12075 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:06.156513 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60910.8us
I20260709 13:17:06.158459 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:06.162729 10196 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
W20260709 13:17:06.160033 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63508.5us
I20260709 13:17:06.162578 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:06.084183 11938 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:06.074018 12106 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:06.179760 10199 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:06.044863 11874 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.045575 12018 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.210083 12050 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.210505 11874 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.212373 12106 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:06.210779 12050 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.210525 12018 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:06.156102 12063 raft_consensus.cc:2764] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:17:06.198573 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:06.199183 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:06.208141 11938 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.165350 11978 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:06.197571 11798 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:06.154137 12096 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:06.225755 10123 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:17:06.252566 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.226570 11879 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.004s
W20260709 13:17:06.323845 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 125486us
W20260709 13:17:06.325778 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 299157us
I20260709 13:17:06.333496 10891 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:17:06.335990 12104 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
I20260709 13:17:06.308871 11976 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:06.341204 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:06.319087 12096 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:06.312532 11879 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:06.355376 12115 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:17:06.402822 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 62765.6us
W20260709 13:17:06.432060 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.296921 12076 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:06.437340 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 272431us
I20260709 13:17:06.436393 11879 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:06.402467 12036 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.666s	user 0.016s	sys 0.000s
I20260709 13:17:06.360423 12096 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:06.518528 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.797060 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.797838 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.780149 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:06.464498 12121 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:17:06.347286 12050 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:06.457906 12113 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:17:06.431751 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:06.805707 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:06.796236 12092 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.961s	user 0.012s	sys 0.001s
I20260709 13:17:06.720811 12075 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:06.465273 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:06.804805 11874 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:06.810552 11933 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:06.823771 11933 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:06.825033 11933 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:06.810799 12068 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.728s	user 0.014s	sys 0.000s
I20260709 13:17:06.822590 12117 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:06.818444 12073 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.822875 11853 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.830866 12073 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.831844 11853 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.833652 12073 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:06.834506 12117 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.437546 12119 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.399s
W20260709 13:17:06.821216 10168 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.046s (SENT)
I20260709 13:17:06.822840 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1019) took 2183 ms. Trace:
I20260709 13:17:06.826265 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1330) took 2056 ms. Trace:
I20260709 13:17:06.836305 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:06.847677 12047 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 2.231s	user 0.012s	sys 0.002s
I20260709 13:17:06.846657 10054 rpcz_store.cc:276] 0709 13:17:04.769662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.320774 (+551112us) service_pool.cc:224] Handling call
0709 13:17:05.321159 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:05.631692 (+310533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.825781 (+1194089us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400136854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:06.826156 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676053888}
I20260709 13:17:06.826269 11933 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.798961 12036 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.819239 12092 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:06.827584 12075 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.819710 10899 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:06.834946 11853 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:06.860816 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58487.6us
I20260709 13:17:06.863917 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1049) took 1181 ms. Trace:
I20260709 13:17:06.852730 11513 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:06.866180 10050 rpcz_store.cc:276] 0709 13:17:05.682783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.683272 (+   489us) service_pool.cc:224] Handling call
0709 13:17:05.716329 (+ 33057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:05.720689 (+  4360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.720730 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.721345 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:17:05.859873 (+138528us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:05.859961 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.860069 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.863623 (+1003554us) raft_consensus.cc:1685] finished
0709 13:17:06.863649 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.863824 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27812,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":6027,"thread_start_us":78444,"threads_started":1,"wal-append.queue_time_us":543912}
I20260709 13:17:06.835942 12068 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.870954 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:06.886422 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:06.886911 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:06.867677 12075 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.897015 12127 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.897782 12127 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:06.898691 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:06.899585 10052 raft_consensus.cc:2376] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3. Re-sending same reply.
I20260709 13:17:06.886610 12092 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:06.905082 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:06.912019 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:06.894851 11879 raft_consensus.cc:2764] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:06.918407 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:06.924839 11513 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.927526 11875 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:06.898873 12127 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:06.935724 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:06.907871 12018 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.003s
I20260709 13:17:06.918450 12017 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:06.936491 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49976 (request call id 21) took 1400 ms. Trace:
I20260709 13:17:06.937863 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:06.938306 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:06.938297 10051 rpcz_store.cc:276] 0709 13:17:05.534496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.536553 (+  2057us) service_pool.cc:224] Handling call
0709 13:17:06.934535 (+1397982us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540010aed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:06.934718 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354725760}
I20260709 13:17:06.942860 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:06.944734 10196 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:06.915143 12096 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:06.915547 12092 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:06.937675 11513 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:06.954668 12017 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:06.954995 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:06.955466 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:06.957471 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:06.994012 10202 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:06.956126 11879 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:06.846935 10048 rpcz_store.cc:276] 0709 13:17:04.639870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.644844 (+  4974us) service_pool.cc:224] Handling call
0709 13:17:04.700081 (+ 55237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:04.727237 (+ 27156us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:05.075947 (+348710us) log.cc:844] Serialized 14 byte log entry
0709 13:17:06.822019 (+1746072us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b18000b45e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287
0709 13:17:06.822207 (+   188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:06.822296 (+    89us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:06.822382 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:06.822526 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:06.822739 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28649,"prepare.queue_time_us":99598,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":102796,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":8792,"spinlock_wait_cycles":22760192,"thread_start_us":447190,"threads_started":3,"wal-append.queue_time_us":263212,"wal-append.run_cpu_time_us":1298,"wal-append.run_wall_time_us":1022376}
I20260709 13:17:07.003829 11879 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.006723 11879 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.000204 10050 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:07.000476 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.000751 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.040683 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.041664 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.042553 10124 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:07.002650 12092 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.007351 12076 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:07.008659 12075 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:07.009467 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.000487 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.045409 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:06.964265 12018 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:17:06.955077 11853 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:07.055825 12018 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:07.056329 12018 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:07.057150 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.057353 12018 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.058806 12018 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:07.066228 10898 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:07.015673 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:07.016620 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:07.018260 11513 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:07.018800 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.019487 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.019757 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.072808 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.073987 10049 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
W20260709 13:17:07.075193 10020 connection.cc:256] RPC call timeout handler was delayed by -2.2954882629999247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:17:07.075389 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:07.076000 10020 connection.cc:256] RPC call timeout handler was delayed by -2.1955882629999905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 13:17:07.076486 10049 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:06.998585 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.020174 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.021582 10048 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:07.024391 10123 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:17:07.080684 10123 pending_rounds.cc:85] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:07.080989 10123 pending_rounds.cc:107] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:17:07.081278 10123 raft_consensus.cc:2889] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:07.026180 10899 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:07.057015 11853 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:07.034468 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.087891 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:07.088177 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:06.939384 12036 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:06.996594 12017 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:07.040638 10126 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:07.044727 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.096987 10049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee79852a0bc04642b58b690930f78267"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:07.098107 10049 raft_consensus.cc:606] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.098655 10049 raft_consensus.cc:614] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:07.048643 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:07.012151 12128 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:17:07.053310 10130 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:07.049245 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.051903 10131 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:07.066288 11879 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.095930 10126 pending_rounds.cc:85] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:07.108240 10126 pending_rounds.cc:107] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:07.108695 10126 raft_consensus.cc:2889] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:07.051862 10203 raft_consensus.cc:1217] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:17:07.069618 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.109833 10203 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:17:07.110245 10047 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:07.113853 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.069806 12117 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.119560 10202 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:07.020865 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.080331 11793 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:07.084887 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.092079 11853 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:07.088833 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.045231 11874 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.102993 12017 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.054387 10891 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 2 LEARNER]: Advancing to term 4
W20260709 13:17:07.142391 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.122138 11879 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:07.145664 12102 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.121s	user 0.006s	sys 0.044s
I20260709 13:17:07.093771 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.051364 12076 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:07.147895 10200 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 2: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.148195 12076 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.053421 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.103458 10130 pending_rounds.cc:85] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:07.150229 10130 pending_rounds.cc:107] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:07.150555 10130 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:07.103653 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.105499 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:06.956601 12021 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.133s
I20260709 13:17:07.080438 10048 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:06.947008 12025 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.122262 10197 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:17:07.076536 10020 connection.cc:256] RPC call timeout handler was delayed by -1.9894882630001121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 13:17:07.124821 10054 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:07.085887 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.126019 10050 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:07.162052 10020 connection.cc:256] RPC call timeout handler was delayed by -1.9336882630000218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
I20260709 13:17:07.162539 10050 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:07.128007 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 116321us
I20260709 13:17:07.089982 12096 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.090504 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.133150 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.129914 12117 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.135288 11874 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.136335 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.137182 10049 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:07.130957 11793 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.151360 10196 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:07.165278 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1039) took 1537 ms. Trace:
I20260709 13:17:07.170423 10049 raft_consensus.cc:606] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.166230 10053 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 3: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:07.171952 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.166536 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.167917 10122 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:07.167577 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.175954 10203 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.177721 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.179423 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.180466 10049 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:07.183274 10205 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:07.186527 10049 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:07.170863 10056 rpcz_store.cc:276] 0709 13:17:05.628321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.629603 (+  1282us) service_pool.cc:224] Handling call
0709 13:17:05.630071 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:05.630484 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.630515 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:05.631084 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:17:05.774593 (+143509us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:05.774643 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.775746 (+  1103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:07.144736 (+1368990us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:07.155633 (+ 10897us) raft_consensus.cc:1685] finished
0709 13:17:07.155678 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.165185 (+  9507us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4873,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":11656960,"thread_start_us":139012,"threads_started":1,"wal-append.queue_time_us":234633}
I20260709 13:17:07.195644 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.179255 10050 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.203413 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.204882 10196 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:07.199440 10124 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:07.205557 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:07.208348 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.209470 10130 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:07.208410 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.210584 10122 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.220144 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.221464 10056 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:07.220232 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.228811 10123 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:07.232501 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.235275 10123 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:07.237788 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.239298 10123 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:07.244710 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.178574 12068 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:07.244581 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.248661 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.249665 10203 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.248754 12134 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:17:07.198055 11875 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.252772 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.199110 11879 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.253682 10056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:07.261792 10056 raft_consensus.cc:606] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.213757 11938 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:07.178056 12117 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.258440 10196 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:07.185454 12096 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.216935 12050 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.271157 10047 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:07.213779 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:07.271883 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 103844us
I20260709 13:17:07.271929 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:07.274935 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.275360 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.279906 10130 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:07.278083 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:07.283836 10900 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:07.286096 10054 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:07.286388 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.207887 12051 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
W20260709 13:17:07.287803 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 105363us
I20260709 13:17:07.284962 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:07.169831 12025 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.292627 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.298977 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.287190 10197 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
W20260709 13:17:07.278674 12019 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.126s	user 0.010s	sys 0.010s
I20260709 13:17:07.301299 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.304450 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.305630 10130 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:07.306067 10196 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.308264 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.285615 12119 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.326140 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:07.328222 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.330245 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.331374 10197 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:07.264920 11875 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.340296 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:07.290577 12050 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.355175 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.355984 10130 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:07.260591 10048 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.321660 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:07.343470 10200 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:07.312958 12019 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.455049 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:07.367722 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:07.333961 11513 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:07.403192 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:07.318507 10897 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:07.471508 12048 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 2.852s	user 0.010s	sys 0.006s
I20260709 13:17:07.431622 10196 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:07.431147 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:07.429081 12136 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:17:07.471413 12137 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:17:07.519994 10047 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:07.430898 12133 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:17:07.505023 10052 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:07.471827 12016 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:07.533761 12140 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:17:07.454299 12050 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:07.536448 12109 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.512s	user 0.016s	sys 0.000s
I20260709 13:17:07.538673 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 94) took 1898 ms. Trace:
I20260709 13:17:07.539292 10893 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:07.539201 10129 rpcz_store.cc:276] 0709 13:17:05.640014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.640218 (+   204us) service_pool.cc:224] Handling call
0709 13:17:05.644394 (+  4176us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:05.664396 (+ 20002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:05.664441 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:05.665106 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 13:17:05.801264 (+136158us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:05.824467 (+ 23203us) log.cc:844] Serialized 14 byte log entry
0709 13:17:05.824764 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:05.824871 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:06.354050 (+529179us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b1800350500. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:07.535634 (+1181584us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:07.538397 (+  2763us) raft_consensus.cc:1685] finished
0709 13:17:07.538419 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:07.538601 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31433,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":9435,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":28595,"spinlock_wait_cycles":67436288,"thread_start_us":135526,"threads_started":1,"wal-append.queue_time_us":189396}
W20260709 13:17:07.546048 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:07.546638 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.548202 12025 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:07.500352 12092 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:17:07.559784 12092 consensus_queue.cc:1243] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:07.561313 12019 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:07.547060 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:07.566642 11962 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:07.567291 11962 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.338445 12119 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.568702 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:07.569550 10895 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:07.569885 11962 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.402788 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:07.572587 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.573405 10896 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:07.573668 10054 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:07.578462 10053 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:07.579909 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:07.581262 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.582446 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.354151 12067 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.170s
I20260709 13:17:07.590045 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.593303 10054 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:07.573638 12096 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.613523 10122 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:07.542933 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.432973 11874 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.527891 10204 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:07.468930 12117 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.333050 12076 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:07.562623 10048 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:07.432859 11793 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.495859 11513 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.673123 10900 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:07.625480 10123 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:07.625869 10205 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:07.664084 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:07.677707 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.678930 10123 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.676182 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:07.697649 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:07.698624 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:07.702365 10019 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.572s (SENT)
I20260709 13:17:07.704701 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.702616 10201 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 3: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.705528 10054 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:07.707015 10054 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:17:07.708626 10054 pending_rounds.cc:85] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 2
W20260709 13:17:07.708630 10019 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.639s (SENT)
I20260709 13:17:07.709076 10054 pending_rounds.cc:107] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:07.709388 10054 raft_consensus.cc:2889] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:07.709312 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:07.720415 10019 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.692s (SENT)
I20260709 13:17:07.720979 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:07.722121 10019 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.694s (SENT)
W20260709 13:17:07.725989 10019 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.816s (SENT)
I20260709 13:17:07.726485 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.729137 10892 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:07.664984 12067 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:07.734807 10019 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.873s (SENT)
I20260709 13:17:07.743552 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:07.735749 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.695775 11513 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.702564 11702 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:07.746291 10129 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.747608 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:07.720316 12141 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.719760 11849 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:07.735524 11938 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:07.768288 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.012s	sys 0.010s
I20260709 13:17:07.796416 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:07.798602 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.014s	sys 0.011s
I20260709 13:17:07.797612 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:07.799576 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 589298us
W20260709 13:17:07.797104 10049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.021s	sys 0.081s
I20260709 13:17:07.800624 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:07.801086 10049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.021s	sys 0.081s
I20260709 13:17:07.801328 10129 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:07.801523 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:07.755913 11702 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:07.744520 12145 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:07.817097 10197 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
W20260709 13:17:07.788245 10051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.000s	sys 0.027s
W20260709 13:17:07.776015 12017 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.603s	user 0.014s	sys 0.002s
I20260709 13:17:07.819535 10052 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:07.820822 10051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.002s	sys 0.029s
I20260709 13:17:07.823992 10050 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.840837 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.843396 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.754225 11838 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:17:07.775780 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:07.852092 12145 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:07.852767 11853 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.007s
I20260709 13:17:07.840284 12017 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.853921 10898 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:17:07.850948 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:07.813310 11991 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.731s	user 0.015s	sys 0.019s
I20260709 13:17:07.808408 11702 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:07.857477 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56904.6us
I20260709 13:17:07.857440 10898 pending_rounds.cc:85] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 4
I20260709 13:17:07.873699 10898 pending_rounds.cc:107] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:17:07.873837 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.874058 10898 raft_consensus.cc:2889] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:07.821321 11938 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:07.879874 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.879959 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:07.839867 11793 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:07.840247 12022 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:07.900914 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:07.870790 11991 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.788s	user 0.016s	sys 0.019s
I20260709 13:17:07.890954 10129 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:07.903084 10052 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:07.753718 12067 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:07.891441 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136164us
I20260709 13:17:07.904893 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.912623 10198 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:07.776932 12141 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:07.881942 11969 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:17:07.921891 12067 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:07.922811 10199 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:07.933916 11879 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:07.934644 11879 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:07.937100 11879 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:07.940848 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:07.941655 10125 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:07.944041 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 185641us
I20260709 13:17:07.871279 11853 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.088500 10048 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:17:07.999643 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.050724 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:08.122362 10055 raft_consensus.cc:2368] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:08.051576 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1057) took 1017 ms. Trace:
I20260709 13:17:07.949419 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:08.002616 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:08.063390 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.052038 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:08.073405 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 170961us
I20260709 13:17:07.953841 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.049685 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:08.127347 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 173633us
I20260709 13:17:08.127537 10122 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:08.121740 10048 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 5
I20260709 13:17:08.128224 10048 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 5.6
W20260709 13:17:08.065755 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.128669 10048 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:08.129707 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.129813 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 116) took 1118 ms. Trace:
I20260709 13:17:07.944386 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:08.131536 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 187237us
I20260709 13:17:08.120554 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.117522 10125 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:08.133332 10050 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:08.126166 10052 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:08.123173 10127 rpcz_store.cc:276] 0709 13:17:07.033853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.067074 (+ 33221us) service_pool.cc:224] Handling call
0709 13:17:07.067712 (+   638us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:07.087135 (+ 19423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:07.087170 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:07.097324 (+ 10154us) log.cc:844] Serialized 26 byte log entry
0709 13:17:07.355277 (+257953us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:07.666363 (+311086us) log.cc:844] Serialized 14 byte log entry
0709 13:17:08.051040 (+384677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.051189 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.051256 (+    67us) raft_consensus.cc:1685] finished
0709 13:17:08.051278 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.051484 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1124171,"prepare.queue_time_us":501197,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":441671,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":19449,"thread_start_us":481780,"threads_started":2,"wal-append.queue_time_us":592289}
W20260709 13:17:08.136839 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:08.139140 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.140245 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:08.140678 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.142220 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.007179 10897 raft_consensus.cc:2368] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3.
I20260709 13:17:08.182174 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.137028 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:08.183899 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43705.3us
I20260709 13:17:08.142576 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.912529 12068 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:08.077088 12036 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.002022 12096 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.074783 11874 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.183735 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:07.913192 12022 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.130664 10048 rpcz_store.cc:276] 0709 13:17:07.011774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.020858 (+  9084us) service_pool.cc:224] Handling call
0709 13:17:07.021391 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:07.562419 (+541028us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b540033a520. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f79a1d 00007fa277f601a4 00007fa277fe2447 00007fa277ffafe7 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419
0709 13:17:08.129414 (+566995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.129729 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120187136}
I20260709 13:17:08.186107 10129 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:08.119232 12017 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:08.001758 12014 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:08.205085 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 198160us
I20260709 13:17:08.215308 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.217320 10900 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.143741 12132 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.224629 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.222710 10125 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:08.219908 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:08.134358 11991 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:08.220248 10122 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:08.145385 11879 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:08.183581 12145 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.239645 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:08.240087 10129 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:08.213136 12171 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.239259 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.217026 12152 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.244500 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.266547 10125 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 3: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:08.274101 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.275223 10125 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:08.237848 12022 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:08.283653  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:08.269558 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.288141 12082 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.246659 11879 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.276165 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:08.336987 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67531.1us
I20260709 13:17:08.338450 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.338366 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.340061 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.340806 10129 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:08.341742 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.342399 10129 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:08.281015 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.253969 10900 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:08.317444 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.344633 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.187279 12130 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.078s
W20260709 13:17:08.348942 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 135807us
I20260709 13:17:08.222172 12014 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.325809  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:08.213514 11513 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.263924 11849 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.354784 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.277485 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:08.328118 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140483us
I20260709 13:17:08.335035 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.356472 10122 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:08.217247 12163 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.145s
W20260709 13:17:08.347437 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.345280 10125 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.219873 12036 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.293931 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1334) took 1246 ms. Trace:
I20260709 13:17:08.239487 11874 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.346601 11991 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:08.233014 12096 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.354971 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:08.321867 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 181693us
I20260709 13:17:08.217732 12160 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.355381 11849 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.218205 12151 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 13:17:08.273530 12017 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.214452 12135 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:17:08.247445 12132 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:08.361253 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 83924.1us
I20260709 13:17:08.345252 11879 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.219627 12165 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.138s
I20260709 13:17:08.359853 10204 rpcz_store.cc:276] 0709 13:17:07.047749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.048122 (+   373us) service_pool.cc:224] Handling call
0709 13:17:08.293582 (+1245460us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:08.293814 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080288512}
I20260709 13:17:08.352283 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.363044 10129 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:17:08.402906 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50786.1us
I20260709 13:17:08.403167 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1067) took 1335 ms. Trace:
I20260709 13:17:08.351573 11938 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.403766 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:08.374362 12166 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:17:08.374922 12154 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:17:08.381186 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.275450 12152 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.389832 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.372113 11849 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.401721 10127 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:08.336333 12162 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:17:08.364324 12019 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:08.404110 11938 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.369135 11991 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.404580 11874 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.389043 10125 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:08.274070 12145 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.403594 10129 rpcz_store.cc:276] 0709 13:17:07.068077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.358793 (+1290716us) service_pool.cc:224] Handling call
0709 13:17:08.360383 (+  1590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.402817 (+ 42434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.403061 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.363704 11912 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.410188 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.374433 10895 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:08.375660 12017 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.413347 10122 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:17:08.414093 11838 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:08.436077 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54992.6us
I20260709 13:17:08.404598 12014 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:08.420261 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63354.7us
I20260709 13:17:08.319743 12171 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.435482 10122 pending_rounds.cc:85] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 1
I20260709 13:17:08.445178 10122 pending_rounds.cc:107] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:08.445788 10122 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:08.441715 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.438827 10125 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:08.448483 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1068) took 1367 ms. Trace:
I20260709 13:17:08.448998 10125 rpcz_store.cc:276] 0709 13:17:07.080740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.432228 (+1351488us) service_pool.cc:224] Handling call
0709 13:17:08.432785 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.448161 (+ 15376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.448415 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.438555 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.452913 10129 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:17:08.453855 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2472) took 1110 ms. Trace:
I20260709 13:17:08.454149 10129 rpcz_store.cc:276] 0709 13:17:07.343308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.437640 (+1094332us) service_pool.cc:224] Handling call
0709 13:17:08.438116 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.453500 (+ 15384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.453782 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.435216 12019 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:08.436751 12017 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.400000 11702 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.457108 10096 leader_election.cc:400] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:08.422487 11991 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.431288 11853 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:08.436698 12094 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.436630 11874 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:08.460780 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:08.457770 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.462370 10096 leader_election.cc:403] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:08.473500 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.456355 12163 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:08.458761 11805 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.477432 11805 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:08.478134 11805 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.479620 11874 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.481091 11702 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:08.463361 12019 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:08.483239 12019 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:08.484730 12169 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:17:08.439594 12176 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:17:08.491427 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.478746 11805 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:08.508405 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50705.9us
I20260709 13:17:08.509904 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.512104 10130 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:17:08.513327 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1061) took 1478 ms. Trace:
I20260709 13:17:08.513712 10130 rpcz_store.cc:276] 0709 13:17:07.034746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.489674 (+1454928us) service_pool.cc:224] Handling call
0709 13:17:08.490137 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.512848 (+ 22711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.513241 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.515974 12162 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.486915 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 316) took 1441 ms. Trace:
I20260709 13:17:08.517599 10891 rpcz_store.cc:276] 0709 13:17:07.045085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.053645 (+  8560us) service_pool.cc:224] Handling call
0709 13:17:07.054125 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:07.852754 (+798629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:07.852796 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:07.853386 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:17:08.274106 (+420720us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:08.316928 (+ 42822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.331881 (+ 14953us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:08.486479 (+154598us) raft_consensus.cc:1685] finished
0709 13:17:08.486514 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:08.486800 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6558,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":151,"thread_start_us":618680,"threads_started":2,"wal-append.queue_time_us":507018}
I20260709 13:17:08.528600 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.531751 10130 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:17:08.532729 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161599us
I20260709 13:17:08.535231 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.543663 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.545722 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2475) took 1004 ms. Trace:
I20260709 13:17:08.546357 10130 rpcz_store.cc:276] 0709 13:17:07.541003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.514705 (+973702us) service_pool.cc:224] Handling call
0709 13:17:08.531397 (+ 16692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.538738 (+  7341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.545599 (+  6861us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.492902 12163 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.553262 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:08.554896 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98041us
I20260709 13:17:08.520493 12176 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:08.519068 12175 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:08.534206 12036 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:08.562469 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 158777us
I20260709 13:17:08.556021 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.567101 10130 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:08.569986 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.570384 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2476) took 1029 ms. Trace:
I20260709 13:17:08.570837 10130 rpcz_store.cc:276] 0709 13:17:07.541320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.553562 (+1012242us) service_pool.cc:224] Handling call
0709 13:17:08.566809 (+ 13247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.569924 (+  3115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.570296 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.550269 11938 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.550518 11513 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:08.574909 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.555545 12094 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:08.553423 12163 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:08.578754 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.579552 10130 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.571311 12014 raft_consensus.cc:2764] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:17:08.610653 12181 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:08.576298 12094 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:08.565778 12151 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:17:08.660300 12154 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.661083 12154 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.662197 11838 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:08.663455 12152 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.664186 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:08.665239 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 139) took 1286 ms. Trace:
I20260709 13:17:08.665710 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.665593 10894 rpcz_store.cc:276] 0709 13:17:07.378617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.388952 (+ 10335us) service_pool.cc:224] Handling call
0709 13:17:08.615409 (+1226457us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:08.665113 (+ 49704us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5718784}
I20260709 13:17:08.666572 10128 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:08.611685 12180 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:08.668918 12180 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:08.670499 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1063) took 1603 ms. Trace:
I20260709 13:17:08.670020 12180 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.670248 11991 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.646255 11938 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:08.648427 11879 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.675251 10129 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:08.675879 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:08.677163 12154 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:08.676065 12135 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.565827 11912 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:17:08.675917 12014 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.683954 12014 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:08.685125 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1348) took 1393 ms (client timeout 1631 ms). Trace:
W20260709 13:17:08.685710 10201 rpcz_store.cc:269] 0709 13:17:07.291803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.291984 (+   181us) service_pool.cc:224] Handling call
0709 13:17:08.684818 (+1392834us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:08.685028 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860561664}
W20260709 13:17:08.652969 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97069us
I20260709 13:17:08.671062 10122 rpcz_store.cc:276] 0709 13:17:07.066983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.357312 (+1290329us) service_pool.cc:224] Handling call
0709 13:17:08.364843 (+  7531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.451406 (+ 86563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.670104 (+218698us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b54000a00d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:08.670399 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110299904}
I20260709 13:17:08.677809 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2474) took 1137 ms. Trace:
I20260709 13:17:08.674710 11874 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:08.658193 12036 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.660035 12094 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.611249 12177 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:17:08.644325 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 54704.9us
W20260709 13:17:08.671434 10198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1329) took 1710 ms (client timeout 2118 ms). Trace:
I20260709 13:17:08.673228 12152 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.643613 12145 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.673887 12173 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:08.673830 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1066) took 1599 ms. Trace:
I20260709 13:17:08.728309 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.677109 10126 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:08.698300 10050 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:17:08.729494 10054 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:08.729650 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2471) took 1416 ms. Trace:
I20260709 13:17:08.656837 10130 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:08.705792 10055 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:08.709861 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.730796 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.731544 10048 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:08.718130 10198 rpcz_store.cc:269] 0709 13:17:06.960823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.960980 (+   157us) service_pool.cc:224] Handling call
0709 13:17:08.671160 (+1710180us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540006ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:08.671348 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003439872}
W20260709 13:17:08.718545 10020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.572s (SENT) [suppressed 25 similar messages]
I20260709 13:17:08.732138 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.722985 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.728219 10128 rpcz_store.cc:276] 0709 13:17:07.067822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.665298 (+1597476us) service_pool.cc:224] Handling call
0709 13:17:08.667062 (+  1764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.733880 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.687912 10129 rpcz_store.cc:276] 0709 13:17:07.540572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.458314 (+917742us) service_pool.cc:224] Handling call
0709 13:17:08.458766 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.675770 (+217004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.677323 (+  1553us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540015ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:08.677720 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497057664}
I20260709 13:17:08.732179 10051 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:08.746419 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.755963 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:08.761591 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:08.705451 12036 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.716590 12151 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.764592 10130 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:17:08.771662 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2477) took 1230 ms. Trace:
I20260709 13:17:08.772125 10130 rpcz_store.cc:276] 0709 13:17:07.541602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.731033 (+1189431us) service_pool.cc:224] Handling call
0709 13:17:08.764218 (+ 33185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.771315 (+  7097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.771582 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.704962 12171 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:08.701391 11793 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.730017 10126 rpcz_store.cc:276] 0709 13:17:07.313431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.675503 (+1362072us) service_pool.cc:224] Handling call
0709 13:17:08.729572 (+ 54069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.775061 10126 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:08.776144 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2486) took 1226 ms. Trace:
I20260709 13:17:08.776600 10126 rpcz_store.cc:276] 0709 13:17:07.549933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.774339 (+1224406us) service_pool.cc:224] Handling call
0709 13:17:08.774793 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.775767 (+   974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.776058 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.770690 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:08.779484 10122 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:08.785107 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:08.789422 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.792524 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:08.733131 12176 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:08.796779 10056 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:08.759100 12073 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:08.798722 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.808684 10128 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:08.810484 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2484) took 1261 ms. Trace:
I20260709 13:17:08.806840 10130 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:08.813112 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2485) took 1263 ms. Trace:
I20260709 13:17:08.813555 10130 rpcz_store.cc:276] 0709 13:17:07.549583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.773393 (+1223810us) service_pool.cc:224] Handling call
0709 13:17:08.806528 (+ 33135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.812785 (+  6257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.813039 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.803460 10122 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:08.816536 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2498) took 1219 ms. Trace:
I20260709 13:17:08.817078 10122 rpcz_store.cc:276] 0709 13:17:07.596722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.787528 (+1190806us) service_pool.cc:224] Handling call
0709 13:17:08.802685 (+ 15157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.816099 (+ 13414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.816447 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.793618 10126 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:08.820771 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2487) took 1270 ms. Trace:
I20260709 13:17:08.805269 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:08.822548 10126 rpcz_store.cc:276] 0709 13:17:07.550228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.777661 (+1227433us) service_pool.cc:224] Handling call
0709 13:17:08.778253 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.820294 (+ 42041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.820669 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:08.812184 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 69190.1us
I20260709 13:17:08.810847 10128 rpcz_store.cc:276] 0709 13:17:07.549194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.770955 (+1221761us) service_pool.cc:224] Handling call
0709 13:17:08.808202 (+ 37247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.810105 (+  1903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.810386 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.826282 10128 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:08.833693 10050 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:08.680408 12135 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.759701 12132 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:08.716149 11849 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:08.836560 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:08.796783 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:08.843545 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.771997 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:08.769658 12018 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:08.753693 11838 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:17:08.744724 12152 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:08.837941 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.836180 12173 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:08.842029 12175 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:08.853794 12073 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:08.850003 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 306406us
I20260709 13:17:08.852514 10125 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:17:08.806654 12178 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:08.786442 12036 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.801025 11853 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.804986 12177 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:08.802408 11793 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:08.890352 10047 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:08.845762 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.803974 12181 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:17:08.858210 12145 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.914589 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:08.920535 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.804525 12185 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:17:08.930491 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 238949us
I20260709 13:17:08.930544 10900 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:08.881789 12175 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.865136 10894 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:08.919099 12178 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.794512 12151 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:08.919931 12022 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.009s
I20260709 13:17:08.945041 10128 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:08.888857 12090 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:08.949533 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97599.2us
I20260709 13:17:08.910848 12018 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:08.957690 10122 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:08.959165 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.977535 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1077) took 1016 ms. Trace:
I20260709 13:17:08.978025 10128 rpcz_store.cc:276] 0709 13:17:07.961007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.825428 (+864421us) service_pool.cc:224] Handling call
0709 13:17:08.826077 (+   649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.976921 (+150844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:08.977442 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:08.964206 12019 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:08.979228 12191 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:08.961920 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47454.7us
I20260709 13:17:08.927083 12177 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.980465 12177 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.983433 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:08.956175 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:08.985553 10893 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.979789 12191 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:08.988729 10864 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.511s (SENT)
I20260709 13:17:08.989020 12191 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:08.989343 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:08.962759 12175 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:08.927412 11853 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.005059 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1073) took 1045 ms. Trace:
I20260709 13:17:09.010337 10130 rpcz_store.cc:276] 0709 13:17:07.959675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.814472 (+854797us) service_pool.cc:224] Handling call
0709 13:17:08.814855 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.815141 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:08.936697 (+121556us) log.cc:844] Serialized 14 byte log entry
0709 13:17:09.004539 (+ 67842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.004654 (+   115us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:09.004719 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.004804 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.004976 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16317,"thread_start_us":155022,"threads_started":2}
W20260709 13:17:09.016430 10864 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.662s (SENT)
I20260709 13:17:08.976194 12082 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:08.975811 12178 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.041411 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:09.046834 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:08.992259 11938 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:08.966326 12145 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.031231 12082 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.988260 12022 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.092823 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:08.997874 12073 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.038339 12173 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.071877 11849 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.064868 12195 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.101181 12213 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:09.094313 12082 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:08.995383 12184 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:17:09.091957 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:09.140520 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 47820.4us
W20260709 13:17:09.132241 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 95833.8us
I20260709 13:17:09.101613 12205 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:09.084605 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 140) took 1681 ms. Trace:
I20260709 13:17:09.141384 12213 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.074801 11938 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.132874 11849 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:09.159547 12205 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.162302 12119 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.085355 12135 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:09.104027 12017 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.000s
I20260709 13:17:09.087162 10899 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:17:09.191250 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 99755.5us
W20260709 13:17:09.094980 11805 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.572s	user 0.016s	sys 0.004s
I20260709 13:17:09.169916 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:09.195537 10053 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:09.170620 10049 pending_rounds.cc:85] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:17:09.196566 10049 pending_rounds.cc:107] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:09.167315 12213 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.169047 11849 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.189260 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:09.084260 12151 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.196923 10049 raft_consensus.cc:2889] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:09.170769 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:09.151495 11793 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.158344 10892 rpcz_store.cc:276] 0709 13:17:07.403597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.427688 (+ 24091us) service_pool.cc:224] Handling call
0709 13:17:07.728178 (+300490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:07.729894 (+  1716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.084266 (+1354372us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54006a0250. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:09.084500 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420604160}
I20260709 13:17:09.101627 12152 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.104439 12018 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.217409 10050 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.135633 12195 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.264909 12201 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.265890 11938 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.265671 12201 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.266515 11938 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.219440 12145 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.267558 12145 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:09.268107 12145 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.268611 12145 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.269625 12174 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:09.270138 12174 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:09.224329 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:09.077798 12202 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:09.245256 12205 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.196831 12075 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:09.231405 12022 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.266346 12195 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.244362 10131 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:09.221055 11849 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:09.272636 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48488.2us
I20260709 13:17:09.271479 11938 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.273351 12202 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.285007 12202 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:09.277305 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 85003.9us
I20260709 13:17:09.277966 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1060) took 2243 ms. Trace:
I20260709 13:17:09.273813 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:09.257899 10893 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:09.282083 12036 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.283342 12201 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.242245 11853 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:09.271193 12163 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.667s	user 0.013s	sys 0.002s
I20260709 13:17:09.237572 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 403) took 2078 ms. Trace:
I20260709 13:17:09.253341 12152 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:09.199463 12017 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.286459 10131 rpcz_store.cc:276] 0709 13:17:07.034550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.243542 (+2208992us) service_pool.cc:224] Handling call
0709 13:17:09.243991 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.277613 (+ 33622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.277891 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:09.299872 10020 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.239s (SENT)
I20260709 13:17:09.332612 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:09.294528 12090 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:09.208222 11805 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.325515 10899 rpcz_store.cc:276] 0709 13:17:07.159711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.163967 (+  4256us) service_pool.cc:224] Handling call
0709 13:17:09.085521 (+1921554us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:09.085730 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.193596 (+107866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.230200 (+ 36604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89117568}
I20260709 13:17:09.299510 10894 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:09.335700 12075 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:17:09.332187 12152 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.343648 11938 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.340211 12201 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:09.354568 10050 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:09.282083 12191 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.357097 12183 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:09.291371 12177 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.365523 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:09.369639 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:09.362936 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:09.360291 12152 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:09.330309 12163 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.371908 12171 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:09.375005 12171 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:09.376148 12171 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.373262 12182 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:09.373235 11938 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:09.391278 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.396251 10895 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:09.403756 12050 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.406361 12050 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.396687 11938 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.405470 12145 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.390853 12182 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.418115 12191 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.415087 11938 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.420826 12036 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.404680 11805 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:09.423815 12179 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.856s	user 0.000s	sys 0.003s
I20260709 13:17:09.359496 12174 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.411520 12022 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.442797 11875 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.442457 12022 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.445375 12017 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.446171 12022 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.430601 12182 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.452605 12151 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:09.453295 12151 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:09.440984 12050 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.466311 12154 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:09.469648 12163 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.449510 12174 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:09.452853 12213 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.494154 12050 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.487196 12022 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:09.470168 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1058) took 2436 ms (client timeout 1841 ms). Trace:
I20260709 13:17:09.487617 12082 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.467134 12154 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:09.506165 10124 rpcz_store.cc:269] 0709 13:17:07.034095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:07.044988 (+ 10893us) service_pool.cc:224] Handling call
0709 13:17:09.469841 (+2424853us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54003d4354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:09.470048 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229914496}
I20260709 13:17:09.507241 12022 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:09.468762 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 453210us
I20260709 13:17:09.512988 12096 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.493551 12175 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.535202 12022 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.535390 12213 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.510246 12178 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.544047 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:09.544968 12175 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.545185 10896 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:09.537338 12082 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.579984 12019 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.580900 12019 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:09.580960 12127 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.581838 12127 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.516299 12145 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.508801 12202 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.535739 11849 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.586936 11849 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.587459 12202 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.532737 12177 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.551539 12175 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.551796 11805 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.630905 11805 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.633627 11805 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.552049 12036 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:09.552439 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.634393 12036 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.558285 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:09.565834 12188 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.772s	user 0.001s	sys 0.001s
I20260709 13:17:09.568508 12014 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.571375 11879 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.496186 12151 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.510578 12050 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.625173 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:09.628912 12175 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.630148 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:09.643249 12151 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.538503 12152 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.641522 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 319) took 1696 ms. Trace:
I20260709 13:17:09.638646 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:09.653936 12178 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.652444 10898 rpcz_store.cc:276] 0709 13:17:07.945060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.245196 (+300136us) service_pool.cc:224] Handling call
0709 13:17:08.272843 (+ 27647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:08.462670 (+189827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:08.462707 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:08.515919 (+ 53212us) log.cc:844] Serialized 52 byte log entry
0709 13:17:08.574084 (+ 58165us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:08.574279 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.172382 (+598103us) spinlock_profiling.cc:243] Waited 55.4 ms on lock 0x7b5400604854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:09.172472 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:09.564895 (+392423us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:09.641097 (+ 76202us) raft_consensus.cc:1685] finished
0709 13:17:09.641133 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.641332 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36414,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":35450,"spinlock_wait_cycles":142819584,"thread_start_us":191275,"threads_started":2,"wal-append.queue_time_us":252692}
I20260709 13:17:09.627388 12132 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.654665 10124 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:09.662920 10126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.001s	sys 0.020s
W20260709 13:17:09.663486 10126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.002s	sys 0.021s
W20260709 13:17:09.665352 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.025s	sys 0.026s
W20260709 13:17:09.665814 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.025s	sys 0.026s
W20260709 13:17:09.666911 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.018s	sys 0.021s
W20260709 13:17:09.669526 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.018s	sys 0.021s
W20260709 13:17:09.669955 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 824198us
I20260709 13:17:09.681002 10896 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:09.659807 11513 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.013s	sys 0.030s
W20260709 13:17:09.685277 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.025s	sys 0.040s
I20260709 13:17:09.654592 12127 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:09.697777 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.025s	sys 0.040s
I20260709 13:17:09.700852 12142 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.637034 12154 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:09.657822 10897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.033s	sys 0.019s
I20260709 13:17:09.702311 12025 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:17:09.711690 12142 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:09.696734 11513 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.013s	sys 0.030s
I20260709 13:17:09.664264 12132 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.687302 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:09.723515 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:09.727080 10124 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:09.713285 12178 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:09.724213 12127 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:17:09.718858 10897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.034s	sys 0.024s
I20260709 13:17:09.739053 11805 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:09.741974 11805 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.730530 11879 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:09.748034 10128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.000s	sys 0.066s
W20260709 13:17:09.748755 10128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.000s	sys 0.066s
I20260709 13:17:09.755455 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1088) took 1364 ms. Trace:
I20260709 13:17:09.756137 10128 rpcz_store.cc:276] 0709 13:17:08.391097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.980036 (+588939us) service_pool.cc:224] Handling call
0709 13:17:08.980395 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:08.980619 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:09.076454 (+ 95835us) log.cc:844] Serialized 14 byte log entry
0709 13:17:09.754619 (+678165us) log.cc:844] Serialized 14 byte log entry
0709 13:17:09.754893 (+   274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:09.754964 (+    71us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:09.755028 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.755153 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:09.755371 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100808,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":2667,"thread_start_us":767936,"threads_started":2}
I20260709 13:17:09.767704 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:09.768603 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.772471 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:09.775306 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:09.778541 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:09.779659 10202 raft_consensus.cc:2368] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6.
I20260709 13:17:09.720441 12017 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:17:09.747608 12154 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:09.784288 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.791591 12036 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.783004 12134 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:09.791993 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:09.792757 12142 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:09.793126 12092 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:09.764318 11849 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.794142 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:09.793952 11962 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:09.792742 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:09.816339 10196 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:09.816417 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:09.744802 11879 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.745476 12178 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.801167 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:09.828897 10198 raft_consensus.cc:2368] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 3. Current term is 4.
I20260709 13:17:09.731220 11859 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:17:09.805931 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:09.813480 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:09.836741 10050 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:09.838191 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:09.840090 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:09.841010 10128 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:09.842337 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:09.804509 10205 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:09.732318 11938 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:09.781520 12213 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:09.825786 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:09.829303 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:09.833647 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:09.756409 12127 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
W20260709 13:17:09.835219 10130 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.031s	sys 0.053s
I20260709 13:17:09.792773 12064 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:17:09.845599 10205 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:09.817296 10202 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:17:09.794685 12212 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 13:17:09.807628 12051 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:17:09.838989 10050 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:10.280109 10130 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.256s	user 0.040s	sys 0.063s
I20260709 13:17:09.922600 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:10.143805 10022 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.249s	user 0.013s	sys 0.050s
I20260709 13:17:10.281287 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:10.281633 10022 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.387s	user 0.017s	sys 0.050s
W20260709 13:17:10.022617 12073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.004s	sys 0.028s
I20260709 13:17:10.282519 12224 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:10.283304 10204 raft_consensus.cc:1217] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-5.6]   Dedup: 5.6->[]
I20260709 13:17:10.283996 10204 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [4.5-5.6]
I20260709 13:17:09.956718 12050 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:10.284595 11879 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:10.003343 12174 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:10.111088 10049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.008s	sys 0.046s
W20260709 13:17:10.286513 10049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.013s	sys 0.047s
W20260709 13:17:10.213824 10131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.004s	sys 0.014s
I20260709 13:17:09.977293 10197 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
W20260709 13:17:10.288050 10131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.007s	sys 0.014s
I20260709 13:17:10.262962 11702 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:10.261312 10053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.004s	sys 0.013s
W20260709 13:17:10.291811 10053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.004s	sys 0.014s
I20260709 13:17:09.973456 10200 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:17:10.291939 11702 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.829121 11849 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:10.294492 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1079) took 2286 ms. Trace:
I20260709 13:17:10.024360 12132 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:10.295797 10054 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:10.024246 10900 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.017s	sys 0.015s
I20260709 13:17:09.894063 12175 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:10.297108 10900 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.022s	sys 0.017s
I20260709 13:17:10.297302 12213 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.297843 12213 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.299810 12212 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:10.300282 12212 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:10.300835 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.302309 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:09.924386 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:09.943511 10122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.062s	sys 0.081s
W20260709 13:17:10.303346 10122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.273s	user 0.066s	sys 0.093s
I20260709 13:17:10.303234 10051 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:10.304481 12036 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.305171 12127 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:10.181139 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.305756 12127 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:10.306564 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.307173 12127 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:10.307678 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:10.308001 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:10.203233 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:10.379541 10124 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.000s
I20260709 13:17:10.380053 10124 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:10.380928 10019 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.104s (SENT)
I20260709 13:17:10.381909 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:10.309162 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:10.383019 10123 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:10.294564 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:09.985441 12217 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:17:09.865469 10198 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:17:10.310227 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1078) took 2348 ms (client timeout 1694 ms). Trace:
W20260709 13:17:10.401788 10127 rpcz_store.cc:269] 0709 13:17:07.961498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.136641 (+175143us) service_pool.cc:224] Handling call
0709 13:17:10.309756 (+2173115us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54001f3454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:10.309901 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498084224}
W20260709 13:17:10.213542 12180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.014s	sys 0.019s
W20260709 13:17:10.416062 12180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.552s	user 0.014s	sys 0.023s
I20260709 13:17:10.416414 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.312319 11938 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:10.298458 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.312966 10202 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.285893 12174 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:10.314785 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 115) took 3986 ms (client timeout 2000 ms). Trace:
I20260709 13:17:10.305389 12036 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:10.288977 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.304535 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:10.426266 10199 rpcz_store.cc:269] 0709 13:17:06.328097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:06.328305 (+   208us) service_pool.cc:224] Handling call
0709 13:17:10.314434 (+3986129us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:10.314627 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3633576832}
I20260709 13:17:10.289860 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 102) took 1590 ms. Trace:
I20260709 13:17:10.427369 10131 rpcz_store.cc:276] 0709 13:17:08.699835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.346385 (+646550us) service_pool.cc:224] Handling call
0709 13:17:09.361698 (+ 15313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.361933 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:09.362267 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:17:10.289136 (+926869us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:10.289271 (+   135us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:10.289363 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.289504 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.289746 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":866,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":6545,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":926233,"threads_started":1}
I20260709 13:17:10.330567 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.332934 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:10.429270 10204 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:10.429414 10129 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.422104 10205 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:10.003041 12019 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:10.342730 12185 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.346194 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2473) took 2822 ms. Trace:
I20260709 13:17:10.350301 12106 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:10.295868 12064 consensus_queue.cc:1243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:10.352743 10055 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.354714 11991 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:10.461795 12185 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:10.354226  9953 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 2 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:10.355825  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:10.360814 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:10.361039 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.292196 10048 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:10.469429 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 322) took 1827 ms. Trace:
I20260709 13:17:10.469668 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:10.470803 10048 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:10.472162 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:10.473430 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:10.473865 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:10.474611 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:10.362725 12145 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:10.364107 10128 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:10.375093 12178 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:10.282296 12073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.008s	sys 0.029s
I20260709 13:17:10.376889 12172 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:09.864567 12152 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:09.831631 11805 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:10.382066 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:10.295073 10126 rpcz_store.cc:276] 0709 13:17:08.009585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.832005 (+822420us) service_pool.cc:224] Handling call
0709 13:17:08.832685 (+   680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.005390 (+172705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.005426 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.009566 (+  4140us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.664023 (+654457us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:09.664091 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.664207 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.190690 (+526483us) raft_consensus.cc:1685] finished
0709 13:17:10.190723 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.294404 (+103681us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4750,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":9089,"thread_start_us":725190,"threads_started":2,"wal-append.queue_time_us":991215}
I20260709 13:17:10.386051 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:10.475056 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:10.308383 12141 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:09.818112 12134 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:09.948262 12182 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:10.415174 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:10.301422 12212 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:10.423540 10196 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.000s
I20260709 13:17:10.426890 10051 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:10.326853 10054 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:10.431571 10205 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 1. Current term is 3. Ops: []
I20260709 13:17:10.447885 10125 rpcz_store.cc:276] 0709 13:17:07.523488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.456881 (+933393us) service_pool.cc:224] Handling call
0709 13:17:08.457302 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.917144 (+459842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.345756 (+1428612us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:10.346091 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907956480}
I20260709 13:17:10.283344 12224 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:10.378407 12175 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:10.460793 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.457386 12106 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:10.481405 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:10.387203 10056 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:10.479597 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97600.3us
I20260709 13:17:10.476198 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:10.484419 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:10.487761 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.404124 10892 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:10.476871 10048 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:10.419443 12221 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.550587 10127 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
W20260709 13:17:10.471302 12219 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:17:11.524233 10896 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 5. Ops: []
W20260709 13:17:11.124888 10129 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
I20260709 13:17:11.534164 10129 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:11.534741 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1105) took 1995 ms (client timeout 2054 ms). Trace:
W20260709 13:17:11.535382 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 103) took 1868 ms (client timeout 2137 ms). Trace:
W20260709 13:17:11.535248 10129 rpcz_store.cc:269] 0709 13:17:09.539553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.332463 (+792910us) service_pool.cc:224] Handling call
0709 13:17:11.534666 (+1202203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.536038 10127 rpcz_store.cc:269] 0709 13:17:09.666935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.474812 (+807877us) service_pool.cc:224] Handling call
0709 13:17:11.535300 (+1060488us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.537976 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1372) took 1576 ms (client timeout 1632 ms). Trace:
I20260709 13:17:10.843302 10055 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:11.538940 10051 rpcz_store.cc:269] 0709 13:17:09.961579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.304074 (+342495us) service_pool.cc:224] Handling call
0709 13:17:11.537898 (+1233824us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.539543 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1365) took 1712 ms (client timeout 2116 ms). Trace:
I20260709 13:17:11.535638 10123 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:11.540166 10055 rpcz_store.cc:269] 0709 13:17:09.827033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.828954 (+  1921us) service_pool.cc:224] Handling call
0709 13:17:11.539432 (+1710478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.533166 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1366) took 1702 ms. Trace:
I20260709 13:17:11.541785 10054 rpcz_store.cc:276] 0709 13:17:09.833150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.833317 (+   167us) service_pool.cc:224] Handling call
0709 13:17:11.533073 (+1699756us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.538029 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 108) took 1176 ms (client timeout 1548 ms). Trace:
I20260709 13:17:11.543745 10126 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:11.538147 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:10.681018 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 132623us
I20260709 13:17:11.538906 10196 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
W20260709 13:17:11.545156 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1363) took 1722 ms (client timeout 2050 ms). Trace:
W20260709 13:17:11.545765 10196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 124) took 1771 ms (client timeout 1970 ms). Trace:
W20260709 13:17:11.546558 10196 rpcz_store.cc:269] 0709 13:17:09.774230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.774400 (+   170us) service_pool.cc:224] Handling call
0709 13:17:11.545667 (+1771267us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:10.825141 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.548128 10050 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:10.715664 10095 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.058s	user 0.022s	sys 0.042s
W20260709 13:17:11.544217 10128 rpcz_store.cc:269] 0709 13:17:10.361309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.475892 (+114583us) service_pool.cc:224] Handling call
0709 13:17:11.537943 (+1062051us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.548770 10095 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.897s	user 0.049s	sys 0.089s
I20260709 13:17:11.548831 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1103) took 1087 ms. Trace:
I20260709 13:17:10.604692 10204 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 3. Ops: []
W20260709 13:17:11.549432 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:10.416109 10200 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:11.550686 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:11.549813 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.95426e+06us
I20260709 13:17:11.543560 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:10.789273 10203 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.005s	sys 0.013s
I20260709 13:17:11.613015 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2487) took 1254 ms. Trace:
I20260709 13:17:10.475420 12145 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:11.613015 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2486) took 1265 ms. Trace:
I20260709 13:17:10.479075 11702 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:11.613615 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:11.622040 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 68115.9us
I20260709 13:17:11.622675 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:11.623580 10196 raft_consensus.cc:2368] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6.
I20260709 13:17:10.479977 11805 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:10.479664 11849 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:10.558395 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 197700us
I20260709 13:17:11.554704 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2499) took 1263 ms. Trace:
I20260709 13:17:10.612869 10202 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:11.614283 10204 rpcz_store.cc:276] 0709 13:17:10.285290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.430131 (+144841us) service_pool.cc:224] Handling call
0709 13:17:10.554790 (+124659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.549883 (+995093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.550217 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.544533 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1109) took 1743 ms (client timeout 2213 ms). Trace:
I20260709 13:17:11.562885 10199 raft_consensus.cc:2368] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:17:11.549468 10050 rpcz_store.cc:276] 0709 13:17:10.461592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.481082 (+ 19490us) service_pool.cc:224] Handling call
0709 13:17:11.548730 (+1067648us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.530915 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.004s	sys 0.007s
W20260709 13:17:11.552047 10122 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.006s	sys 0.013s
I20260709 13:17:11.616272 10054 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:11.615532 10203 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.774s	user 0.012s	sys 0.030s
W20260709 13:17:11.545670 10056 rpcz_store.cc:269] 0709 13:17:09.822738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.823853 (+  1115us) service_pool.cc:224] Handling call
0709 13:17:11.545060 (+1721207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:10.740957 12173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.181s	user 0.007s	sys 0.060s
I20260709 13:17:11.613761 10205 rpcz_store.cc:276] 0709 13:17:10.285669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.430975 (+145306us) service_pool.cc:224] Handling call
0709 13:17:10.431371 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.539582 (+1108211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.539829 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.668572 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:10.620105 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:10.777367 12018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.341s	user 0.005s	sys 0.068s
I20260709 13:17:11.670230 10050 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:17:11.670780 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.05086e+06us
I20260709 13:17:11.670468 10198 rpcz_store.cc:276] 0709 13:17:10.291129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.427099 (+135970us) service_pool.cc:224] Handling call
0709 13:17:11.554589 (+1127490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.621513 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 109) took 1259 ms. Trace:
W20260709 13:17:10.765954 12171 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.338s	user 0.017s	sys 0.057s
W20260709 13:17:11.690130 12171 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.262s	user 0.017s	sys 0.078s
W20260709 13:17:11.691190 10170 connection.cc:256] RPC call timeout handler was delayed by -1.0579628220000359s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
I20260709 13:17:11.553493 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 419) took 1246 ms. Trace:
W20260709 13:17:11.691583 10156 heartbeater.cc:646] Failed to heartbeat to 127.8.183.126:40341 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.183.126:40341 timed out after 15.000s (SENT)
I20260709 13:17:11.691996 10892 rpcz_store.cc:276] 0709 13:17:10.307012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.307257 (+   245us) service_pool.cc:224] Handling call
0709 13:17:11.545718 (+1238461us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.693077 12173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.133s	user 0.031s	sys 0.088s
I20260709 13:17:11.694444 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.695272 10205 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:11.696368 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.697714 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.698979 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.699144 12021 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:11.700237 10128 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:11.700811 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1115) took 1142 ms. Trace:
I20260709 13:17:11.701188 10128 rpcz_store.cc:276] 0709 13:17:10.558975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.549069 (+990094us) service_pool.cc:224] Handling call
0709 13:17:11.700733 (+151664us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.701669 12021 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:11.702339 12021 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:11.702445 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.710783 10128 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:17:11.300565 12227 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:17:11.671645 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1084) took 1387 ms. Trace:
I20260709 13:17:11.736109 10050 rpcz_store.cc:276] 0709 13:17:10.283752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.668279 (+1384527us) service_pool.cc:224] Handling call
0709 13:17:11.671503 (+  3224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.668944 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1097) took 1324 ms. Trace:
I20260709 13:17:11.672542 10127 rpcz_store.cc:276] 0709 13:17:10.361587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.537798 (+1176211us) service_pool.cc:224] Handling call
0709 13:17:11.621403 (+ 83605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.673729 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2489) took 1387 ms. Trace:
W20260709 13:17:11.669225 10126 rpcz_store.cc:269] 0709 13:17:09.801515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.483955 (+682440us) service_pool.cc:224] Handling call
0709 13:17:11.544432 (+1060477us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.668046 10122 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.364s	user 0.006s	sys 0.017s
I20260709 13:17:11.676884 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:11.594482 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:10.597862 12151 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.949s	user 0.015s	sys 0.007s
W20260709 13:17:11.404866 12232 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:10.615389 12073 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:11.182931 12177 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.689821 12106 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:11.737671 10054 rpcz_store.cc:276] 0709 13:17:10.344620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.543080 (+1198460us) service_pool.cc:224] Handling call
0709 13:17:11.668873 (+125793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.738456 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.003726 12067 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:10.478045 12172 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:11.668046 11805 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.686801 10124 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:17:11.759351 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2504) took 2206 ms (client timeout 2104 ms). Trace:
W20260709 13:17:11.760042 10124 rpcz_store.cc:269] 0709 13:17:09.552669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.381532 (+828863us) service_pool.cc:224] Handling call
0709 13:17:11.759222 (+1377690us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.761582 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.541837 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:11.513839 12154 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.704s	user 0.018s	sys 0.000s
W20260709 13:17:11.442171 12202 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.093s	user 0.016s	sys 0.008s
W20260709 13:17:11.212886 11513 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.447s	user 0.022s	sys 0.043s
I20260709 13:17:11.671931 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:11.703675 10197 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:17:11.705339 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.412s	user 0.034s	sys 0.043s
I20260709 13:17:11.689927 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:11.812366 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.519s	user 0.035s	sys 0.043s
I20260709 13:17:11.813972 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.624402 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:11.699000 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.726135 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2495) took 1437 ms. Trace:
I20260709 13:17:11.815944 10205 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
W20260709 13:17:11.669042 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.008s	sys 0.007s
I20260709 13:17:11.012777 10894 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:11.565596 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2497) took 1261 ms. Trace:
I20260709 13:17:11.745486 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.613695 12036 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:11.747462 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.749034 10201 raft_consensus.cc:2410] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:11.739320 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.749285 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.531934 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.737051 12214 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:17:11.825819 10051 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:11.827453 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 155599us
I20260709 13:17:11.746862 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:11.741285 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:11.828313 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.740170 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2500) took 1036 ms. Trace:
I20260709 13:17:11.829401 10196 rpcz_store.cc:276] 0709 13:17:10.703789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.676405 (+972616us) service_pool.cc:224] Handling call
0709 13:17:11.740048 (+ 63643us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.750715 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.766680 12073 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:11.709900 12018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.274s	user 0.015s	sys 0.078s
W20260709 13:17:11.737774 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2510) took 2183 ms (client timeout 2876 ms). Trace:
W20260709 13:17:11.846648 10128 rpcz_store.cc:269] 0709 13:17:09.554438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.702087 (+2147649us) service_pool.cc:224] Handling call
0709 13:17:11.737686 (+ 35599us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.738765  9950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 195) took 15527 ms (client timeout 15000 ms). Trace:
I20260709 13:17:11.743173 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:11.848233 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1093) took 1561 ms (client timeout 1733 ms). Trace:
W20260709 13:17:11.848721 10048 rpcz_store.cc:269] 0709 13:17:10.286322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.474378 (+188056us) service_pool.cc:224] Handling call
0709 13:17:11.848151 (+1373773us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.849889 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:11.850740 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1376) took 1436 ms. Trace:
I20260709 13:17:11.851104 10048 rpcz_store.cc:276] 0709 13:17:10.414904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.849553 (+1434649us) service_pool.cc:224] Handling call
0709 13:17:11.850670 (+  1117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.852108 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:11.742456 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.439s	user 0.009s	sys 0.013s
W20260709 13:17:10.765867 12211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.014s	sys 0.040s
I20260709 13:17:11.549736 12127 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.743516 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:11.755620 10127 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:11.016839 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 138) took 1847 ms. Trace:
I20260709 13:17:11.857359 10049 rpcz_store.cc:276] 0709 13:17:09.169757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.169912 (+   155us) service_pool.cc:224] Handling call
0709 13:17:09.170354 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.240255 (+ 69901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.240309 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.240979 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:17:10.287058 (+1046079us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:10.306917 (+ 19859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.307044 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.016467 (+709423us) raft_consensus.cc:1685] finished
0709 13:17:11.016505 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.016739 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1029167,"threads_started":1,"wal-append.queue_time_us":1714871}
W20260709 13:17:11.858253 10020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.004s	sys 0.012s
W20260709 13:17:11.859153 10020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.004s	sys 0.012s
I20260709 13:17:11.762862 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.859906 10056 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.746865 12106 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:11.682539 10123 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:11.774145  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:11.867911 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1101) took 2342 ms (client timeout 2165 ms). Trace:
W20260709 13:17:11.868419 10123 rpcz_store.cc:269] 0709 13:17:09.525893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.414794 (+888901us) service_pool.cc:224] Handling call
0709 13:17:11.867820 (+1453026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.553647 12175 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:11.785562 10864 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.011s	sys 0.004s
W20260709 13:17:11.872833 10864 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.012s	sys 0.005s
I20260709 13:17:11.838315 12173 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:11.873247 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.20605e+06us
W20260709 13:17:11.873414  9953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.020s	sys 0.013s
W20260709 13:17:11.873953  9953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.020s	sys 0.013s
I20260709 13:17:11.873781 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.874850 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1116) took 1259 ms. Trace:
I20260709 13:17:11.686090 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:11.772753 12213 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.474s	user 0.015s	sys 0.001s
I20260709 13:17:11.875275 10128 rpcz_store.cc:276] 0709 13:17:10.615890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.847615 (+1231725us) service_pool.cc:224] Handling call
0709 13:17:11.874775 (+ 27160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.875844 10129 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:11.856344 12211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.964s	user 0.024s	sys 0.040s
I20260709 13:17:11.767202 11805 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:11.777137 12162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.008s	sys 0.072s
W20260709 13:17:11.803015 10900 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.505s	user 0.006s	sys 0.010s
W20260709 13:17:11.896109 10900 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.598s	user 0.006s	sys 0.011s
I20260709 13:17:11.747583 11849 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:17:11.746389 12177 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:11.621416 12221 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:11.806281 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1088) took 1521 ms. Trace:
I20260709 13:17:11.936951 10055 rpcz_store.cc:276] 0709 13:17:10.284889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.541384 (+1256495us) service_pool.cc:224] Handling call
0709 13:17:11.806191 (+264807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.938290 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:11.938829 10055 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:11.806892 10124 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:11.815806 10199 rpcz_store.cc:276] 0709 13:17:10.288477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.427587 (+139110us) service_pool.cc:224] Handling call
0709 13:17:11.725998 (+1298411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.939502 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1106) took 2181 ms. Trace:
I20260709 13:17:11.939555 10129 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:11.939868 10124 rpcz_store.cc:276] 0709 13:17:09.758115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.761165 (+2003050us) service_pool.cc:224] Handling call
0709 13:17:11.939409 (+178244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.940253 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 107) took 1579 ms (client timeout 2042 ms). Trace:
W20260709 13:17:10.993543 12189 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 2.057s	user 0.004s	sys 0.009s
I20260709 13:17:11.775022 12019 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:11.827688 10131 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:11.948215 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 110) took 1574 ms (client timeout 1685 ms). Trace:
I20260709 13:17:11.948324 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:11.950232 10131 rpcz_store.cc:269] 0709 13:17:10.373994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.514988 (+140994us) service_pool.cc:224] Handling call
0709 13:17:11.948078 (+1433090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.829838 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:10.550220 12132 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:11.832258 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1117) took 1186 ms. Trace:
I20260709 13:17:11.952710 10126 rpcz_store.cc:276] 0709 13:17:10.645582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.739011 (+1093429us) service_pool.cc:224] Handling call
0709 13:17:11.832166 (+ 93155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.834746 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:11.955359 10198 raft_consensus.cc:2376] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3. Re-sending same reply.
I20260709 13:17:11.957037 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.925426 12224 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:11.836105 10050 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:11.811247 12202 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.026s	sys 0.013s
I20260709 13:17:11.788686 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 320) took 3694 ms. Trace:
I20260709 13:17:11.840762 10196 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: [3.2-4.3]
I20260709 13:17:11.972114 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2488) took 1686 ms. Trace:
W20260709 13:17:11.811578 11513 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.058s	user 0.027s	sys 0.070s
I20260709 13:17:11.839315 12036 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:11.824612 10200 rpcz_store.cc:276] 0709 13:17:10.289219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.294151 (+  4932us) service_pool.cc:224] Handling call
0709 13:17:11.550549 (+1256398us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.853194 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.550s	user 0.009s	sys 0.013s
I20260709 13:17:11.853735 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:11.972885 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:11.973841 10896 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 2.
I20260709 13:17:11.854588 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.975339 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1085) took 1691 ms. Trace:
I20260709 13:17:11.975767 10048 rpcz_store.cc:276] 0709 13:17:10.284054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.854275 (+1570221us) service_pool.cc:224] Handling call
0709 13:17:11.975262 (+120987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.882201 12213 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:11.859537 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:11.987864 10049 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:11.989477 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:11.861323 10204 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 5. Ops: []
I20260709 13:17:11.991838 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2490) took 1705 ms. Trace:
I20260709 13:17:11.992411 10204 rpcz_store.cc:276] 0709 13:17:10.286702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.843716 (+1557014us) service_pool.cc:224] Handling call
0709 13:17:11.861097 (+ 17381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:11.991428 (+130331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.991740 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:11.862679 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 852949us
W20260709 13:17:11.847955  9950 rpcz_store.cc:269] 0709 13:16:56.211730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.215413 (+  3683us) service_pool.cc:224] Handling call
0709 13:17:01.421224 (+5205811us) spinlock_profiling.cc:243] Waited 3.26 s on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:17:11.738570 (+10317346us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:04.821332 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:05.025667 (+204335us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:05.029082 (+  3415us) write_op.cc:435] Acquired schema lock
0709 13:17:05.038173 (+  9091us) tablet.cc:662] Decoding operations
0709 13:17:05.253297 (+215124us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:05.258051 (+  4754us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:05.266209 (+  8158us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:17:05.370233 (+104024us) tablet.cc:701] Row locks acquired
0709 13:17:05.370256 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:17:05.370481 (+   225us) write_op.cc:270] Start()
0709 13:17:05.370647 (+   166us) write_op.cc:276] Timestamp: P: 1783603025370422 usec, L: 0
0709 13:17:05.370685 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.371689 (+  1004us) log.cc:844] Serialized 2906 byte log entry
0709 13:17:05.633529 (+261840us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:05.866347 (+232818us) op_driver.cc:464] REPLICATION: finished
0709 13:17:06.322003 (+455656us) write_op.cc:301] APPLY: starting
0709 13:17:06.322103 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:06.322317 (+   214us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:06.322337 (+    20us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:06.351844 (+ 29507us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:06.352122 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:17:06.352185 (+    63us) write_op.cc:312] APPLY: finished
0709 13:17:06.352506 (+   321us) log.cc:844] Serialized 59 byte log entry
0709 13:17:06.352708 (+   202us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:06.352853 (+   145us) write_op.cc:454] Released schema lock
0709 13:17:06.352922 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:06.353361 (+   439us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:06.353392 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":296310,"reported_tablets":20,"spinlock_wait_cycles":7502947200,"child_traces":[["op",{"apply.queue_time_us":450906,"apply.run_cpu_time_us":3775,"apply.run_wall_time_us":31361,"num_ops":7,"prepare.queue_time_us":84023,"prepare.run_cpu_time_us":22985,"prepare.run_wall_time_us":640696,"raft.queue_time_us":161835,"raft.run_cpu_time_us":40178,"raft.run_wall_time_us":471374,"replication_time_us":478324,"spinlock_wait_cycles":53249280,"thread_start_us":532223,"threads_started":4,"wal-append.queue_time_us":110767,"wal-append.run_cpu_time_us":41729,"wal-append.run_wall_time_us":1872079}]]}
W20260709 13:17:11.861330 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1100) took 2338 ms (client timeout 3031 ms). Trace:
I20260709 13:17:11.837630 11702 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:11.756304 12067 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:11.859519 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:11.891171 12176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.572s	user 0.016s	sys 0.030s
W20260709 13:17:11.890779 12165 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.004s	user 0.027s	sys 0.041s
I20260709 13:17:12.003172 10200 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:11.910951 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:11.782106 12021 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.003728 10892 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:12.003863 12176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.685s	user 0.019s	sys 0.030s
W20260709 13:17:12.003773 12165 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.117s	user 0.030s	sys 0.041s
I20260709 13:17:11.909754 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:12.007002 10898 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.935266 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 415) took 2602 ms. Trace:
W20260709 13:17:11.913312 12162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.580s	user 0.009s	sys 0.072s
W20260709 13:17:11.940697 10129 rpcz_store.cc:269] 0709 13:17:10.361039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.685682 (+1324643us) service_pool.cc:224] Handling call
0709 13:17:11.940174 (+254492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.008008 10900 rpcz_store.cc:276] 0709 13:17:09.333290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.333445 (+   155us) service_pool.cc:224] Handling call
0709 13:17:09.344486 (+ 11041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:09.344875 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:10.297789 (+952914us) log.cc:844] Serialized 14 byte log entry
0709 13:17:11.934667 (+1636878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:11.934745 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:11.934820 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.934932 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.935175 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1483689,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"thread_start_us":2538789,"threads_started":2,"wal-append.queue_time_us":1654856}
I20260709 13:17:12.016664 10129 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.017414 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1096) took 2693 ms. Trace:
I20260709 13:17:11.891981 12173 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:11.944157 10051 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:11.796339 12145 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:10.470134 10891 rpcz_store.cc:276] 0709 13:17:08.642468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.642647 (+   179us) service_pool.cc:224] Handling call
0709 13:17:08.643091 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.835920 (+192829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:08.840226 (+  4306us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:08.924395 (+ 84169us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.330108 (+405713us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:09.335582 (+  5474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:09.341373 (+  5791us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:10.469117 (+1127744us) raft_consensus.cc:1685] finished
0709 13:17:10.469151 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:10.469339 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16353,"raft.queue_time_us":164031,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":25317,"thread_start_us":450290,"threads_started":2,"wal-append.queue_time_us":712932,"wal-append.run_cpu_time_us":21682,"wal-append.run_wall_time_us":1833369}
I20260709 13:17:11.738651 10202 rpcz_store.cc:276] 0709 13:17:10.286346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.302018 (+ 15672us) service_pool.cc:224] Handling call
0709 13:17:11.673624 (+1371606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.040074 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:11.999655 12134 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:11.828928 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:12.047704 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 218844us
I20260709 13:17:12.050565 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.000939 12224 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:12.052631 12162 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:12.015143 12019 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:11.963330 12132 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:11.972559 10196 rpcz_store.cc:276] 0709 13:17:10.286034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.840017 (+1553983us) service_pool.cc:224] Handling call
0709 13:17:11.840456 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:11.971713 (+131257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.972029 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.977080 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:12.077925 10048 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:11.830716 12154 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.935076 12160 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:11.983537 10205 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: [2.2-3.3]
W20260709 13:17:11.994397 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4376107400000819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 13:17:11.994568 10127 rpcz_store.cc:269] 0709 13:17:09.523089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.738048 (+2214959us) service_pool.cc:224] Handling call
0709 13:17:11.861219 (+123171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.092110 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2496) took 1803 ms. Trace:
I20260709 13:17:11.996616 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:11.997756 11702 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.969892 10897 rpcz_store.cc:276] 0709 13:17:08.082701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:08.263641 (+180940us) service_pool.cc:224] Handling call
0709 13:17:08.355989 (+ 92348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:08.576784 (+220795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:08.644044 (+ 67260us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:08.858296 (+214252us) log.cc:844] Serialized 26 byte log entry
0709 13:17:09.752239 (+893943us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:09.757661 (+  5422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:10.356003 (+598342us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540050f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:10.356078 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.735856 (+1379778us) raft_consensus.cc:1685] finished
0709 13:17:11.735904 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:11.775284 (+ 39380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54865,"raft.queue_time_us":281503,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":33073,"spinlock_wait_cycles":240094976,"thread_start_us":915441,"threads_started":2,"wal-append.queue_time_us":1170001}
W20260709 13:17:11.914083 10047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.538s	user 0.012s	sys 0.046s
I20260709 13:17:12.009804 10123 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3. Re-sending same reply.
W20260709 13:17:12.099403 10047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.723s	user 0.020s	sys 0.058s
I20260709 13:17:12.099749 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1114) took 1549 ms. Trace:
W20260709 13:17:12.010399 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 474853us
I20260709 13:17:12.100302 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1359) took 2290 ms. Trace:
I20260709 13:17:12.015241 10050 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:12.017779 10129 rpcz_store.cc:276] 0709 13:17:09.323928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.016077 (+2692149us) service_pool.cc:224] Handling call
0709 13:17:12.016468 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.017104 (+   636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.017348 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:11.943051 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.019182 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:12.023603 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1090) took 1738 ms (client timeout 2125 ms). Trace:
I20260709 13:17:12.103055 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1374) took 2110 ms. Trace:
I20260709 13:17:11.980372 12222 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:12.102098 10050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1100) took 1738 ms (client timeout 2165 ms). Trace:
I20260709 13:17:12.103574 10049 rpcz_store.cc:276] 0709 13:17:09.993112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.991031 (+1997919us) service_pool.cc:224] Handling call
0709 13:17:12.102975 (+111944us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.113677 10050 rpcz_store.cc:269] 0709 13:17:10.364949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.737433 (+1372484us) service_pool.cc:224] Handling call
0709 13:17:12.102006 (+364573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.100819 10047 rpcz_store.cc:276] 0709 13:17:09.810228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.811388 (+  1160us) service_pool.cc:224] Handling call
0709 13:17:09.819432 (+  8044us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.835451 (+ 16019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.835499 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:10.363820 (+528321us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.099905 (+1736085us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:12.099952 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.100037 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.100085 (+    48us) raft_consensus.cc:1685] finished
0709 13:17:12.100097 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.100243 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15446,"raft.queue_time_us":19564,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":48069,"thread_start_us":1723636,"threads_started":1,"wal-append.queue_time_us":1708025}
I20260709 13:17:12.114598 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:11.951208 10128 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:12.075750 10199 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:12.076920 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:11.958483 10198 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:12.104861 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.007553 12067 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:12.080214 12249 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:17:12.081218 12242 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:17:12.082038 12241 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 13:17:12.082847 12240 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:17:12.083595 12233 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:17:12.083712 12182 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:12.084161 12224 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.087046 12151 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.090682 12162 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:12.092134 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2909107400000721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:17:12.139894 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1262107400000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 13:17:12.092757 10205 rpcz_store.cc:276] 0709 13:17:10.288831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.982418 (+1693587us) service_pool.cc:224] Handling call
0709 13:17:11.983055 (+   637us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:12.091777 (+108722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.092053 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.140338 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0812107400001878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22799999999999998s for a response
I20260709 13:17:12.140208 10055 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:12.093021 10127 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:17:11.995748 10131 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:17:12.129935 12236 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.293s	user 0.000s	sys 0.002s
I20260709 13:17:12.115074 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.143877 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1097) took 2819 ms. Trace:
I20260709 13:17:12.144359 10127 rpcz_store.cc:276] 0709 13:17:09.324340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.092168 (+2767828us) service_pool.cc:224] Handling call
0709 13:17:12.092680 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.143510 (+ 50830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.143791 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.145848 10127 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:12.144487 10050 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:12.100306 10123 rpcz_store.cc:276] 0709 13:17:10.550024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.869254 (+1319230us) service_pool.cc:224] Handling call
0709 13:17:12.099662 (+230408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.116902 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:12.103462 10051 rpcz_store.cc:269] 0709 13:17:10.285482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.635060 (+1349578us) service_pool.cc:224] Handling call
0709 13:17:12.023504 (+388444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.148098 10051 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:12.148960 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1081) took 1866 ms. Trace:
I20260709 13:17:12.149425 10051 rpcz_store.cc:276] 0709 13:17:10.282817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.147565 (+1864748us) service_pool.cc:224] Handling call
0709 13:17:12.147901 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.148613 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.148882 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.145375 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:12.123559 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2505) took 2570 ms. Trace:
I20260709 13:17:12.150549 10128 rpcz_store.cc:276] 0709 13:17:09.552996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.927869 (+2374873us) service_pool.cc:224] Handling call
0709 13:17:12.123464 (+195595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.124101 10049 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:12.151777 10049 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:11.957417 12073 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.091073 12160 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.076795 12019 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:12.128798 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.130156 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:12.105814 12247 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.176s	user 0.001s	sys 0.001s
I20260709 13:17:12.132548 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.132839 10048 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:12.126185 10125 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:17:12.134128 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57377.5us
I20260709 13:17:12.102064 11702 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.137334 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.171566 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1107) took 1396 ms. Trace:
I20260709 13:17:12.138258 12151 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.123296 12145 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:12.093459 10898 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:12.141845 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1094) took 3438 ms. Trace:
I20260709 13:17:12.235975 10131 rpcz_store.cc:276] 0709 13:17:08.703771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.951047 (+3247276us) service_pool.cc:224] Handling call
0709 13:17:11.995374 (+ 44327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.141514 (+146140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.141774 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.104681 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:12.237972 10196 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:12.130713 12238 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:17:12.239100 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1082) took 1956 ms. Trace:
I20260709 13:17:12.245501 10051 rpcz_store.cc:276] 0709 13:17:10.283153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.150333 (+1867180us) service_pool.cc:224] Handling call
0709 13:17:12.238193 (+ 87860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.238543 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.238588 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:12.238662 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:12.238711 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.238798 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.238994 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.247345 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 212732us
I20260709 13:17:12.153362 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2503) took 2601 ms. Trace:
I20260709 13:17:12.248660 10127 rpcz_store.cc:276] 0709 13:17:09.552313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.145295 (+2592982us) service_pool.cc:224] Handling call
0709 13:17:12.145665 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.152945 (+  7280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.153258 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.249939 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 506488us
I20260709 13:17:12.251253 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1107) took 2492 ms. Trace:
I20260709 13:17:12.251606 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:12.252951 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.166036 10049 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:12.256479 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.256721 10049 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.164315 10047 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:12.171977 10199 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:12.172067 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.087761 12154 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.172003 10048 rpcz_store.cc:276] 0709 13:17:10.774921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.976780 (+1201859us) service_pool.cc:224] Handling call
0709 13:17:12.171048 (+194268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.137195 12172 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:12.231756 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 104) took 2564 ms. Trace:
I20260709 13:17:12.236045 12175 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.154644 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.141880 12245 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:12.256906 10022 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.037s	sys 0.053s
I20260709 13:17:12.163780 12067 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.267258 10047 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 13:17:12.266683 10203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.011s
W20260709 13:17:12.278615 10203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.006s	sys 0.011s
I20260709 13:17:12.252759 10127 rpcz_store.cc:276] 0709 13:17:09.759157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.249974 (+2490817us) service_pool.cc:224] Handling call
0709 13:17:12.250552 (+   578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.250804 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.250848 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:12.250903 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:12.250946 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.251010 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.251177 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.266374 10170 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.012s
I20260709 13:17:12.268870 10123 rpcz_store.cc:276] 0709 13:17:09.667328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.147047 (+2479719us) service_pool.cc:224] Handling call
0709 13:17:12.231107 (+ 84060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.231373 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.231396 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:12.231453 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:12.231497 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.231566 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.231688 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.280202 10170 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.006s	sys 0.012s
W20260709 13:17:12.266883 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.002s	sys 0.014s
W20260709 13:17:12.285526 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.003s	sys 0.014s
I20260709 13:17:12.271140 10048 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:17:12.286720 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1089) took 2001 ms. Trace:
I20260709 13:17:12.287230 10048 rpcz_store.cc:276] 0709 13:17:10.285187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.270363 (+1985176us) service_pool.cc:224] Handling call
0709 13:17:12.270818 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.286217 (+ 15399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.286618 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.287990 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.271761 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.125576 12222 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:12.273556 10022 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.037s	sys 0.053s
I20260709 13:17:12.172721 12254 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:12.266196 10021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.000s	sys 0.016s
W20260709 13:17:12.296315 10021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.000s	sys 0.017s
W20260709 13:17:12.297161 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.278890 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1087) took 1994 ms. Trace:
I20260709 13:17:12.297987 10047 rpcz_store.cc:276] 0709 13:17:10.284625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.266262 (+1981637us) service_pool.cc:224] Handling call
0709 13:17:12.266894 (+   632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.278517 (+ 11623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.278818 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39680}
I20260709 13:17:12.236853 12151 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:12.281307 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109292us
I20260709 13:17:12.316001 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:12.255052 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 630677us
I20260709 13:17:12.174340 12019 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:12.290904 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 621758us
I20260709 13:17:12.321316 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.323072 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:12.329795 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:12.330036 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:12.335830 10051 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:12.269886 10196 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:12.303689 10123 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:17:12.281831 10019 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 3.334s (SENT)
W20260709 13:17:12.337785 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56011.2us
I20260709 13:17:12.337949 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1108) took 2573 ms. Trace:
I20260709 13:17:12.338657 10123 rpcz_store.cc:276] 0709 13:17:09.764892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.280530 (+2515638us) service_pool.cc:224] Handling call
0709 13:17:12.303097 (+ 22567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.337559 (+ 34462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.337865 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:12.307971 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56435.6us
I20260709 13:17:12.313524 10056 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:12.291616 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 599389us
W20260709 13:17:12.340873 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1104) took 1879 ms (client timeout 2054 ms). Trace:
W20260709 13:17:12.343358 10056 rpcz_store.cc:269] 0709 13:17:10.461958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.687937 (+1225979us) service_pool.cc:224] Handling call
0709 13:17:12.340781 (+652844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.344647 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.347919 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.297434 12172 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:12.304277 12245 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:12.316355 12165 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.326763 12214 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:12.335180 12177 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.366356 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:12.331406 11513 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.373277 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.390221 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.374763 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:12.362903 10049 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:12.392234 10049 pending_rounds.cc:85] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 2
I20260709 13:17:12.393133 10049 pending_rounds.cc:107] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:12.393491 10049 raft_consensus.cc:2889] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:12.395355 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1369) took 2503 ms. Trace:
I20260709 13:17:12.396874 10049 rpcz_store.cc:276] 0709 13:17:09.892025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.122531 (+2230506us) service_pool.cc:224] Handling call
0709 13:17:12.123901 (+  1370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.394856 (+270955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.395247 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.398217 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:12.399107 10049 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:12.325042 12162 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.358105 12185 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.366115 12165 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.410821 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 142) took 3119 ms. Trace:
I20260709 13:17:12.410962 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.360606 12213 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:12.365090 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:12.419564 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1092) took 3850 ms. Trace:
W20260709 13:17:12.383074 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.422000 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2945958249997602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:17:12.422428 10095 connection.cc:256] RPC call timeout handler was delayed by -2.1252908949998073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2887s for a response
W20260709 13:17:12.424320 10095 connection.cc:256] RPC call timeout handler was delayed by -2.023695824999777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 13:17:12.424708 10095 connection.cc:256] RPC call timeout handler was delayed by -1.9741958249999243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:17:12.425046 10095 connection.cc:256] RPC call timeout handler was delayed by -1.9426958249998734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 13:17:12.425885 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3612419710000268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:17:12.426297 10095 connection.cc:256] RPC call timeout handler was delayed by -1.010241970999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:17:12.342378 12266 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:17:12.383381 12265 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.471551 12265 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.471678 12245 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.474388 12265 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.475813 12211 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:12.476357 12211 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:12.477336 12211 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.478613 12211 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.448459 12177 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:12.479234 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1093) took 3775 ms. Trace:
I20260709 13:17:12.479899 10124 rpcz_store.cc:276] 0709 13:17:08.703490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.940547 (+3237057us) service_pool.cc:224] Handling call
0709 13:17:11.940907 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.243423 (+302516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.243472 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.244292 (+   820us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.368321 (+124029us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:12.368387 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.369083 (+   696us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.478808 (+109725us) raft_consensus.cc:1685] finished
0709 13:17:12.478839 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.479078 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58413,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":14864,"thread_start_us":407266,"threads_started":2,"wal-append.queue_time_us":232555}
I20260709 13:17:12.482479 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.318749 12254 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.385378 12271 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:12.449515 10864 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.000s (SENT)
I20260709 13:17:12.400969 12214 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:12.463336 12213 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:12.464535 12256 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:17:12.464874 12132 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.428216 12176 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.465348 12258 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:12.399977 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:12.427775 12185 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:12.421204 10130 rpcz_store.cc:276] 0709 13:17:08.569585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.023751 (+454166us) service_pool.cc:224] Handling call
0709 13:17:09.024166 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:10.280759 (+1256593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:10.280798 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:10.281384 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.816914 (+1535530us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:11.838307 (+ 21393us) log.cc:844] Serialized 14 byte log entry
0709 13:17:11.838614 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.838728 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:11.961316 (+122588us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b180042f2e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:12.419217 (+457901us) raft_consensus.cc:1685] finished
0709 13:17:12.419316 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.419480 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22074,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":233693,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":21821,"spinlock_wait_cycles":9927936,"thread_start_us":2657748,"threads_started":2,"wal-append.queue_time_us":1441323}
I20260709 13:17:12.483243 12268 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:12.484103 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57175.4us
I20260709 13:17:12.484794 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1091) took 3916 ms. Trace:
I20260709 13:17:12.485435 10124 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:12.487743 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:12.526046 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:12.493155 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 118692us
I20260709 13:17:12.479691 12177 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:12.522935 10122 rpcz_store.cc:276] 0709 13:17:08.568184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.028234 (+460050us) service_pool.cc:224] Handling call
0709 13:17:09.028860 (+   626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:10.303829 (+1274969us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:11.739682 (+1435853us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.296437 (+556755us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.306738 (+ 10301us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.307003 (+   265us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:12.307450 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.310006 (+  2556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.310142 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.484527 (+174385us) raft_consensus.cc:1685] finished
0709 13:17:12.484549 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.484720 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50358,"prepare.queue_time_us":532272,"prepare.run_cpu_time_us":5816,"prepare.run_wall_time_us":646968,"raft.queue_time_us":171586,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":9072,"spinlock_wait_cycles":39597696,"thread_start_us":3183218,"threads_started":3,"wal-append.queue_time_us":698058}
I20260709 13:17:12.530318 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.530598 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.531068 10130 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:12.531495 10122 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:12.532428 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.533067 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.420148 10053 rpcz_store.cc:276] 0709 13:17:09.291647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.291852 (+   205us) service_pool.cc:224] Handling call
0709 13:17:09.292378 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.292922 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:09.446799 (+153877us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b1800350140. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:17:10.292647 (+845848us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.239783 (+1947136us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.240554 (+   771us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.244364 (+  3810us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:12.305133 (+ 60769us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.388802 (+ 83669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.389073 (+   271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.410488 (+ 21415us) raft_consensus.cc:1685] finished
0709 13:17:12.410516 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.410728 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1920387,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":72,"raft.queue_time_us":331821,"raft.run_cpu_time_us":443,"raft.run_wall_time_us":41993,"spinlock_wait_cycles":3848576,"thread_start_us":2934055,"threads_started":5,"wal-append.queue_time_us":1587806}
I20260709 13:17:12.533679 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.534724 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.366920 12172 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.531342 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:12.536808 10124 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.423487 12165 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.540310 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:12.535789 10053 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:12.543450 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.538689 12172 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.423207 11513 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:17:12.544270 10053 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:12.528949 12177 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.545684 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.546905 10053 raft_consensus.cc:2410] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:12.507525 10049 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:12.495179 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:12.551414 12172 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:12.567610 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:12.578498 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 129040us
I20260709 13:17:12.560209 12258 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.559609 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.494587 12265 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:12.582651 12214 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.581751 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.520097 10897 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:12.552587 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:12.604063 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 51581.6us
I20260709 13:17:12.518601 12213 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:12.586845 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.588532 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:12.575317 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 247440us
I20260709 13:17:12.497133 12264 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:12.499598 12259 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.592756 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:12.496999 12036 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:17:12.610764 10049 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:12.612655 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:12.614483 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 296587us
I20260709 13:17:12.618594 12162 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:12.619051 12162 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:12.619130 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.620024 12162 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.621166 12162 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.523056 12272 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:12.584605 12154 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:12.624385 10898 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:17:12.610791 10053 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:12.580181 12270 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:12.578835 12224 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.585577 12172 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:12.581332 12185 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:12.707551 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 105) took 3040 ms. Trace:
W20260709 13:17:12.634052 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 286256us
I20260709 13:17:12.617180 12259 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.641686 10124 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:17:12.697317 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115638us
I20260709 13:17:12.710374 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.596096 12202 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.671710 10053 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:12.724736 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1095) took 2437 ms. Trace:
I20260709 13:17:12.603446 12276 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:12.676069 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.585071 12132 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:17:12.693050 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:12.726111 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.616554 12269 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:12.691676 12275 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
W20260709 13:17:12.726969 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.698566 12213 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.727656 12275 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:12.728406 12275 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.729737 12165 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:12.700002 12127 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.829s	user 0.010s	sys 0.005s
I20260709 13:17:12.700727 11702 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.701637 12036 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.523298 12176 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.603848 12265 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.703689 12172 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:12.637050 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.708174 10128 rpcz_store.cc:276] 0709 13:17:09.667585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.151298 (+2483713us) service_pool.cc:224] Handling call
0709 13:17:12.261017 (+109719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.261441 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.261474 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.273557 (+ 12083us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.608774 (+335217us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:12.608819 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.608908 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.707228 (+ 98320us) raft_consensus.cc:1685] finished
0709 13:17:12.707250 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.707459 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":247600,"threads_started":1,"wal-append.queue_time_us":367498}
W20260709 13:17:12.773578 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.713842 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.720043 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:12.775537 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 55627.7us
I20260709 13:17:12.723248 12282 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:12.711572 10130 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:12.725417 10053 rpcz_store.cc:276] 0709 13:17:10.286881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.634528 (+2347647us) service_pool.cc:224] Handling call
0709 13:17:12.634970 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.724354 (+ 89384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.724648 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:12.788715 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:12.790009 10053 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:12.735559 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 195407us
W20260709 13:17:12.743449 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.616605 12267 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:12.794054 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:12.780128 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 168631us
I20260709 13:17:12.753739 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.580575 12268 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.668090 12270 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:17:12.798434 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44863.3us
I20260709 13:17:12.798698 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1110) took 2458 ms. Trace:
I20260709 13:17:12.799981 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.785478 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:12.799775 10131 rpcz_store.cc:276] 0709 13:17:10.340829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.281718 (+1940889us) service_pool.cc:224] Handling call
0709 13:17:12.332282 (+ 50564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.335296 (+  3014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.335336 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.335925 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.627186 (+291261us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:12.627227 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.627298 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:12.798416 (+171118us) raft_consensus.cc:1685] finished
0709 13:17:12.798447 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:12.798620 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26842,"raft.queue_time_us":2643,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":13004,"thread_start_us":242328,"threads_started":1,"wal-append.queue_time_us":358960}
I20260709 13:17:12.801041 10053 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:12.809478 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:12.810475 10053 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:12.772317 11805 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.845s	user 0.013s	sys 0.005s
I20260709 13:17:12.812564 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:12.814728 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.623955 12264 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.751310 12132 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.765697 12036 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.761793 12276 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:12.839509 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54177.7us
I20260709 13:17:12.718379 12259 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.787310 12177 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.762290 12271 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:12.800499 12174 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:12.822928 10864 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.412s (SENT)
I20260709 13:17:12.812467 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:12.878216 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66020.5us
W20260709 13:17:12.877947 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55080.5us
I20260709 13:17:12.816676 10053 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:12.814988 12285 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.799367 12275 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.815807 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:12.869814 12213 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:12.797904 12127 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:17:12.869971 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:12.871922 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:12.798450 12202 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:12.879467 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:12.813513 12270 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:12.837330 12267 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:12.837347 10894 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:17:12.839355 12036 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.814436 12201 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:12.890702 12067 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.001s
I20260709 13:17:12.668228 12272 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.894958 10131 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:12.839239 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:12.836987 11805 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.898847 12202 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:12.922014 12261 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 13:17:12.859360 12132 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.888875 12174 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:12.939168 12132 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.939335 12018 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:12.941650 11805 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:12.942677 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:12.943677 10202 raft_consensus.cc:2368] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:12.944924 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:12.945783 10202 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:12.927696 12257 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 13:17:12.948099 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 117) took 3182 ms (client timeout 1662 ms). Trace:
I20260709 13:17:12.948238 12176 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:12.948457 12278 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:17:12.948738 12294 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:12.949028 12268 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:12.922881 12160 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.008s
I20260709 13:17:12.949671 12280 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:12.950095 12067 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:12.910429 12292 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:12.991768 12292 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.994606 12292 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.969236 12271 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.996132 12269 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:17:12.997766 12290 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:17:12.976547 12145 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.708s	user 0.014s	sys 0.000s
I20260709 13:17:12.892140 12185 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.987488 12067 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:12.989562 12280 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.917996 12201 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:12.879784 12177 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.926322 12296 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:12.950467 12298 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:12.946871 12258 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:12.980546 12272 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:12.910706 12213 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:12.893424 12285 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:12.878875 12276 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:12.969164 12294 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:12.985184 12212 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.004688 12254 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.031060 12298 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:13.020643 12259 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.040673 12264 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.034307 12265 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:13.041077 12298 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.037628 12294 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:13.034946 12296 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.059329 12285 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.059837 12296 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:13.107328 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 329546us
I20260709 13:17:13.108916 10056 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:17:13.109750 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1091) took 2823 ms. Trace:
I20260709 13:17:13.111079 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1092) took 2824 ms. Trace:
I20260709 13:17:13.110713 10048 rpcz_store.cc:276] 0709 13:17:10.285768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.288360 (+2002592us) service_pool.cc:224] Handling call
0709 13:17:12.288793 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.289125 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:12.628438 (+339313us) log.cc:844] Serialized 14 byte log entry
0709 13:17:13.108202 (+479764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.108450 (+   248us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:13.108645 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.108851 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.109362 (+   511us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":271225,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":776402,"threads_started":2,"wal-append.queue_time_us":427151}
I20260709 13:17:13.112147 10047 rpcz_store.cc:276] 0709 13:17:10.286036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.299324 (+2013288us) service_pool.cc:224] Handling call
0709 13:17:12.328585 (+ 29261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.328912 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:12.328949 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.329586 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.507415 (+177829us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:12.507469 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.507541 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.110219 (+602678us) raft_consensus.cc:1685] finished
0709 13:17:13.110409 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.110808 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":138100,"threads_started":1,"wal-append.queue_time_us":402144}
I20260709 13:17:12.880791 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.116312 10053 raft_consensus.cc:2376] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6. Re-sending same reply.
I20260709 13:17:13.119378 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.121213 10053 raft_consensus.cc:2376] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5. Re-sending same reply.
W20260709 13:17:13.125151 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:12.947474 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:13.012270 12019 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.004s
I20260709 13:17:13.145629 11702 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:13.146140 12021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.126s	user 0.008s	sys 0.008s
I20260709 13:17:12.935880 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.132527 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 119) took 3365 ms. Trace:
I20260709 13:17:13.148733 10131 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:12.962150 10201 rpcz_store.cc:269] 0709 13:17:09.765953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.766068 (+   115us) service_pool.cc:224] Handling call
0709 13:17:12.947804 (+3181736us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:12.948003 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4445175168}
I20260709 13:17:13.130147 10200 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:12.960638 10205 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2.
I20260709 13:17:12.983718 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:13.145663 12019 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.155493 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:13.155927 10123 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:13.148552 10203 rpcz_store.cc:276] 0709 13:17:09.766670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.766861 (+   191us) service_pool.cc:224] Handling call
0709 13:17:09.793155 (+ 26294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.809741 (+ 16586us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:11.669404 (+1859663us) log.cc:844] Serialized 14 byte log entry
0709 13:17:12.296638 (+627234us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.297047 (+   409us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.297222 (+   175us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:12.297251 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.297340 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.131729 (+834389us) raft_consensus.cc:1685] finished
0709 13:17:13.131784 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.132230 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84643,"prepare.queue_time_us":446318,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":47656,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7337,"spinlock_wait_cycles":22038784,"thread_start_us":2413681,"threads_started":2,"wal-append.queue_time_us":735210}
I20260709 13:17:13.156927 10201 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:13.157991 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1374) took 1038 ms. Trace:
I20260709 13:17:13.158628 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:13.159062 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.159665 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.130005 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1095) took 4425 ms. Trace:
W20260709 13:17:13.131319 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.162076 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.111181 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:13.131412 10199 raft_consensus.cc:2368] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 3. Current term is 4.
I20260709 13:17:13.131393 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:13.163940 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 54088.2us
I20260709 13:17:13.161898 10126 rpcz_store.cc:276] 0709 13:17:08.704014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.954017 (+3250003us) service_pool.cc:224] Handling call
0709 13:17:11.954664 (+   647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:11.979016 (+ 24352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:11.979067 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.014547 (+ 35480us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.152977 (+138430us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:12.153043 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.449028 (+295985us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b54002eccd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:12.453258 (+  4230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.128679 (+675421us) raft_consensus.cc:1685] finished
0709 13:17:13.128753 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.129552 (+   799us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44461,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":32221,"spinlock_wait_cycles":54569216,"thread_start_us":90374,"threads_started":1,"wal-append.queue_time_us":555784}
W20260709 13:17:13.164404 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.165839 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:13.146904 12021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.008s	sys 0.008s
I20260709 13:17:13.161785 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:13.145422 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.001304 12165 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.160044 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1366) took 1042 ms. Trace:
I20260709 13:17:13.158476 10201 rpcz_store.cc:276] 0709 13:17:12.119196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.154738 (+1035542us) service_pool.cc:224] Handling call
0709 13:17:13.157879 (+  3141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.131492 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.005497 12145 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:13.179319 10203 rpcz_store.cc:276] 0709 13:17:12.117552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.158007 (+1040455us) service_pool.cc:224] Handling call
0709 13:17:13.159956 (+  1949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.167210 12019 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.166889 10126 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:17:13.186537 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.195927 12301 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:13.175931 11702 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:13.214283 11702 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.195894 12201 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.216598 12301 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:13.217471 12201 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:12.986816 12160 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.014972 12067 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.201807 12259 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:13.212643 10864 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.853s (SENT)
I20260709 13:17:13.216028 12145 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:13.221344 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:13.236145 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1370) took 1117 ms. Trace:
I20260709 13:17:13.224835 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1083) took 2941 ms. Trace:
I20260709 13:17:13.238711 10051 rpcz_store.cc:276] 0709 13:17:10.283449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.247084 (+1963635us) service_pool.cc:224] Handling call
0709 13:17:12.266925 (+ 19841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.336609 (+ 69684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.214888 (+878279us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:13.224535 (+  9647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157776128}
I20260709 13:17:13.232152 12145 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.219082 12301 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.220659 10896 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
W20260709 13:17:13.289273 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.295169 10053 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:13.309815 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.242992 10201 rpcz_store.cc:276] 0709 13:17:12.118364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.215087 (+1096723us) service_pool.cc:224] Handling call
0709 13:17:13.236007 (+ 20920us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:13.389776 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.327819 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.318446 10053 raft_consensus.cc:2376] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4. Re-sending same reply.
W20260709 13:17:13.318277 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 458019us
I20260709 13:17:13.392369 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.323729 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.404100 12145 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.415788 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:13.392117 12213 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:13.425097 12213 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:13.418366 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:13.425868 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.431098 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:13.432487 10196 raft_consensus.cc:2376] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3. Re-sending same reply.
I20260709 13:17:13.433208 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 131) took 1192 ms. Trace:
I20260709 13:17:13.433671 10196 rpcz_store.cc:276] 0709 13:17:12.240822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.430450 (+1189628us) service_pool.cc:224] Handling call
0709 13:17:13.433111 (+  2661us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:13.436025 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.418557 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:13.442507 12172 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.010s
I20260709 13:17:13.443324 12172 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.444097 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:13.445200 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1369) took 1327 ms. Trace:
I20260709 13:17:13.445549 10198 rpcz_store.cc:276] 0709 13:17:12.118119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.443457 (+1325338us) service_pool.cc:224] Handling call
0709 13:17:13.445077 (+  1620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.446731 10054 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:13.450039 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1355) took 1345 ms. Trace:
I20260709 13:17:13.446921 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:13.440408 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:13.450502 10204 rpcz_store.cc:276] 0709 13:17:12.104672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.448364 (+1343692us) service_pool.cc:224] Handling call
0709 13:17:13.448991 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.449476 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.449511 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.449578 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:13.449621 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.449680 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.449795 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.442428 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.451426 12224 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.465866 10204 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:13.462702 12224 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:13.458451 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2493) took 3148 ms (client timeout 1513 ms). Trace:
I20260709 13:17:13.472404 10130 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:13.471515 10201 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:13.467319 10197 rpcz_store.cc:269] 0709 13:17:10.287825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:10.288608 (+   783us) service_pool.cc:224] Handling call
0709 13:17:13.435649 (+3147041us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54001fed54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.435882 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2935034240}
I20260709 13:17:13.456579 12172 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:13.476687 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.477717 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1356) took 1369 ms. Trace:
I20260709 13:17:13.478593 10201 rpcz_store.cc:276] 0709 13:17:12.104975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.405228 (+1300253us) service_pool.cc:224] Handling call
0709 13:17:13.474488 (+ 69260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:13.479660 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.472317 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1361) took 1350 ms. Trace:
I20260709 13:17:13.480419 10204 rpcz_store.cc:276] 0709 13:17:12.116311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.456367 (+1340056us) service_pool.cc:224] Handling call
0709 13:17:13.465183 (+  8816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.466557 (+  1374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.466965 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.477718 10051 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3. Re-sending same reply.
I20260709 13:17:13.483832 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:13.489746 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 277464us
I20260709 13:17:13.495216 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.495352 10128 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:13.452414 12294 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:13.503543 10049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.003s	sys 0.026s
I20260709 13:17:13.507951 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:13.510138 10049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.004s	sys 0.026s
I20260709 13:17:13.510965 12177 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.511844 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1094) took 3225 ms. Trace:
I20260709 13:17:13.512573 10130 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:13.513897 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 126) took 1380 ms. Trace:
W20260709 13:17:13.514837 12270 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.617s	user 0.019s	sys 0.010s
I20260709 13:17:13.515195 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.513263 10049 rpcz_store.cc:276] 0709 13:17:10.286606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.621883 (+2335277us) service_pool.cc:224] Handling call
0709 13:17:12.622322 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:12.622623 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:12.626105 (+  3482us) log.cc:844] Serialized 14 byte log entry
0709 13:17:13.511174 (+885069us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.511329 (+   155us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:13.511450 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.511544 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.511760 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":455,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":125391,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":18268,"thread_start_us":883861,"threads_started":2,"wal-append.queue_time_us":357045}
I20260709 13:17:13.514405 10201 rpcz_store.cc:276] 0709 13:17:12.133447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.479690 (+1346243us) service_pool.cc:224] Handling call
0709 13:17:13.512072 (+ 32382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.513367 (+  1295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.513410 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.513497 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:13.513532 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.513621 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.513786 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":221,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":208}
W20260709 13:17:13.517477 12202 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.005s
I20260709 13:17:13.519536 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.502635 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:13.517714 12151 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.186s	user 0.012s	sys 0.016s
I20260709 13:17:13.521497 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.517928 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 130) took 1277 ms. Trace:
I20260709 13:17:13.525300 10126 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:13.523031 10204 rpcz_store.cc:276] 0709 13:17:12.240474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.481130 (+1240656us) service_pool.cc:224] Handling call
0709 13:17:13.516134 (+ 35004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.517523 (+  1389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.517549 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.517613 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:13.517643 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.517732 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.517858 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":258,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":202}
W20260709 13:17:13.534240 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58819.4us
I20260709 13:17:13.535939 11849 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:13.536664 11849 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:13.537247 11849 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.525817 12270 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.543295 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:13.545607 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:13.546680 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:13.548916 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.550349 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 133) took 1308 ms. Trace:
I20260709 13:17:13.552227 12222 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:13.545490 12036 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.596s	user 0.021s	sys 0.003s
I20260709 13:17:13.556895 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.558822 10201 rpcz_store.cc:276] 0709 13:17:12.241696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.522679 (+1280983us) service_pool.cc:224] Handling call
0709 13:17:13.543694 (+ 21015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.548995 (+  5301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.549061 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.549225 (+   164us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:13.549296 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.549415 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.550043 (+   628us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":415,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":178}
I20260709 13:17:13.538846 12221 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.561076 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.564551 10051 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.555409 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:13.563103 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.566444 10126 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:13.544721 11702 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:13.569417 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.570274 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.530295 12151 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.525192 12202 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.565769 10895 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.583525 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:13.588775 10891 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:13.589401 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.589604 10128 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:13.594533 12211 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.046s	user 0.014s	sys 0.013s
I20260709 13:17:13.592362 10049 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.568861 12036 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.601928 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.601744 10048 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:13.591678 11702 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:13.619593 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.614048 10131 raft_consensus.cc:2368] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4.
I20260709 13:17:13.616094 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:13.615525 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.625878 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.629796 10130 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:13.615047 10897 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:13.630638 12162 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.009s	user 0.028s	sys 0.000s
I20260709 13:17:13.632256 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 129) took 1391 ms. Trace:
I20260709 13:17:13.632395 12211 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:13.632949 10203 rpcz_store.cc:276] 0709 13:17:12.239903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.615696 (+1375793us) service_pool.cc:224] Handling call
0709 13:17:13.631213 (+ 15517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.608449 12201 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.635738 12201 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.637669 12294 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.640046 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.644256 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:13.647905 12268 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:13.648519 12268 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:13.648414 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.649227 12268 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.650432 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.651309 12151 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.651585 10126 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:13.652212 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2531) took 1043 ms. Trace:
I20260709 13:17:13.652591 10126 rpcz_store.cc:276] 0709 13:17:12.608573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.571497 (+962924us) service_pool.cc:224] Handling call
0709 13:17:13.652143 (+ 80646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.653980 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.609535 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 150) took 1252 ms. Trace:
I20260709 13:17:13.645547 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 148) took 1309 ms. Trace:
I20260709 13:17:13.651388 10130 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:13.662920 10051 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:13.659029 10196 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:13.619822 12292 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.569s	user 0.015s	sys 0.021s
I20260709 13:17:13.660243 10896 rpcz_store.cc:276] 0709 13:17:12.356684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.357028 (+   344us) service_pool.cc:224] Handling call
0709 13:17:13.609215 (+1252187us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540063ef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.609440 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920761216}
I20260709 13:17:13.669845 10049 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:13.662236 10900 rpcz_store.cc:276] 0709 13:17:12.336396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.336608 (+   212us) service_pool.cc:224] Handling call
0709 13:17:13.643781 (+1307173us) spinlock_profiling.cc:243] Waited 971 us on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:13.645457 (+  1676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233600}
I20260709 13:17:13.613617 12036 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.674192 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.613997 12298 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:13.659227 12201 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.659250 12294 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.683302 12019 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.684180 12019 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.684944 10895 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:13.682264 10892 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:13.687650 12294 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.688683 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 149) took 1332 ms. Trace:
I20260709 13:17:13.686950 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:13.692744 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 137862us
I20260709 13:17:13.693481 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.658988 11849 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:13.696995 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.697414 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.694377 10202 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:13.696206 10048 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:13.702152 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:13.698467 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:13.686925 12019 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:13.712580 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1373) took 3745 ms. Trace:
I20260709 13:17:13.712641 12162 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:13.714269 12162 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:13.714432 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2506) took 4161 ms (client timeout 3334 ms). Trace:
W20260709 13:17:13.717015 10125 rpcz_store.cc:269] 0709 13:17:09.553289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.745002 (+2191713us) service_pool.cc:224] Handling call
0709 13:17:13.713856 (+1968854us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.714189 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708850432}
I20260709 13:17:13.718257 10129 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:13.706339 12214 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:13.707862 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1106) took 3245 ms (client timeout 3204 ms). Trace:
I20260709 13:17:13.716070 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.713156 10050 rpcz_store.cc:276] 0709 13:17:09.967633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.146691 (+2179058us) service_pool.cc:224] Handling call
0709 13:17:12.267734 (+121043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:12.268058 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:12.268086 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:12.295930 (+ 27844us) log.cc:844] Serialized 26 byte log entry
0709 13:17:12.623452 (+327522us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:12.624700 (+  1248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:12.624786 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.114692 (+489906us) raft_consensus.cc:1685] finished
0709 13:17:13.114759 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.712027 (+597268us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400199950. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.712368 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237442432,"thread_start_us":267019,"threads_started":1,"wal-append.queue_time_us":637796}
I20260709 13:17:13.724232 12294 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.720533 10127 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:13.715090 10128 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:13.704607 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.722301 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 147) took 1388 ms. Trace:
I20260709 13:17:13.718757 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.732900 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2514) took 1206 ms. Trace:
I20260709 13:17:13.733393 10127 rpcz_store.cc:276] 0709 13:17:12.526108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.715512 (+1189404us) service_pool.cc:224] Handling call
0709 13:17:13.732818 (+ 17306us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:13.719758 10055 rpcz_store.cc:269] 0709 13:17:10.484715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.942815 (+1458100us) service_pool.cc:224] Handling call
0709 13:17:13.707377 (+1764562us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54002369d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.707584 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84135680}
I20260709 13:17:13.739061 10125 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.739621 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.740352 10126 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:13.740952 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2536) took 1131 ms. Trace:
I20260709 13:17:13.741186 10126 rpcz_store.cc:276] 0709 13:17:12.609936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.676556 (+1066620us) service_pool.cc:224] Handling call
0709 13:17:13.740874 (+ 64318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.676334 12292 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.744318 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:13.744678 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.721428 12160 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.747637 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.721827 12280 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:13.749536 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.751684 10130 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:13.751556 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.755616 10053 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:13.754801 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:13.758895 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.689596 10892 rpcz_store.cc:276] 0709 13:17:12.356146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.361668 (+  5522us) service_pool.cc:224] Handling call
0709 13:17:13.661878 (+1300210us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:13.679872 (+ 17994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:13.688135 (+  8263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.688469 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25023232}
I20260709 13:17:13.757098 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2519) took 1226 ms. Trace:
I20260709 13:17:13.761480 10130 rpcz_store.cc:276] 0709 13:17:12.530890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.696740 (+1165850us) service_pool.cc:224] Handling call
0709 13:17:13.756978 (+ 60238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.763667 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.766405 10047 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:13.734218 11702 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:17:13.770318 10051 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:13.772085 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.772986 10051 raft_consensus.cc:2376] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5. Re-sending same reply.
I20260709 13:17:13.774300 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.779398 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:13.780611 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45576us
I20260709 13:17:13.780911 12271 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:13.781695 12271 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:13.783035 12271 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.754709 11849 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.786368 12292 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:13.788571 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44849.9us
I20260709 13:17:13.787477 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.794312 10055 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:13.795248 10128 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.718740 12162 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.796681 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:13.759981 12270 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.798377 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.798185 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.798923 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.802100 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:13.802974 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 101321us
I20260709 13:17:13.807341 10049 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:13.806384 12067 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.540s	user 0.027s	sys 0.009s
I20260709 13:17:13.809717 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.809551 12067 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.811400 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.812335 10197 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:13.732669 10891 rpcz_store.cc:276] 0709 13:17:12.333711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.356789 (+ 23078us) service_pool.cc:224] Handling call
0709 13:17:12.983556 (+626767us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:13.722037 (+738481us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b540050f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.722234 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990531328}
I20260709 13:17:13.813802 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.817700 10049 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:13.819199 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.819458 10125 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:13.819937 10055 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.820959 10053 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:13.822190 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 127) took 1676 ms (client timeout 2042 ms). Trace:
W20260709 13:17:13.822749 10202 rpcz_store.cc:269] 0709 13:17:12.145564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.962615 (+817051us) service_pool.cc:224] Handling call
0709 13:17:13.821904 (+859289us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54001ef5d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.822088 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991746304}
I20260709 13:17:13.818823 10050 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:13.818104 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:13.844368 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.841892 12019 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:13.844022 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:13.863221 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:13.863663 10095 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:13.850672 10127 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.864735 12294 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:13.874629 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1102) took 4346 ms. Trace:
I20260709 13:17:13.866076 12177 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:17:13.863750 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.876256 12177 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:13.876771 12177 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:13.878391 12318 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:17:13.866314 12268 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.883338 12177 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:13.885329 12177 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:13.886802 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1086) took 3602 ms (client timeout 2309 ms). Trace:
W20260709 13:17:13.887348 10054 rpcz_store.cc:269] 0709 13:17:10.284352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:11.747081 (+1462729us) service_pool.cc:224] Handling call
0709 13:17:13.886438 (+2139357us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400086954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.886677 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713330944}
I20260709 13:17:13.888831 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:13.889665 10054 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:13.890590 10051 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:13.865607 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.873534 10055 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:13.875207 10129 rpcz_store.cc:276] 0709 13:17:09.528510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.102566 (+2574056us) service_pool.cc:224] Handling call
0709 13:17:13.717831 (+1615265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.718811 (+   980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.874367 (+155556us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400126bd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.874546 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3713325440}
I20260709 13:17:13.871531 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1362) took 1754 ms. Trace:
I20260709 13:17:13.873801 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.898476 10197 rpcz_store.cc:276] 0709 13:17:12.116546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.476339 (+1359793us) service_pool.cc:224] Handling call
0709 13:17:13.503386 (+ 27047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.812852 (+309466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.871010 (+ 58158us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540035a2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:13.871393 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709790976}
I20260709 13:17:13.866830 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.876900 10049 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:13.863992 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 106) took 3501 ms. Trace:
I20260709 13:17:13.899549 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1134) took 1089 ms. Trace:
I20260709 13:17:13.899892 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.900159 10051 rpcz_store.cc:276] 0709 13:17:12.806483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.773882 (+967399us) service_pool.cc:224] Handling call
0709 13:17:13.895574 (+121692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.864140 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.904762 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.868674 12173 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.905752 10128 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:13.864943 12036 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:13.906821 12036 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.909246 12224 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.909626 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.912285 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:13.913617 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:13.916088 10129 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:13.915948 12067 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.916604 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1111) took 3575 ms. Trace:
I20260709 13:17:13.919198 10124 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:13.919591 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 79962.7us
I20260709 13:17:13.920207 10128 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.886356 12268 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.872311 12019 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.923710 12019 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:13.831077 12185 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.165s
I20260709 13:17:13.923735 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1113) took 3375 ms. Trace:
I20260709 13:17:13.921109 12294 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:13.921640 10053 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:13.928802 12294 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.929291 12294 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:13.930344 10053 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:13.931331 10198 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:13.900051 10130 rpcz_store.cc:276] 0709 13:17:10.360686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.763712 (+3403026us) service_pool.cc:224] Handling call
0709 13:17:13.802559 (+ 38847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.843307 (+ 40748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.843355 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.862008 (+ 18653us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:13.862063 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.862154 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.862321 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14548,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10204}
I20260709 13:17:13.927850 12268 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:13.934492 12268 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:13.935379 10130 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:13.936744 10125 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.936463 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1119) took 3217 ms. Trace:
I20260709 13:17:13.936946 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2512) took 1411 ms. Trace:
I20260709 13:17:13.937470 10122 rpcz_store.cc:276] 0709 13:17:12.525533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.532742 (+  7209us) service_pool.cc:224] Handling call
0709 13:17:13.936551 (+1403809us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.936761 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601624832}
I20260709 13:17:13.939280 10128 rpcz_store.cc:276] 0709 13:17:10.548778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.916738 (+3367960us) service_pool.cc:224] Handling call
0709 13:17:13.919792 (+  3054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.923336 (+  3544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.923644 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.940253 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 111) took 2216 ms. Trace:
I20260709 13:17:13.864257 11849 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:13.916905 10020 connection.cc:256] RPC call timeout handler was delayed by -2.4392839520000962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 13:17:13.941129 11849 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:13.941732 10020 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.572s (SENT)
I20260709 13:17:13.921893 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:13.920435 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.942698 11849 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.943176 10128 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:13.944571  9953 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 114) took 9828 ms. Trace:
I20260709 13:17:13.942585 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1096) took 3655 ms. Trace:
I20260709 13:17:13.956019 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1098) took 3611 ms. Trace:
I20260709 13:17:13.955138 12185 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:13.953158 12294 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.956954 12185 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.957621 10055 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:13.947028 12018 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:13.958757 12018 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.958833 12294 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.959297 12018 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:13.959736 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1101) took 3571 ms. Trace:
I20260709 13:17:13.960224 10055 rpcz_store.cc:276] 0709 13:17:10.388100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.958540 (+3570440us) service_pool.cc:224] Handling call
0709 13:17:13.959003 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.959273 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.959303 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.959376 (+    73us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:13.959461 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.959525 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.959662 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.955356 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2517) took 1428 ms. Trace:
I20260709 13:17:13.909246 12036 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.963752 10128 rpcz_store.cc:276] 0709 13:17:12.526950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.940473 (+1413523us) service_pool.cc:224] Handling call
0709 13:17:13.940960 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.954938 (+ 13978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.955251 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.954394 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.942539 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:13.946750 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:13.939391 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72660.5us
I20260709 13:17:13.968750 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2515) took 1442 ms. Trace:
I20260709 13:17:13.969393 10131 rpcz_store.cc:276] 0709 13:17:12.526392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.150258 (+623866us) service_pool.cc:224] Handling call
0709 13:17:13.968458 (+818200us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:13.968653 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036378880}
I20260709 13:17:13.970271 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:13.948508 12243 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.941s	user 0.007s	sys 0.005s
I20260709 13:17:13.971680 11702 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.949448 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:13.919255 10123 rpcz_store.cc:276] 0709 13:17:10.341180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.340024 (+1998844us) service_pool.cc:224] Handling call
0709 13:17:12.365161 (+ 25137us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.156675 (+791514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.916244 (+759569us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b540023afd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:13.916535 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817382656}
I20260709 13:17:13.957558 12185 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:13.866771 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:13.956598 10049 rpcz_store.cc:276] 0709 13:17:10.344989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.904953 (+3559964us) service_pool.cc:224] Handling call
0709 13:17:13.905481 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.955579 (+ 50098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.955612 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.955688 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:13.955731 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.955813 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.955939 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.983238 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:13.973524 10122 rpcz_store.cc:276] 0709 13:17:11.724252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.938676 (+2214424us) service_pool.cc:224] Handling call
0709 13:17:13.939287 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.939814 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.939844 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.939895 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:13.939948 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.940027 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.940174 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3684,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":170}
I20260709 13:17:13.974985 10130 rpcz_store.cc:276] 0709 13:17:10.719087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.934575 (+3215488us) service_pool.cc:224] Handling call
0709 13:17:13.935084 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.936104 (+  1020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.936362 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.956147 10051 rpcz_store.cc:276] 0709 13:17:10.343845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.916999 (+3573154us) service_pool.cc:224] Handling call
0709 13:17:13.941651 (+ 24652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.942064 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.942107 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:13.942156 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:13.942248 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.942322 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.942478 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:13.931344 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1099) took 3586 ms. Trace:
I20260709 13:17:13.978485 10125 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:13.944357 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.981046 12036 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:13.981377 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1361) took 4166 ms. Trace:
I20260709 13:17:13.991067 10052 rpcz_store.cc:276] 0709 13:17:09.814774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:09.817793 (+  3019us) service_pool.cc:224] Handling call
0709 13:17:09.818314 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:09.830380 (+ 12066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:09.830427 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:09.892074 (+ 61647us) log.cc:844] Serialized 26 byte log entry
0709 13:17:11.973705 (+2081631us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:11.973754 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:11.973847 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:13.947487 (+1973640us) spinlock_profiling.cc:243] Waited 8.11 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:13.972372 (+ 24885us) raft_consensus.cc:1685] finished
0709 13:17:13.972400 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:13.981072 (+  8672us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5894,"raft.queue_time_us":11708,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":18654080,"thread_start_us":1670034,"threads_started":1,"wal-append.queue_time_us":1980063}
I20260709 13:17:13.981741 12224 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:13.986213 10128 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:17:13.972420 11702 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:13.995342 12067 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:13.995659 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54010.3us
I20260709 13:17:13.995972 12067 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:13.996438 12067 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:13.996471 10020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.577s (SENT) [suppressed 80 similar messages]
I20260709 13:17:13.997382 11702 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:13.982652 12202 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.988175 12173 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:14.000475 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1380) took 1661 ms (client timeout 2040 ms). Trace:
W20260709 13:17:14.001124 10056 rpcz_store.cc:269] 0709 13:17:12.339528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.344197 (+  4669us) service_pool.cc:224] Handling call
0709 13:17:14.000142 (+1655945us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b540010a9d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.000388 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751046400}
W20260709 13:17:14.003935 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 176334us
I20260709 13:17:14.004355 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 115) took 1107 ms. Trace:
W20260709 13:17:13.997572 10020 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.694s (SENT)
W20260709 13:17:14.005959 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.005743 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:13.988337 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:13.989002 12106 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:14.012686 10127 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:13.985544 10053 rpcz_store.cc:276] 0709 13:17:10.345291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.929384 (+3584093us) service_pool.cc:224] Handling call
0709 13:17:13.929990 (+   606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.930900 (+   910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:13.931235 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.005180 10123 rpcz_store.cc:276] 0709 13:17:12.897255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.992805 (+1095550us) service_pool.cc:224] Handling call
0709 13:17:13.996786 (+  3981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.003891 (+  7105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.003938 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.004010 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:14.004047 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.004122 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.004279 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.016364 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:14.019810 12212 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.019610 12106 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.020630 12212 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:14.023181 10865 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.548s (SENT)
I20260709 13:17:14.022791 12106 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.023810 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.027010 12298 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:14.027568 12270 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.028620 12298 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:14.027385 11702 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:14.029659 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46640.5us
I20260709 13:17:14.030185 11513 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:14.029861 12021 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:14.030750 11513 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.031361 12036 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:14.032065 12021 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:14.032382 12294 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:14.032648 12021 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.033336 12175 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:14.033999 12175 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:14.033890 12021 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.029844 12298 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:14.036243 10054 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:14.020311 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 153764us
I20260709 13:17:14.044185 10125 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:17:14.045432 10124 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:14.026990 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1109) took 2892 ms. Trace:
I20260709 13:17:14.048707 10052 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:14.050019 10056 rpcz_store.cc:276] 0709 13:17:11.135760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.025626 (+2889866us) service_pool.cc:224] Handling call
0709 13:17:14.026162 (+   536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.026545 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.026577 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.026631 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:14.026674 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.026737 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.026878 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.048658 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.053346 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.054163 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2525) took 1515 ms. Trace:
I20260709 13:17:14.055684 10125 rpcz_store.cc:276] 0709 13:17:12.532804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.012466 (+1479662us) service_pool.cc:224] Handling call
0709 13:17:14.035671 (+ 23205us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:14.047463 (+ 11792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.047923 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.057229 10049 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:13.935981 12268 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.065585 10124 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:14.062947 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1377) took 3642 ms. Trace:
I20260709 13:17:14.064291 10125 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:14.064823 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.067593 10049 rpcz_store.cc:276] 0709 13:17:10.415294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.012716 (+3597422us) service_pool.cc:224] Handling call
0709 13:17:14.057048 (+ 44332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.057701 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.057880 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.069124 10125 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:14.073419 10122 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 2. Current term is 4. Ops: []
I20260709 13:17:14.074218 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2524) took 1541 ms. Trace:
I20260709 13:17:14.074594 10122 rpcz_store.cc:276] 0709 13:17:12.532496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.013252 (+1480756us) service_pool.cc:224] Handling call
0709 13:17:14.073186 (+ 59934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.073925 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.074154 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.046607 12298 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.077329 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1378) took 3661 ms. Trace:
I20260709 13:17:14.077875 10051 rpcz_store.cc:276] 0709 13:17:10.415614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.996926 (+3581312us) service_pool.cc:224] Handling call
0709 13:17:14.061786 (+ 64860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.076908 (+ 15122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.076939 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.076998 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:14.077039 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.077109 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.077259 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.082212  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:14.065670 12106 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:14.060501 12175 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.084847 11702 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:14.085609 11702 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.032145 11513 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.088372 11513 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:14.088442 12298 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.089998 12021 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:14.090312 12298 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:14.089237 12211 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:14.091988 12211 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.092108 10054 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:14.093011 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1110) took 2854 ms. Trace:
I20260709 13:17:14.086736 11702 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.051604 12212 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:14.093746 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.093313 12211 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.096232 12268 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:14.100454 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 102904us
I20260709 13:17:14.101650 10127 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:14.102695 10048 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:14.104099 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1119) took 2411 ms. Trace:
W20260709 13:17:14.104537 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.072517 12270 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:14.101302 12185 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.105724 12270 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:14.104555 10048 rpcz_store.cc:276] 0709 13:17:11.692705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.101953 (+2409248us) service_pool.cc:224] Handling call
0709 13:17:14.102455 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.103715 (+  1260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.104003 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.109148 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1120) took 2416 ms. Trace:
I20260709 13:17:14.117616 10048 rpcz_store.cc:276] 0709 13:17:11.693061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.107901 (+2414840us) service_pool.cc:224] Handling call
0709 13:17:14.108371 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.108722 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.108762 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.108842 (+    80us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.108885 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.108942 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.109058 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.106998 12270 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.109490 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1113) took 2439 ms. Trace:
I20260709 13:17:14.093379 10054 rpcz_store.cc:276] 0709 13:17:11.239248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.062080 (+2822832us) service_pool.cc:224] Handling call
0709 13:17:14.091648 (+ 29568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.092623 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.092936 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.120600 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1112) took 2577 ms. Trace:
I20260709 13:17:14.120884 10049 rpcz_store.cc:276] 0709 13:17:11.669661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.108198 (+2438537us) service_pool.cc:224] Handling call
0709 13:17:14.108647 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.109123 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:14.109154 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.109199 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:14.109234 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.109296 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.109419 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":97}
I20260709 13:17:14.121085 10053 rpcz_store.cc:276] 0709 13:17:11.530338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.072271 (+2541933us) service_pool.cc:224] Handling call
0709 13:17:14.106648 (+ 34377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.106912 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.106939 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.107005 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:14.107035 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.107087 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.107234 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.085638 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:14.106155 12276 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:13.955090  9953 rpcz_store.cc:276] 0709 13:17:04.115761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.132395 (+ 16634us) service_pool.cc:224] Handling call
0709 13:17:13.944279 (+9811884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:12.026228 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:12.036324 (+ 10096us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:12.036367 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:17:12.036388 (+    21us) tablet.cc:662] Decoding operations
0709 13:17:12.079689 (+ 43301us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:12.079804 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:12.079837 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:17:12.080234 (+   397us) tablet.cc:701] Row locks acquired
0709 13:17:12.080258 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:17:12.080438 (+   180us) write_op.cc:270] Start()
0709 13:17:12.080599 (+   161us) write_op.cc:276] Timestamp: P: 1783603032080383 usec, L: 0
0709 13:17:12.080654 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:17:12.269007 (+188353us) log.cc:844] Serialized 913 byte log entry
0709 13:17:13.041497 (+772490us) op_driver.cc:464] REPLICATION: finished
0709 13:17:13.069430 (+ 27933us) write_op.cc:301] APPLY: starting
0709 13:17:13.069572 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:13.069724 (+   152us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:13.069855 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:13.070214 (+   359us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:13.070361 (+   147us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:17:13.070395 (+    34us) write_op.cc:312] APPLY: finished
0709 13:17:13.319736 (+249341us) log.cc:844] Serialized 29 byte log entry
0709 13:17:13.447052 (+127316us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:13.447342 (+   290us) write_op.cc:454] Released schema lock
0709 13:17:13.447453 (+   111us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:13.777338 (+329885us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277fffe28 00007fa27ba3a581 00007fa27ba39d58 00007fa27ba3e9f1 00007fa27ba3e9a9 00007fa27ba3e939 00007fa27ba3e901 00007fa27ba3db2d 000000000038f354 000000000038f199
0709 13:17:13.777707 (+   369us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:13.777744 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4671809,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":26916,"apply.run_cpu_time_us":4462,"apply.run_wall_time_us":379036,"num_ops":2,"prepare.queue_time_us":1000654,"prepare.run_cpu_time_us":28814,"prepare.run_wall_time_us":1751530,"raft.queue_time_us":306276,"raft.run_cpu_time_us":15764,"raft.run_wall_time_us":23991,"replication_time_us":958095,"spinlock_wait_cycles":24329600,"thread_start_us":1398162,"threads_started":4,"wal-append.queue_time_us":448587}]]}
I20260709 13:17:14.136766 12134 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:14.137998 12259 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:14.138751 12259 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.139387 10054 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:14.126776 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:14.032596 12036 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.144928 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1123) took 1344 ms. Trace:
I20260709 13:17:14.145725 10054 rpcz_store.cc:276] 0709 13:17:12.795820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.132121 (+1336301us) service_pool.cc:224] Handling call
0709 13:17:14.139155 (+  7034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.139995 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.140316 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.146946 10125 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:14.146006 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.152292 10051 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:17:14.152976 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1114) took 2462 ms. Trace:
I20260709 13:17:14.153211 10051 rpcz_store.cc:276] 0709 13:17:11.690897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.122271 (+2431374us) service_pool.cc:224] Handling call
0709 13:17:14.147329 (+ 25058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.152705 (+  5376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.152925 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:14.118531 12018 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:14.160642 10050 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:14.163690 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1375) took 2044 ms. Trace:
I20260709 13:17:14.164227 10198 rpcz_store.cc:276] 0709 13:17:12.119503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.446373 (+1326870us) service_pool.cc:224] Handling call
0709 13:17:14.163430 (+717057us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54001d0954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.163616 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080669952}
I20260709 13:17:14.164659 10053 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:17:14.163861 12325 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.071s	user 0.000s	sys 0.003s
W20260709 13:17:14.157274 10020 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.915s (SENT)
I20260709 13:17:14.170933 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.176522 12171 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:14.180676 10051 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
W20260709 13:17:14.181780 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96272.4us
W20260709 13:17:14.166131 12267 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.117s	user 0.001s	sys 0.031s
I20260709 13:17:14.190981 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1375) took 3784 ms. Trace:
I20260709 13:17:14.191531 10055 rpcz_store.cc:276] 0709 13:17:10.401277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.961260 (+3559983us) service_pool.cc:224] Handling call
0709 13:17:13.961734 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.962078 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:14.017251 (+ 55173us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.185516 (+168265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.185582 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.185636 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.185716 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.185891 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7300,"prepare.queue_time_us":208050,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"thread_start_us":163415,"threads_started":2}
I20260709 13:17:14.193183 10125 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:14.169724 12134 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:17:14.199560 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 146) took 1032 ms. Trace:
I20260709 13:17:14.200057 10054 rpcz_store.cc:276] 0709 13:17:13.166888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.147422 (+980534us) service_pool.cc:224] Handling call
0709 13:17:14.157314 (+  9892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.190089 (+ 32775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:14.190137 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.199212 (+  9075us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:14.199270 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.199341 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.199487 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:14.204152 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:14.189359 12267 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.001s	sys 0.033s
I20260709 13:17:14.198005 11513 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:14.215855 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2515) took 2338 ms. Trace:
I20260709 13:17:14.223839 11513 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:14.234169 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:14.235782 12327 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:17:14.236456 12259 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.244711 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 119) took 1112 ms. Trace:
W20260709 13:17:14.252004 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106113us
I20260709 13:17:14.253933 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1112) took 3907 ms. Trace:
W20260709 13:17:14.258319 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.254534 10126 rpcz_store.cc:276] 0709 13:17:10.341480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.747253 (+3405773us) service_pool.cc:224] Handling call
0709 13:17:13.795761 (+ 48508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:13.832977 (+ 37216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:13.833000 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:13.842285 (+  9285us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.016973 (+174688us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.066290 (+ 49317us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.067247 (+   957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.067455 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:14.088387 (+ 20932us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b180042f0a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:14.239010 (+150623us) raft_consensus.cc:1685] finished
0709 13:17:14.248682 (+  9672us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.248884 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29551,"prepare.queue_time_us":49435,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":10008,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":4585,"spinlock_wait_cycles":22128256,"thread_start_us":143552,"threads_started":1,"wal-append.queue_time_us":277520}
I20260709 13:17:14.180286 12019 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.245354 10123 rpcz_store.cc:276] 0709 13:17:13.127874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.016544 (+888670us) service_pool.cc:224] Handling call
0709 13:17:14.055609 (+ 39065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.120179 (+ 64570us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:14.135870 (+ 15691us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.238920 (+103050us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.238991 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:14.239038 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.239344 (+   306us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.239518 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12153,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1930,"raft.queue_time_us":3228,"thread_start_us":83596,"threads_started":1}
I20260709 13:17:14.207893 12267 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:14.251976 12185 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.252300 12018 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:14.306767 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58161.3us
W20260709 13:17:14.307411 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 128) took 2161 ms (client timeout 2000 ms). Trace:
I20260709 13:17:14.307886 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:14.308032 10199 rpcz_store.cc:269] 0709 13:17:12.145905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.625968 (+480063us) service_pool.cc:224] Handling call
0709 13:17:14.307010 (+1681042us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.307287 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929694592}
I20260709 13:17:14.312668 12307 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:14.316706 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:14.319898 12330 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.228s	user 0.003s	sys 0.000s
I20260709 13:17:14.321924 12021 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.289767 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:14.298118 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1102) took 3837 ms. Trace:
W20260709 13:17:14.314941 10019 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.577s (SENT)
I20260709 13:17:14.325076 10052 rpcz_store.cc:276] 0709 13:17:10.461147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.993465 (+3532318us) service_pool.cc:224] Handling call
0709 13:17:14.047831 (+ 54366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.049679 (+  1848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.297689 (+248010us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b5400087350. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.297962 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123804928}
W20260709 13:17:14.325968 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.300009 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2522) took 1768 ms. Trace:
I20260709 13:17:14.326694 10128 rpcz_store.cc:276] 0709 13:17:12.531907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.965212 (+1433305us) service_pool.cc:224] Handling call
0709 13:17:13.985815 (+ 20603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:13.994988 (+  9173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.299608 (+304620us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b54001c6e50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.299887 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45382528}
I20260709 13:17:14.301887 12329 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:14.302038 12202 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:14.303490 12259 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:14.305776 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.336041 10124 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:14.331521 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 151794us
I20260709 13:17:14.228101 10200 rpcz_store.cc:276] 0709 13:17:11.877968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.004681 (+126713us) service_pool.cc:224] Handling call
0709 13:17:12.005063 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.152127 (+1147064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.208308 (+1056181us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003106d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:14.215757 (+  7449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2585132416}
I20260709 13:17:14.355190 12185 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:14.355877 12185 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:14.357167 12185 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.322741 12021 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.359706 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:14.363590 10126 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:14.362618 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 168036us
W20260709 13:17:14.367820 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.846s (SENT)
I20260709 13:17:14.368571 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.358888 12185 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:14.372251 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:14.391527 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.410766 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:14.411794 10126 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:14.412963 10128 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:17:14.411813 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46562.2us
W20260709 13:17:14.416086 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 237926us
I20260709 13:17:14.415630 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2523) took 1883 ms. Trace:
W20260709 13:17:14.416756 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49018.8us
I20260709 13:17:14.362255 12307 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:14.432695 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.431527 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1385) took 1854 ms. Trace:
W20260709 13:17:14.436798 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 127196us
I20260709 13:17:14.435698 10048 rpcz_store.cc:276] 0709 13:17:12.576639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.119303 (+1542664us) service_pool.cc:224] Handling call
0709 13:17:14.119863 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.132753 (+ 12890us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:14.274339 (+141586us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.429234 (+154895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.429294 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.430628 (+  1334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.431282 (+   654us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.431442 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":80155,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":2196,"thread_start_us":248738,"threads_started":2}
I20260709 13:17:14.419000 10131 rpcz_store.cc:276] 0709 13:17:12.532196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.970443 (+1438247us) service_pool.cc:224] Handling call
0709 13:17:13.982991 (+ 12548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.027775 (+ 44784us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:14.027810 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.028649 (+   839us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.053004 (+ 24355us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:14.053063 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.053166 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:14.318763 (+265597us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:14.372026 (+ 53263us) raft_consensus.cc:1685] finished
0709 13:17:14.372051 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.415568 (+ 43517us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25855,"raft.queue_time_us":31364,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":40611712,"thread_start_us":23691,"threads_started":1,"wal-append.queue_time_us":62722}
W20260709 13:17:14.446503 10020 connection.cc:256] RPC call timeout handler was delayed by -2.2549977500000296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:17:14.447257 10020 connection.cc:256] RPC call timeout handler was delayed by -2.232751313000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2279s for a response
W20260709 13:17:14.447736 10020 connection.cc:256] RPC call timeout handler was delayed by -2.2036416690000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
I20260709 13:17:14.439775 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:14.448859 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:14.450825 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.439291 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.453094 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.371891 12067 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.452633 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.466372 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:14.434824 12201 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.434417 12171 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:14.371518 11513 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.486824 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.465998 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:14.489579 10020 connection.cc:256] RPC call timeout handler was delayed by -1.3459416690000126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3187s for a response
I20260709 13:17:14.497025 10131 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 3.
I20260709 13:17:14.498147 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.499238 10201 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:14.502184 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.503708 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.505417 12021 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:14.506199 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1358) took 2340 ms (client timeout 1830 ms). Trace:
I20260709 13:17:14.506686 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.507879 10202 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:14.507508 12171 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.509068 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.510372 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:14.511276 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.506500 11513 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:14.512014 10203 raft_consensus.cc:2376] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
I20260709 13:17:14.510176 12335 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 13:17:14.506699 10205 rpcz_store.cc:269] 0709 13:17:12.115492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:12.144946 (+ 29454us) service_pool.cc:224] Handling call
0709 13:17:14.428775 (+2283829us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.455823 (+ 27048us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786934144}
I20260709 13:17:14.512444 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.444891 12307 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.540776 10204 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:14.548606 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:14.552238 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1135) took 1745 ms (client timeout 1896 ms). Trace:
W20260709 13:17:14.552733 10196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1372) took 2434 ms (client timeout 3019 ms). Trace:
W20260709 13:17:14.552817 10047 rpcz_store.cc:269] 0709 13:17:12.806739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.114691 (+307952us) service_pool.cc:224] Handling call
0709 13:17:14.551951 (+1437260us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b54001b8354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.552156 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366764288}
W20260709 13:17:14.553339 10196 rpcz_store.cc:269] 0709 13:17:12.118720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.434511 (+1315791us) service_pool.cc:224] Handling call
0709 13:17:14.552533 (+1118022us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.552657 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402636672}
I20260709 13:17:14.555833 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:14.558228 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:14.514118 12019 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.533214 12106 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:14.536957 12276 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:14.568039 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 127580us
I20260709 13:17:14.565102 10198 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:14.570200 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.577679 10200 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 2: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:14.577421 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62757.4us
I20260709 13:17:14.577235 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:14.564680 12106 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:14.600867 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.567008 12276 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:14.559790 12224 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:17:14.622218 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.609134 12106 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.545885 12267 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.612481 12276 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:14.623278 12282 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.566s	user 0.012s	sys 0.006s
W20260709 13:17:14.633126 10864 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.970s (SENT)
I20260709 13:17:14.631362 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:14.650650 12339 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.653010 10896 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:14.622224 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:14.661966 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.663739 12359 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:14.664404 12359 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:14.664808 12359 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:14.668241 11702 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.659019 12298 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.678081 12298 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.663784 12067 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:14.684456 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1388) took 1545 ms. Trace:
I20260709 13:17:14.685025 10049 rpcz_store.cc:276] 0709 13:17:13.139832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.128396 (+988564us) service_pool.cc:224] Handling call
0709 13:17:14.140968 (+ 12572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:14.184964 (+ 43996us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:14.378173 (+193209us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.632817 (+254644us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:14.641038 (+  8221us) log.cc:844] Serialized 26 byte log entry
0709 13:17:14.641208 (+   170us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:14.641243 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.641321 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:14.684170 (+ 42849us) raft_consensus.cc:1685] finished
0709 13:17:14.684197 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.684355 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2155,"prepare.queue_time_us":130202,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":127746,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":2955,"thread_start_us":330806,"threads_started":3,"wal-append.queue_time_us":251792}
I20260709 13:17:14.688102 12307 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:14.692689 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59630.7us
W20260709 13:17:14.694244 10864 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.002s (SENT)
W20260709 13:17:14.654212 12282 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.777s	user 0.014s	sys 0.007s
I20260709 13:17:14.723382 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:14.728223 12165 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.730079 12298 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:14.731072 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.737272 10892 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:14.741062 12212 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:14.741888 12212 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.744439 12212 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.745234 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:14.746312 10049 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:14.746903 12298 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:14.749207 12298 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:14.749687 12298 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.749830 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:14.750614 10047 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:17:14.752893 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.750761 12298 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.754351 12298 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.756233 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:14.756464 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 421) took 1579 ms. Trace:
I20260709 13:17:14.757265 10048 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:14.758054 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:14.757009 10897 rpcz_store.cc:276] 0709 13:17:13.192785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.193611 (+   826us) service_pool.cc:224] Handling call
0709 13:17:13.614567 (+420956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:13.630599 (+ 16032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.756183 (+1125584us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540063ef50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:14.756372 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965457280}
I20260709 13:17:14.760171 11702 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:14.762310 10095 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.631s (SENT)
I20260709 13:17:14.763000 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:14.769222 10047 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:14.765650 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:14.727396 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162487us
I20260709 13:17:14.789130 10047 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:17:14.789785 10047 pending_rounds.cc:85] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:17:14.790158 10047 pending_rounds.cc:107] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:14.790570 10047 raft_consensus.cc:2889] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:14.791388 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:14.793757 10049 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:14.773762 12185 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:14.799069 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.788335 12202 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:14.810106 10049 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:14.794723 12165 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:17:14.811954 12165 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:14.812426 12165 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.813545 11702 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.813472 12165 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:14.817157 10129 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:14.822521 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:14.833917 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1400) took 1090 ms. Trace:
I20260709 13:17:14.835495 10897 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:14.835883 10047 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:14.816031 12202 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:14.808010 12212 raft_consensus.cc:2764] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:17:14.844383 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45398.7us
W20260709 13:17:14.848526 10095 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.715s (SENT)
I20260709 13:17:14.815006 12165 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:14.850356 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:14.857069 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:14.858072 10047 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:17:14.861572 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:14.862663 12259 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.001s
I20260709 13:17:14.865366 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:14.867288 10130 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:14.867601 10123 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:14.847993 12323 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.814s	user 0.004s	sys 0.001s
I20260709 13:17:14.834976 10050 rpcz_store.cc:276] 0709 13:17:13.743940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.161826 (+417886us) service_pool.cc:224] Handling call
0709 13:17:14.162164 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.162426 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:14.421549 (+259123us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.656187 (+234638us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.833049 (+176862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:14.833137 (+    88us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:14.833207 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.833333 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:14.833775 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22573,"prepare.queue_time_us":113736,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":101,"raft.queue_time_us":104192,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":206,"thread_start_us":585375,"threads_started":4,"wal-append.queue_time_us":267145}
I20260709 13:17:14.870884 12267 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.873579 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:14.878414 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.879328 12106 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.882160 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:14.843532 12018 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:17:14.884874 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.900213 10056 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:14.911230 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 70881.3us
I20260709 13:17:14.912691 12352 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:14.902951 12106 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.907992 12358 raft_consensus.cc:2764] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:14.916409 12365 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.917280 12201 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:14.917774 12201 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:14.918172 12201 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:14.926738 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:14.853269 12185 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:17:14.848136 12202 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.902978 12271 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:17:14.971346 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1127) took 1236 ms. Trace:
I20260709 13:17:14.971863 12259 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.971370 11513 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:17:14.972009 10124 rpcz_store.cc:276] 0709 13:17:13.735021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.125152 (+390131us) service_pool.cc:224] Handling call
0709 13:17:14.131378 (+  6226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.343823 (+212445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:14.970976 (+627153us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:14.971217 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469993344}
I20260709 13:17:14.975600 11513 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:14.976176 11513 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.977378 11513 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.971534 12018 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:17:14.979427 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:14.979912 12018 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:14.980561 12018 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:14.980933 12018 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:14.982039 12018 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:14.983932 11513 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:14.985737 12018 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:14.988585 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:14.995549 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:14.996332 10130 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:14.998368 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:15.001756 10126 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:15.005007 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:15.005837 10130 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:14.851847 12175 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
W20260709 13:17:14.957864 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45727.7us
I20260709 13:17:14.950587 12106 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:15.009898 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100155us
I20260709 13:17:14.928493 12212 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:17:15.018392 12212 consensus_queue.cc:1243] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:15.019102 12212 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:15.019577 12212 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:15.019657 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 332) took 1184 ms. Trace:
W20260709 13:17:15.021147 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 121038us
I20260709 13:17:15.020745 12212 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.022195 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:15.022418 12212 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:15.023317 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66221.8us
I20260709 13:17:15.024017 12276 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.025430 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:15.026544 12276 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:14.929143 12365 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:17:15.027410 12357 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:14.960414 12271 consensus_queue.cc:1243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:15.028416 12357 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:15.029780 12357 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:15.030179 12357 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.030530 12357 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:15.033576 12359 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.041975 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:15.068181 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:15.071066 10131 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:15.061344 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:15.076022 10123 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:15.078395 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:15.080147 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.081532 10126 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:15.078511 10125 pending_rounds.cc:85] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:17:15.085745 10125 pending_rounds.cc:107] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:17:15.086234 10125 raft_consensus.cc:2889] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:15.086829 10049 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:15.073580 12365 consensus_queue.cc:1243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:15.068640 12162 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:15.098097 10128 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:15.062960 12282 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:15.077538 12224 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:15.088451 12276 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:15.068334 12201 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.105085 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:15.105167 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.112612 10051 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:15.112363 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90292.9us
I20260709 13:17:15.101243 12365 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:15.092433 11702 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
W20260709 13:17:15.131098 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.112506 12276 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.135000 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:15.138103 10095 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.562s (SENT)
I20260709 13:17:15.107396 12224 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:15.119567 12365 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.105429 12162 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:15.146993 12202 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.143031 12276 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:15.192307 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:15.197204 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:15.177867 12365 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:15.201527 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63500.2us
I20260709 13:17:15.119390 12282 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:15.208966 12165 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:15.199755 12067 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:15.218744 10051 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:15.189755 12259 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:15.253451 12259 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:15.254630 12259 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.256155 12259 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:15.259354 12224 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:15.261207 12224 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:15.204474 12367 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:17:15.222846 12373 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:17:15.199646 12202 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:15.224666 12369 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:17:15.200637 12021 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:17:15.243733 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:15.282059 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:15.244496 12162 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:15.244910 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.245772 12201 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:15.246373 10198 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:15.249716 12357 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.252246 12282 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:15.020289 10891 rpcz_store.cc:276] 0709 13:17:13.835788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:14.439777 (+603989us) service_pool.cc:224] Handling call
0709 13:17:14.548134 (+108357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.611748 (+ 63614us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:14.727750 (+116002us) log.cc:844] Serialized 14 byte log entry
0709 13:17:14.975243 (+247493us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18000effe0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:17:15.019066 (+ 43823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:15.019187 (+   121us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:15.019264 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.019359 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:15.019527 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23189,"prepare.queue_time_us":179155,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":43957,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"spinlock_wait_cycles":5568768,"thread_start_us":76408,"threads_started":2,"wal-append.queue_time_us":302974}
I20260709 13:17:15.262872 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.256155 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.273512 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:15.276038 10095 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.694s (SENT)
I20260709 13:17:15.267643 12359 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:15.259354 12165 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:15.329732 10051 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:15.342726 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:15.343071 10864 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.685s (SENT)
I20260709 13:17:15.343640 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:15.343879 10198 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:15.347097 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 221531us
I20260709 13:17:15.355043 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:15.355403 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:15.357052 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 159984us
I20260709 13:17:15.337721 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:15.349633 12372 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
I20260709 13:17:15.352305 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.387310 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:15.388342 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:15.391142 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48095.3us
W20260709 13:17:15.401934 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:15.404644 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 128694us
I20260709 13:17:15.393998 10202 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:15.406200 10198 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:15.406792 10204 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:15.394248 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.367813 12271 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:15.425304 10051 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:15.389171 12165 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:15.436211 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:15.444393 10095 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.731s (SENT)
I20260709 13:17:15.445286 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.446010 10198 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:15.448349 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:15.410244 12245 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:15.432307 12201 raft_consensus.cc:2764] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:17:15.435243 12212 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:15.402872 11702 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:17:15.473291 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:15.440574 12276 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:17:15.472466 12382 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:17:15.489711 10200 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:15.490790 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.493403 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.495065 10198 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:15.466321 12245 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:15.500011 10054 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:15.500305 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55946.5us
I20260709 13:17:15.407938 12383 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:17:15.472757 12365 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.488137 12386 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:15.513741 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:15.515585 10095 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.745s (SENT)
I20260709 13:17:15.516145 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:15.521472 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.523052 12268 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:15.527331 12212 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:15.528152 12212 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:15.529418 12358 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:15.529906 12358 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:15.507522 12202 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:15.546947 10050 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:15.547605 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:15.547797 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:15.546303 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.549891 10048 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:17:15.555332 10095 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.796s (SENT)
I20260709 13:17:15.564914 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:15.545949 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.564895 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:15.544978 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.565860 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.116237 10050 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:15.823237 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.117142 10200 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:17:15.623224 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.547361 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:15.647379 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78889.8us
I20260709 13:17:15.617823 10054 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:15.699039 10047 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:17:15.836126 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 57436.3us
I20260709 13:17:15.611760 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.663179 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1118) took 4961 ms. Trace:
W20260709 13:17:15.836838 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.658897 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:16.124394 10129 rpcz_store.cc:276] 0709 13:17:10.701450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:13.921491 (+3220041us) service_pool.cc:224] Handling call
0709 13:17:13.921922 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:14.821352 (+899430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.662742 (+841390us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b5400389ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:15.663038 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058256256}
W20260709 13:17:15.614283 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:16.122949 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 567607us
W20260709 13:17:16.126031 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 534630us
I20260709 13:17:15.638796 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:16.126307 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 609131us
I20260709 13:17:15.786211 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.117372 10055 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:16.133208 10199 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:15.750391 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:15.821485 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:15.659862 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.619306 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.641419 10130 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 13:17:16.130399 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.135447 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1157) took 1013 ms. Trace:
I20260709 13:17:16.125854 10048 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:16.135880 10130 rpcz_store.cc:276] 0709 13:17:15.122446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.122676 (+   230us) service_pool.cc:224] Handling call
0709 13:17:15.640689 (+518013us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:15.641030 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.135063 (+494033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.135371 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25912576}
I20260709 13:17:16.136952 10196 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:15.636601 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:16.131930 10204 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4. Re-sending same reply.
W20260709 13:17:16.170601 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47644.8us
I20260709 13:17:16.169163 10052 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:16.115895 10205 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:16.178793 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.180158 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:16.181016 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59942.2us
I20260709 13:17:16.181782 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.182896 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1403) took 1088 ms. Trace:
I20260709 13:17:16.183377 10053 rpcz_store.cc:276] 0709 13:17:15.094693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.748025 (+653332us) service_pool.cc:224] Handling call
0709 13:17:16.182844 (+434819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:16.184315 10203 raft_consensus.cc:2368] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 7.
W20260709 13:17:16.183385 10864 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.005s (SENT)
W20260709 13:17:16.189901 10095 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.819s (SENT)
I20260709 13:17:16.190601 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:15.657500 12396 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:17:16.193884 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:15.536361 12357 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:15.771835 12259 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.003s
I20260709 13:17:16.194633 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.195830 10205 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:16.197036 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.531028 12212 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:15.552213 12276 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:16.169004 12268 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.204461 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.204713 10050 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:16.198357 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.639168 12282 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.553781 12365 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:16.197379 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.185515 12359 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.202864 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:15.637317 12384 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:16.206696 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:16.207513 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.195654 10197 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:16.214787 10200 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:17:16.215794 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83789.8us
I20260709 13:17:16.218077 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.198405 12019 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:16.221685 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.222572 10204 raft_consensus.cc:2368] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4.
I20260709 13:17:16.223637 10053 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:17:15.536221 12358 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:15.540729 12245 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.206061 12378 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:16.233560 10864 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.042s (SENT)
W20260709 13:17:16.235576 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45715.1us
I20260709 13:17:16.236881 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.237674 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.238257 10205 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:16.238895 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.240873 10204 raft_consensus.cc:2376] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:16.231925 12298 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.250456 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:16.250646 10095 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.830s (SENT)
I20260709 13:17:16.251643 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:16.252378 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.211318 12268 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.240082 12384 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:16.239951 12358 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:16.215227 12282 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.236919 12019 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:16.255817 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.220777 12359 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:16.249300 12271 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.000s
I20260709 13:17:16.248950 12165 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:16.261408 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1410) took 1133 ms. Trace:
I20260709 13:17:16.254400 12245 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:16.270797 10056 rpcz_store.cc:276] 0709 13:17:15.128553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.379972 (+251419us) service_pool.cc:224] Handling call
0709 13:17:15.392944 (+ 12972us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:15.393614 (+   670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:15.393654 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:15.408847 (+ 15193us) log.cc:844] Serialized 52 byte log entry
0709 13:17:15.500577 (+ 91730us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:15.500641 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:15.500737 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:16.261139 (+760402us) raft_consensus.cc:1685] finished
0709 13:17:16.261173 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.261331 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":217,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":9219,"thread_start_us":36286,"threads_started":1,"wal-append.queue_time_us":101906}
I20260709 13:17:16.265663 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.269297 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.276813 10201 raft_consensus.cc:2376] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6. Re-sending same reply.
I20260709 13:17:16.282886 10197 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:16.283511 11702 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:16.285091 12268 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:16.285835 12268 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:16.290030 10202 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:16.253751 12298 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.296144 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.265923 12271 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:16.312335 12276 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:16.313628 12276 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:16.314035 12276 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:16.270081 12384 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:16.316829 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.327915 10197 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:16.328835 10202 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:16.316555 12402 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:16.369321 10204 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:16.365886 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:16.370244 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 163610us
I20260709 13:17:16.297349 12365 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:16.366881 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 188213us
I20260709 13:17:16.374295 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.376922 11513 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:17:16.378621 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.381817 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.381595 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.384325 10197 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:16.386809 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.388391 12405 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.389142 12405 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.387640 10197 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:16.394799 10205 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:16.402594 11513 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.403541 11513 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:16.403824 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 222072us
I20260709 13:17:16.405578 11513 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:16.406749 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.408416 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.410574 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.372257 12267 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:16.387063 12282 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:16.451875 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.444653 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:16.442507 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.447664 11513 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:16.463166 11513 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:17:16.468773 11513 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.463959 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:16.489926 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 210007us
I20260709 13:17:16.490415 10197 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:17:16.451763 10205 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:16.475528 10202 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:16.492525 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.493999 11513 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.330868 12276 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.449492 12407 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:16.513182 10125 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:17:16.516175 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64365.3us
I20260709 13:17:16.525025 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.531431 10198 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:16.532701 11513 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:16.485716 12365 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:16.491173 12282 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:16.537577 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1383) took 1019 ms. Trace:
I20260709 13:17:16.547861 10198 rpcz_store.cc:276] 0709 13:17:15.518026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.559253 (+ 41227us) service_pool.cc:224] Handling call
0709 13:17:16.537474 (+978221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:16.550091 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.551297 10204 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:17:16.552968 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108443us
I20260709 13:17:16.555897 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.482468 12267 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.401292 12019 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.534008 12405 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:16.577569 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 329563us
I20260709 13:17:16.515508 12412 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:16.558640 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:16.546782 12282 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:16.593050 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.601863 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.607298 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.614439 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.619787 10205 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:16.587306 12359 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:16.599104 12282 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:16.546214 12384 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.598150 12276 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:16.643622 12419 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:16.644114 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:16.644156 12391 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.172s	user 0.018s	sys 0.035s
I20260709 13:17:16.645015 10048 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:16.646039 10048 raft_consensus.cc:606] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.646443 10048 raft_consensus.cc:614] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:16.601331 12405 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.609990 12365 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:16.648285 10199 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:16.618306 12404 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.587236 12412 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.653223 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.665189 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.676430 10196 raft_consensus.cc:2376] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 2. Re-sending same reply.
I20260709 13:17:16.677299 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1395) took 1113 ms. Trace:
I20260709 13:17:16.677770 10196 rpcz_store.cc:276] 0709 13:17:15.564207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.179250 (+615043us) service_pool.cc:224] Handling call
0709 13:17:16.664942 (+485692us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:16.677209 (+ 12267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3356544}
I20260709 13:17:16.623853 12185 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:16.623217 12201 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:16.683233 10199 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:17:16.683900 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105420us
W20260709 13:17:16.686168 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61760.6us
I20260709 13:17:16.687749 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.630683 12298 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:16.699779 10204 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:17:16.700495 10204 pending_rounds.cc:85] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 2
I20260709 13:17:16.700846 10204 pending_rounds.cc:107] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:17:16.701195 10204 raft_consensus.cc:2889] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:16.711511 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 230831us
W20260709 13:17:16.712524 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 195264us
I20260709 13:17:16.674517 12365 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:16.716823 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161047us
W20260709 13:17:16.717574 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 437688us
I20260709 13:17:16.718421 12421 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.721529 12419 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.725768 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:16.726889 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.656286 12271 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:17:16.774088 10196 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:17:16.775236 10196 pending_rounds.cc:85] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 1
I20260709 13:17:16.775475 10196 pending_rounds.cc:107] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:17:16.659996 12270 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:17:16.782155 10047 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:16.781277 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.783147 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1170) took 1164 ms. Trace:
I20260709 13:17:16.783577 10047 rpcz_store.cc:276] 0709 13:17:15.618487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.781432 (+1162945us) service_pool.cc:224] Handling call
0709 13:17:16.781773 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.782790 (+  1017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.783038 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:16.755959 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:16.766867 10095 connection.cc:256] RPC call timeout handler was delayed by -1.6542235820002134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26320000000000005s for a response
W20260709 13:17:16.773977 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47220.1us
W20260709 13:17:16.709481 12173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.005s	sys 0.020s
I20260709 13:17:16.775653 10196 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:16.755054 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1392) took 1208 ms. Trace:
W20260709 13:17:16.778258 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90664.6us
I20260709 13:17:16.778653 10049 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 13:17:16.779600 10129 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:16.780138 10053 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:16.780498 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:16.781240 10124 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:17:16.784791 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.794463 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.790707 10203 raft_consensus.cc:2376] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:16.788231 10201 rpcz_store.cc:276] 0709 13:17:15.546537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.618912 (+ 72375us) service_pool.cc:224] Handling call
0709 13:17:16.754782 (+1135870us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:16.754964 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326345472}
I20260709 13:17:16.682727 12405 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.809482 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1168) took 1191 ms. Trace:
I20260709 13:17:16.901065 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:16.923085 10049 rpcz_store.cc:276] 0709 13:17:15.617936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.777939 (+1160003us) service_pool.cc:224] Handling call
0709 13:17:16.778384 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:16.789439 (+ 11055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.809386 (+ 19947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:16.904405 10124 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:16.905602 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1169) took 1287 ms. Trace:
W20260709 13:17:16.804454 10095 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.910s (SENT)
W20260709 13:17:16.786377 12173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.009s	sys 0.021s
I20260709 13:17:16.907265 10204 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:16.913988 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.912602 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:16.930347 10050 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:16.908291 10052 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:16.910907 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:16.786512 12019 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.736986 12365 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:16.938184 12019 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.774022 12421 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.943969 12267 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:16.946066 12421 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:16.923815 10051 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:16.695451 12185 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.925055 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:16.924805 10054 rpcz_store.cc:276] 0709 13:17:15.618218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.777870 (+1159652us) service_pool.cc:224] Handling call
0709 13:17:16.905181 (+127311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.905370 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.905386 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:16.905418 (+    32us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:16.905442 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:16.905475 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:16.905549 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:16.785897 12410 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.682789 12404 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:16.927717 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 133314us
I20260709 13:17:16.805344 12420 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:16.933315 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:16.786494 12201 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:16.787561 12378 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:16.973325 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:16.900710 10005 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 105112us
I20260709 13:17:16.937318 10129 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:16.956259 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:16.973330 10202 raft_consensus.cc:2376] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
I20260709 13:17:16.980962 10052 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:17:16.984326 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:16.975504 10124 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:16.779224 12419 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.951783 12405 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:16.724504 12358 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:16.995319 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 239652us
W20260709 13:17:16.997217 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.028s (SENT)
I20260709 13:17:16.969116 10201 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
W20260709 13:17:17.028523 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63384.8us
I20260709 13:17:17.039336 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:16.967875 12359 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.039980 10201 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:17.040437 12359 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.068439 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 263994us
W20260709 13:17:17.075574 10127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.008s	sys 0.052s
W20260709 13:17:17.076246 10127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.008s	sys 0.053s
I20260709 13:17:17.101498 10202 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:16.969027 12268 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:17:16.975225 12435 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:16.947845 12365 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:16.986148 12433 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:17.112216 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115042us
W20260709 13:17:17.120922 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81784.5us
I20260709 13:17:17.122375 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.030201 12358 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:17.154695 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:17.154783 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63544.3us
W20260709 13:17:17.120998 12398 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:17:17.157456 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.072471 12270 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.983235 12201 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:16.988243 12378 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.169736 10095 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.118s (SENT)
I20260709 13:17:16.977231 12410 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:17.173838 10201 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:17.196739 10202 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:17.204071 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.098757 12298 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.120759 12421 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:17.217207 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.152606 12405 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:17.223385 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 289053us
I20260709 13:17:17.228211 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:17.170849 12358 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:17.237190 10197 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.002s
I20260709 13:17:17.241124 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.249032 10197 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:17.264173 10204 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:17.265703 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1410) took 1497 ms. Trace:
I20260709 13:17:17.266398 10197 rpcz_store.cc:276] 0709 13:17:15.768368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.493096 (+724728us) service_pool.cc:224] Handling call
0709 13:17:17.265557 (+772461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:17.108636 12434 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.133s
W20260709 13:17:17.275367 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.205727 12435 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:17.255437 10202 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:17:17.244691 12384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.005s	sys 0.044s
I20260709 13:17:17.296365 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.267485 12376 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.494s	user 0.018s	sys 0.023s
I20260709 13:17:17.310551 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:17.314575 10892 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:17.315361 12419 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.316579 12404 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.244262 12421 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.319901 12425 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 13:17:17.320257 12421 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.321622 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.322446 10896 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:17.238981 12259 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 13:17:17.324743 12432 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 13:17:17.326450 12426 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
W20260709 13:17:17.328069 12428 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:17.329347 12378 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.330370 12378 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:17.220647 12270 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.330401 12430 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:17:17.304538 10197 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:17.217231 12165 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:17:17.306053 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.342842 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.343708 10202 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:17.344764 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.291056 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:17.305576 12384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.005s	sys 0.045s
I20260709 13:17:17.238009 12298 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:17.364753 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.360360 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.312829 12376 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.540s	user 0.018s	sys 0.024s
I20260709 13:17:17.380632 10202 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:17.394572 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.351596 12173 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.316579 12410 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:17.426774 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 306955us
I20260709 13:17:17.430284 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.353665 12378 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:17:17.417285 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 137) took 1020 ms. Trace:
I20260709 13:17:17.356469 12412 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.384624 12359 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.402937 12019 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:17.509234 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 299968us
I20260709 13:17:17.390852 12420 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.520934 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.508208 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 343538us
I20260709 13:17:17.514987 10126 rpcz_store.cc:276] 0709 13:17:16.397168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.397438 (+   270us) service_pool.cc:224] Handling call
0709 13:17:16.757878 (+360440us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:16.758086 (+   208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.758341 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:16.807917 (+ 49576us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.393017 (+585100us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.412217 (+ 19200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.412307 (+    90us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:17.412384 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.412497 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.417174 (+  4677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18660,"prepare.queue_time_us":46609,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":3862,"raft.queue_time_us":162462,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":3928192,"thread_start_us":486813,"threads_started":2,"wal-append.queue_time_us":452197}
I20260709 13:17:17.571027 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.393059 12434 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
W20260709 13:17:17.594213 10198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.001s	sys 0.016s
I20260709 13:17:17.595824 10197 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:17.596879 10202 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:17.620113 10198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.001s	sys 0.017s
I20260709 13:17:17.621984 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.637987 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:17.639389 10095 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.141s (SENT)
W20260709 13:17:17.640516 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.702s (SENT)
I20260709 13:17:17.640611 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.641844 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:17.642529 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.085s (SENT)
I20260709 13:17:17.642742 10896 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:17.643745 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.649518 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.649650 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 409837us
I20260709 13:17:17.650697 10197 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:17.429411 12173 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:17.619882 12359 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.656764 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:17.511000 12442 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.219s
W20260709 13:17:17.597883 12271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.006s	sys 0.011s
I20260709 13:17:17.654876 10202 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:17.667814 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.639588 12270 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.670585 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.621428 12165 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 13:17:17.665552 12271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.007s	sys 0.012s
I20260709 13:17:17.661507 12404 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 13:17:17.694669 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 92671.5us
I20260709 13:17:17.665452 12365 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.653254 12421 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
W20260709 13:17:17.709779 12276 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.070s	user 0.009s	sys 0.005s
I20260709 13:17:17.710314 12437 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:17:17.709465 12462 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.623538 12420 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:17.681180 12443 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.740547 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.740824 10095 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.215s (SENT)
I20260709 13:17:17.773310 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:17.785558 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.786428 10197 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:17.694377 12201 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.702373 12422 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.760448 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.665743 12359 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:17.709503 12173 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:17.806402 10865 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.689s (SENT)
W20260709 13:17:17.807967 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.722s (SENT)
I20260709 13:17:17.808452 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:17.805999 10896 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:17.761251 12276 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.816859 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:17.818166 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:17.819104 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.772217 12462 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:17.824249 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.015s
W20260709 13:17:17.884817 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.011s	sys 0.016s
I20260709 13:17:17.849743 12365 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.825138 12453 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:17:17.824585 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:17.826313 12451 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 13:17:17.827627 12458 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:17:17.828567 12450 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:17:17.880455 10895 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:17.784608 12443 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:17.832849 12420 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:17.834410 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:17.834717 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:17.824585 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:17.903163 12352 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.345s	user 0.017s	sys 0.021s
I20260709 13:17:17.840502 10197 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:17.887480 12376 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:17.835882 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:17.867568 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59614.3us
W20260709 13:17:17.919900 10865 leader_election.cc:336] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.559s (SENT)
I20260709 13:17:17.920609 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:17.872218 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:17.811728 12201 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:17.820717 12211 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:17.819747 12298 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:17.824551 12447 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:17:17.933652 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1417) took 1142 ms. Trace:
W20260709 13:17:17.888092 10020 connection.cc:256] RPC call timeout handler was delayed by -1.7909688239999468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
I20260709 13:17:17.893520 10900 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:17.820202 12276 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.940263 10020 connection.cc:256] RPC call timeout handler was delayed by -1.7567688239998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:17:17.941066 10020 connection.cc:256] RPC call timeout handler was delayed by -1.5227688239999679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 13:17:17.941424 10020 connection.cc:256] RPC call timeout handler was delayed by -1.3868688239999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 13:17:17.942174 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:17.910312 12270 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:17.812937 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 156284us
I20260709 13:17:17.811379 12422 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:17.910344 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76010.6us
I20260709 13:17:17.912891 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:17.759593 10202 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:17.905601 12358 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.000s
I20260709 13:17:17.889045 12464 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:17.914248 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 79656.1us
I20260709 13:17:17.934317 10053 rpcz_store.cc:276] 0709 13:17:16.791543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.791696 (+   153us) service_pool.cc:224] Handling call
0709 13:17:16.792072 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.792341 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:17.098399 (+306058us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.933053 (+834654us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.933136 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:17.933210 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.933320 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.933537 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6493,"prepare.queue_time_us":533979,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":263370,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":7605,"thread_start_us":1138443,"threads_started":3}
W20260709 13:17:17.911990 12352 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.362s	user 0.018s	sys 0.021s
W20260709 13:17:17.830049 12282 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.195s	user 0.014s	sys 0.000s
I20260709 13:17:17.900028 12365 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:17.953797 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:17.948249 12444 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.347s	user 0.001s	sys 0.001s
I20260709 13:17:17.992619 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1419) took 1085 ms. Trace:
I20260709 13:17:17.982995 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1439) took 1180 ms. Trace:
I20260709 13:17:17.993352 10201 rpcz_store.cc:276] 0709 13:17:16.766656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.040916 (+274260us) service_pool.cc:224] Handling call
0709 13:17:17.041310 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.218268 (+176958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.946827 (+728559us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:17.947086 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287866752}
I20260709 13:17:17.936385 12173 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:17.918751 12420 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:17.994138 12473 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.004709 12473 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.007550 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:17.985620 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1421) took 2093 ms. Trace:
W20260709 13:17:18.008905 12454 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.232s	user 0.000s	sys 0.002s
I20260709 13:17:18.008704 10205 rpcz_store.cc:276] 0709 13:17:15.892097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.492117 (+600020us) service_pool.cc:224] Handling call
0709 13:17:17.951070 (+1458953us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003548d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:17.985359 (+ 34289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289529600}
I20260709 13:17:17.988119 12358 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.011621 12435 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:18.012043 12435 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:18.012307 10048 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:18.013020 12435 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:17.991573 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49502.9us
I20260709 13:17:18.014091 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1413) took 1246 ms. Trace:
I20260709 13:17:18.014566 10048 rpcz_store.cc:276] 0709 13:17:16.767545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.768393 (+   848us) service_pool.cc:224] Handling call
0709 13:17:16.786309 (+ 17916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.012897 (+1226588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.013841 (+   944us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400086e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:18.014030 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2818954624}
I20260709 13:17:18.015184 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:17.935063 12201 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.018043 12201 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.930027 12376 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.019747 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1173) took 1227 ms. Trace:
I20260709 13:17:18.022815 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:17.995447 10197 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:18.022586 11513 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:17:17.989792 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.982141 12245 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:18.013589 12358 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.021015 12443 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.997465 12462 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:17.999645 12420 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:17.995293 12282 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.018705 12201 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.020287 10047 rpcz_store.cc:276] 0709 13:17:16.792321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.799356 (+  7035us) service_pool.cc:224] Handling call
0709 13:17:16.911334 (+111978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.254770 (+343436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:17.254818 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.254878 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:17.254930 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.019451 (+764521us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54001b7454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:18.019503 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.019656 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789038976}
I20260709 13:17:18.025836 10202 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:18.042330 11513 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:18.046833 11513 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:18.035717 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 43838.3us
I20260709 13:17:18.031677 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1430) took 1755 ms. Trace:
I20260709 13:17:18.035841 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1167) took 2418 ms. Trace:
I20260709 13:17:18.047505 11513 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.035485 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:17.993052 10056 rpcz_store.cc:276] 0709 13:17:16.907104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.908866 (+  1762us) service_pool.cc:224] Handling call
0709 13:17:16.909347 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.909641 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.909676 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.921694 (+ 12018us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.209312 (+287618us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:17.209356 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.209433 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.992312 (+782879us) raft_consensus.cc:1685] finished
0709 13:17:17.992342 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.992533 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":225270,"threads_started":1,"wal-append.queue_time_us":431887}
I20260709 13:17:18.047520 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:18.055505 12359 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:18.056083 12467 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:17:18.036204 12458 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:17:18.057605 12486 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:18.058854 12422 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:18.003808 10892 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:18.059763 12487 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:17:18.061476 12487 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:18.061959 12487 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:17.998208 12483 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.046561 12358 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:18.054720 10202 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:18.067773 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:18.047713 10199 rpcz_store.cc:276] 0709 13:17:16.228374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.685527 (+457153us) service_pool.cc:224] Handling call
0709 13:17:16.714502 (+ 28975us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.714994 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.715030 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.774365 (+ 59335us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.218644 (+444279us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:17.651966 (+433322us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.911175 (+259209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.911293 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:17.949339 (+ 38046us) raft_consensus.cc:1685] finished
0709 13:17:17.949360 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:17.983838 (+ 34478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8645,"prepare.queue_time_us":76516,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":9292,"raft.queue_time_us":158618,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":57762,"thread_start_us":874820,"threads_started":3,"wal-append.queue_time_us":594315}
I20260709 13:17:18.069691 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.014314 12435 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:18.048098 10055 rpcz_store.cc:276] 0709 13:17:15.617596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.779071 (+1161475us) service_pool.cc:224] Handling call
0709 13:17:16.779483 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.779797 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:16.779838 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.808629 (+ 28791us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.245009 (+436380us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:17.245056 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:17.245133 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.007760 (+762627us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:18.025658 (+ 17898us) raft_consensus.cc:1685] finished
0709 13:17:18.025685 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.035746 (+ 10061us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13346688,"thread_start_us":257213,"threads_started":1,"wal-append.queue_time_us":701958}
I20260709 13:17:17.998358 12476 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:18.079095 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:17.998100 12456 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:18.039094 10891 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:18.079648 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.151815 10054 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:18.045684 12211 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:18.003727 12490 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:18.158725 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 507672us
I20260709 13:17:18.160249 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:18.161437 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:18.163029 12376 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:18.163831 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 423021us
I20260709 13:17:18.165022 12211 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.100150 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:18.166867 12457 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:18.103101 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 88009.7us
I20260709 13:17:18.146924 12298 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.168639 12298 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.132279 12276 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.133262 12282 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.063033 12487 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.171075 12378 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:18.123456 12473 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.140162 12365 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.172070 12468 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:18.174571 12474 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:18.175029 12463 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:18.175987 12211 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.175812 12463 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.176256 12474 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:18.178161 12448 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:18.137472 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 143) took 1409 ms. Trace:
I20260709 13:17:18.178675 12463 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.182114 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:18.077715 12201 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.184386 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:18.168684 12457 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:18.076602 12483 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.204198 10200 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:18.178856 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:18.183526 10899 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 4 LEARNER]: Advancing to term 6
I20260709 13:17:18.205248 10203 raft_consensus.cc:606] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:18.205786 10203 raft_consensus.cc:614] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:18.206107 12483 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.206800 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1427) took 1979 ms. Trace:
I20260709 13:17:18.205283 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 155) took 1088 ms. Trace:
I20260709 13:17:18.207918 10200 rpcz_store.cc:276] 0709 13:17:17.117036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.203273 (+1086237us) service_pool.cc:224] Handling call
0709 13:17:18.203841 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.204876 (+  1035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.205196 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:18.207355 10196 rpcz_store.cc:276] 0709 13:17:16.227455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.678921 (+451466us) service_pool.cc:224] Handling call
0709 13:17:16.762833 (+ 83912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.787757 (+ 24924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.206557 (+1418800us) spinlock_profiling.cc:243] Waited 66.2 ms on lock 0x7b54001bfb50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:18.206724 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152252672}
I20260709 13:17:17.997961 12481 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:18.145251 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1170) took 1749 ms. Trace:
I20260709 13:17:18.149322 10047 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:18.151989 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.148429 10056 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:17.994985 12464 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.150743 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:18.081705 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.140899 12486 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.125258 12420 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.151048 12476 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:18.165518 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3060915910002677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:17:18.167079 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:18.167779 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:18.169298 10020 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.171591 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.103926 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.293516 10052 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:18.293654 10202 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:18.154197 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 120995us
W20260709 13:17:18.294461 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2115915910001149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 13:17:18.203224 10125 rpcz_store.cc:276] 0709 13:17:16.727950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.728153 (+   203us) service_pool.cc:224] Handling call
0709 13:17:16.728723 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.989408 (+260685us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:17.042904 (+ 53496us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.382006 (+339102us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.382477 (+   471us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.382673 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:17.382746 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:17.382820 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.136233 (+753413us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b1800316c60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419
0709 13:17:18.137124 (+   891us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:18.137176 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.137382 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26441,"prepare.queue_time_us":131041,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":3980,"raft.queue_time_us":2296815,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8155,"spinlock_wait_cycles":613706368,"thread_start_us":304226,"threads_started":1,"wal-append.queue_time_us":624294}
I20260709 13:17:18.216194 11513 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.295579 10054 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:18.210704 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:18.294889 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1422915910002303s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
I20260709 13:17:18.080928 10050 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:18.210304 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.300298 10050 pending_rounds.cc:85] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 3
I20260709 13:17:18.302238 10050 pending_rounds.cc:107] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:17:18.302748 10050 raft_consensus.cc:2889] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:18.305415 10201 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:18.182368 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:18.196308 12459 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:18.312681 10200 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:18.311666 10048 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:18.300792 10127 rpcz_store.cc:276] 0709 13:17:16.395553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.395783 (+   230us) service_pool.cc:224] Handling call
0709 13:17:16.438240 (+ 42457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.438628 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:17.076768 (+638140us) log.cc:844] Serialized 14 byte log entry
0709 13:17:17.393514 (+316746us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.144615 (+751101us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b180042f040. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:18.144799 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:18.144860 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:18.144912 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.145011 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.145172 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":716455,"prepare.queue_time_us":131210,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":775460,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6783,"spinlock_wait_cycles":45468800,"thread_start_us":874405,"threads_started":2,"wal-append.queue_time_us":1046092}
I20260709 13:17:18.318611 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.319895 10047 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:18.315626 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:18.171618 12378 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.321108 11513 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.323606 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.307204 12463 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:18.203191 12457 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.171025 12298 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:18.401753 12494 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:17:18.318957 12481 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.331248 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:18.331610 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 140) took 1850 ms. Trace:
I20260709 13:17:18.331951 12267 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:17:18.410367 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:18.335091 12411 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.712s	user 0.003s	sys 0.000s
I20260709 13:17:18.336045 12492 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:18.425721 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:18.336021 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 166802us
I20260709 13:17:18.336495 12449 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:18.336953 12365 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:18.337388 12437 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:18.338322 12419 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:18.347059 10897 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:18.350406 10056 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:18.353467 10055 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:18.354950 12476 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.355458 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2530) took 1237 ms. Trace:
I20260709 13:17:18.356046 12201 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.303992 12468 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.359938 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 338) took 1239 ms. Trace:
I20260709 13:17:18.368238 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1436) took 1603 ms. Trace:
I20260709 13:17:18.178778 12448 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:18.376271 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:18.378820 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:18.190400 12412 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.391661 10054 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:17:18.455524 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 124441us
I20260709 13:17:18.392362 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:18.399533 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83810.4us
I20260709 13:17:18.398255 12407 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.403378 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:18.474779 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 82506.8us
I20260709 13:17:18.403929 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:18.477988 10131 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:18.405898 10123 rpcz_store.cc:276] 0709 13:17:16.481046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.481350 (+   304us) service_pool.cc:224] Handling call
0709 13:17:16.499380 (+ 18030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:16.604134 (+104754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.604177 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:16.604246 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:16.604285 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.331311 (+1727026us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54001428d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:18.331371 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.331535 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240341888}
I20260709 13:17:18.191363 12410 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:18.407382 10048 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:18.402689 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:18.500131 10122 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:17:18.413971 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 242208us
I20260709 13:17:18.402508 12481 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:18.431455 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 123321us
W20260709 13:17:18.501775 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.394454 12378 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:18.436654 12464 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:18.434669 12173 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:18.442477 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.426723 10128 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 2.
I20260709 13:17:18.405133 12457 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:18.448798 10200 rpcz_store.cc:276] 0709 13:17:17.118332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.208907 (+1090575us) service_pool.cc:224] Handling call
0709 13:17:18.209308 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.316233 (+106925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.355089 (+ 38856us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b54002df0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:18.355363 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216422272}
I20260709 13:17:18.516676 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "db052bef7536453fa2d7f5a00e314a18"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:18.517621 10200 raft_consensus.cc:606] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Received request to transfer leadership to TS db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:18.518201 10200 raft_consensus.cc:614] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:18.518033 12448 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.493052 12259 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:18.411787 12211 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:18.519519 12259 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.414346 12268 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:18.180059 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:18.430847 12449 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:18.481760 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:18.404414 12463 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.309437 12486 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.497640 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1390) took 2951 ms. Trace:
I20260709 13:17:18.499010 10048 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:18.496934 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:18.447261 12419 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:18.444027 12437 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:18.514106 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:18.511090 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:18.499593 12405 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:17:18.517692  9951 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.454530 10204 rpcz_store.cc:276] 0709 13:17:16.764462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.966129 (+201667us) service_pool.cc:224] Handling call
0709 13:17:17.263360 (+297231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:17.264998 (+  1638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.367950 (+1102952us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54002189d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:18.368143 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682400640}
I20260709 13:17:18.461659 10051 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:18.462949 12468 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.447176 12492 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.606844 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:18.796473 11513 raft_consensus.cc:2764] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:17:18.796448 10898 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:18.797032 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 285976us
W20260709 13:17:18.798026 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 355719us
I20260709 13:17:18.799242 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:18.800098 10123 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.801307 10895 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:18.610716 12276 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:18.591260 10865 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.167s (SENT)
I20260709 13:17:18.806845 12449 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:18.473214 12201 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:18.574679 12411 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:17:18.521534 12410 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.812354 12419 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.813966 12410 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.795197 10130 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:18.807520 10198 rpcz_store.cc:276] 0709 13:17:15.545960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:16.662806 (+1116846us) service_pool.cc:224] Handling call
0709 13:17:16.663372 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:16.671666 (+  8294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:16.671718 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:16.681617 (+  9899us) log.cc:844] Serialized 26 byte log entry
0709 13:17:17.621248 (+939631us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:17.658149 (+ 36901us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.168416 (+510267us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b18003afa80. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:18.476876 (+308460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.476990 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:18.477060 (+    70us) raft_consensus.cc:1685] finished
0709 13:17:18.497322 (+ 20262us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:18.497560 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":921,"prepare.queue_time_us":34132,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":156540,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":9879,"spinlock_wait_cycles":7132544,"thread_start_us":1154622,"threads_started":2,"wal-append.queue_time_us":1006876}
W20260709 13:17:18.707974 12433 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.020s	sys 0.066s
W20260709 13:17:18.836344 12433 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.021s	sys 0.070s
I20260709 13:17:18.510311 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 337) took 1399 ms. Trace:
I20260709 13:17:18.822635 12259 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:18.591485 12464 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.225s
W20260709 13:17:18.757462 10899 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.004s
I20260709 13:17:18.519519 12448 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:18.817083 10125 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:18.735774 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.513942 12404 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.290s
W20260709 13:17:18.521272 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47068.8us
I20260709 13:17:18.806903 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.809818 12201 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.724447 12185 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:18.519834 12211 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:18.879228 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.921739 10048 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:18.509877 12412 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:18.608598 12406 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:18.885771 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.514438 12498 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:18.884138  9951 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.900532 12365 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:18.892992 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 93883.3us
I20260709 13:17:18.808911 12449 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:18.454239 10891 rpcz_store.cc:276] 0709 13:17:17.120845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.182687 (+ 61842us) service_pool.cc:224] Handling call
0709 13:17:18.359521 (+1176834us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54006a0254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:18.359793 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509781760}
I20260709 13:17:18.494153 12407 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:18.901393 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.953691 10124 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:17:18.911259 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:18.924081 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:18.924373 10051 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:17:18.924829 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.021s	sys 0.032s
W20260709 13:17:18.960803 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.021s	sys 0.032s
I20260709 13:17:18.962709 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:18.935891 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:18.966662 10198 raft_consensus.cc:606] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.967340 10198 raft_consensus.cc:614] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:18.814491 12419 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:18.892555 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92931.1us
I20260709 13:17:18.917939 12405 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.005980 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.007370 10048 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:18.969425 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:19.008682 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.923887 12211 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:18.988716 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:18.996168 10123 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:18.936841 12498 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.013948 12498 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:18.939049 12365 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:19.017648 12365 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:19.020243 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.021842  9951 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 5, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:17:19.030679 10201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.013s	sys 0.013s
W20260709 13:17:19.033010 10201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.013s	sys 0.013s
W20260709 13:17:19.041591 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 128861us
I20260709 13:17:19.042160 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:18.937458 12462 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:18.952477 10899 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:19.044468 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:18.949389 12335 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.143512 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.057643 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.066560 12185 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.145121 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 156) took 2027 ms. Trace:
I20260709 13:17:18.931609 12406 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.043618 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:18.882593 10897 rpcz_store.cc:276] 0709 13:17:17.110754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:17.120418 (+  9664us) service_pool.cc:224] Handling call
0709 13:17:17.254158 (+133740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.346812 (+1092654us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b54003555a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa2780061c0 00007fa278007231 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c
0709 13:17:18.454797 (+107985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.499152 (+ 44355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15188864}
I20260709 13:17:19.045714 12463 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:19.126323 10051 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:19.040884 12405 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.070602 10899 pending_rounds.cc:85] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 3
I20260709 13:17:19.111727 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.133198 10898 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:19.063594 12407 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:19.070612 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 541448us
W20260709 13:17:19.142087 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.143715 12413 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.046588  9951 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.063274 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.145648 10203 rpcz_store.cc:276] 0709 13:17:17.117365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.206796 (+1089431us) service_pool.cc:224] Handling call
0709 13:17:18.210500 (+  3704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.240024 (+ 29524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.240069 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.297978 (+ 57909us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.717941 (+419963us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:18.755422 (+ 37481us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.773306 (+ 17884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.773401 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.144846 (+371445us) raft_consensus.cc:1685] finished
0709 13:17:19.144875 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.145038 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10287,"prepare.queue_time_us":176275,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":46137,"raft.run_cpu_time_us":516,"raft.run_wall_time_us":55849,"thread_start_us":394757,"threads_started":1,"wal-append.queue_time_us":638724}
W20260709 13:17:19.142084 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 97772.6us
I20260709 13:17:19.047525 12435 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.145577 10125 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:19.172065 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:19.273725 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.558s (SENT) [suppressed 42 similar messages]
W20260709 13:17:19.288218 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 126922us
W20260709 13:17:19.231833 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.173143 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1183) took 1058 ms. Trace:
I20260709 13:17:19.293802 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:19.196874 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.297722 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.296553 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:19.294456 10055 rpcz_store.cc:276] 0709 13:17:18.114764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.151702 (+ 36938us) service_pool.cc:224] Handling call
0709 13:17:19.172787 (+1021085us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b54003cfad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:19.173021 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137983744}
I20260709 13:17:19.297163  9951 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:17:19.293442 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 121478us
W20260709 13:17:19.303736 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.305097 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:19.302610 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:19.298040 10199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.020s	sys 0.045s
I20260709 13:17:19.307773 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:19.309474 10199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.020s	sys 0.046s
I20260709 13:17:19.309710 10053 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:19.312745 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.312734 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 157) took 1397 ms. Trace:
I20260709 13:17:19.313625 10053 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:19.316355 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.305639 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.181869 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2546) took 1788 ms. Trace:
I20260709 13:17:19.316572 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:19.320132 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:19.322585 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.323645 10125 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:19.326844 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.327665 10124 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
W20260709 13:17:19.194278 12496 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.674s	user 0.005s	sys 0.015s
I20260709 13:17:19.329221 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.191557 12486 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.330380 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.313807 10199 rpcz_store.cc:276] 0709 13:17:17.915312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.297162 (+381850us) service_pool.cc:224] Handling call
0709 13:17:18.313850 (+ 16688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.314101 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:18.693967 (+379866us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.310474 (+616507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.310591 (+   117us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:19.312128 (+  1537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.312345 (+   217us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.312594 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7812,"prepare.queue_time_us":84781,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":11231,"raft.queue_time_us":179,"thread_start_us":934055,"threads_started":2}
I20260709 13:17:19.343384 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:19.318801 10196 rpcz_store.cc:276] 0709 13:17:17.393678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.210120 (+816442us) service_pool.cc:224] Handling call
0709 13:17:18.259275 (+ 49155us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:18.261069 (+  1794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.261122 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:18.377324 (+116202us) log.cc:844] Serialized 52 byte log entry
0709 13:17:18.785697 (+408373us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:18.785776 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.785900 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.181579 (+395679us) raft_consensus.cc:1685] finished
0709 13:17:19.181609 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.181791 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1064,"raft.queue_time_us":5437,"spinlock_wait_cycles":30869248,"thread_start_us":382811,"threads_started":1,"wal-append.queue_time_us":657357}
I20260709 13:17:19.330569 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.317214 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:19.345607 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.346026 10055 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:19.334745 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.346451 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.307339 11702 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:19.163800 12468 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:19.184830 12492 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:19.334525  9951 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:19.192828 12412 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:19.300282 10899 pending_rounds.cc:107] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:17:19.348320 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.349888 10053 raft_consensus.cc:2410] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:17:19.349445 10052 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.009s	sys 0.023s
I20260709 13:17:19.149178 12462 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.375751 10053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:19.352950 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.349401 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:19.300387 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.390532 12464 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:19.391036 12464 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.391448 12464 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:19.350587 11702 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.365692 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:19.174078 12335 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.174s
I20260709 13:17:19.196293 12259 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.335240 12420 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:19.372536 12498 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:19.334992 12019 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.335961 12439 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:17:19.339257 10892 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.016s	sys 0.052s
I20260709 13:17:19.338510 12463 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:19.372609 10052 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.010s	sys 0.023s
I20260709 13:17:19.335312 12407 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:19.376026 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.382820  9951 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.384311 10053 raft_consensus.cc:606] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:19.460978 10053 raft_consensus.cc:614] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:19.340891 12435 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:19.365315 10056 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:19.398295 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 354760us
I20260709 13:17:19.399048 11702 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:19.399906 12502 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:17:19.358878 12404 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:19.477119 12404 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.477730 12404 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:19.479538 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1186) took 1293 ms. Trace:
I20260709 13:17:19.368014 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:19.480727 12259 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.481437 10048 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:19.481762 12259 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.372093 10899 raft_consensus.cc:2889] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:19.482370 12259 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:19.417487 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51951.9us
I20260709 13:17:19.483523 12462 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:19.483814 12406 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.484355 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:19.462915 10892 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.017s	sys 0.052s
I20260709 13:17:19.484275 11702 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:19.413432 12413 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.486232 12413 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.487169 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 443) took 1306 ms. Trace:
I20260709 13:17:19.487618 10899 rpcz_store.cc:276] 0709 13:17:18.180827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.182558 (+  1731us) service_pool.cc:224] Handling call
0709 13:17:18.183037 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.486834 (+1303797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.487066 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:19.418412 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.453739 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1178) took 1387 ms. Trace:
I20260709 13:17:19.490311 10049 rpcz_store.cc:276] 0709 13:17:18.066298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.067253 (+   955us) service_pool.cc:224] Handling call
0709 13:17:18.067806 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.068301 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.068332 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.068847 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.277536 (+208689us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:18.277590 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.277678 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.193463 (+915785us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:19.420582 (+227119us) raft_consensus.cc:1685] finished
0709 13:17:19.420611 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.453605 (+ 32994us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":319,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":24250,"spinlock_wait_cycles":41398528,"thread_start_us":165917,"threads_started":1,"wal-append.queue_time_us":303013}
I20260709 13:17:19.407770 12185 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:19.470218 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.359475 12486 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:17:19.416816 12498 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.467199 12439 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:19.472826  9951 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 4, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.484738 12405 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.468863 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:19.518060 10056 raft_consensus.cc:2410] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:19.460018 12420 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:19.503466 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:19.503728 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.480062 10047 rpcz_store.cc:276] 0709 13:17:18.186578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.323447 (+136869us) service_pool.cc:224] Handling call
0709 13:17:18.324006 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.335887 (+ 11881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.335923 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.336592 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.757357 (+420765us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:18.772926 (+ 15569us) log.cc:844] Serialized 14 byte log entry
0709 13:17:18.886624 (+113698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:18.886732 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.479242 (+592510us) raft_consensus.cc:1685] finished
0709 13:17:19.479273 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.479463 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124545,"prepare.queue_time_us":76275,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":137412,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":29267,"thread_start_us":420244,"threads_started":1,"wal-append.queue_time_us":823373}
I20260709 13:17:19.504870 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:19.489343 12449 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.002s
I20260709 13:17:19.509976 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.459890 12463 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:19.460170 12407 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:19.503093 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.487010 11702 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:19.511226 12413 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.511436 12185 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:19.484652 12406 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:17:19.529671 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.531936 10197 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:19.515024 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.550081  9951 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.552625 12439 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:19.555976 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1194) took 1366 ms. Trace:
I20260709 13:17:19.626456 10054 rpcz_store.cc:276] 0709 13:17:18.189404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.478234 (+288830us) service_pool.cc:224] Handling call
0709 13:17:18.516940 (+ 38706us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.828664 (+311724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.828697 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.872477 (+ 43780us) log.cc:844] Serialized 26 byte log entry
0709 13:17:19.360918 (+488441us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:19.360968 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.361072 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:19.555657 (+194585us) raft_consensus.cc:1685] finished
0709 13:17:19.555685 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.555889 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18272,"raft.queue_time_us":312379,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":72709,"thread_start_us":450411,"threads_started":1,"wal-append.queue_time_us":660998}
W20260709 13:17:19.629781 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 241031us
I20260709 13:17:19.631681 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:19.632437 10127 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:19.633589 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.637715 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.639197 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.639875 10197 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:19.640816 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.642875 10197 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:19.558494 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.517735 12498 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:19.657045 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:19.659852 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.660687 10054 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:19.661176 10124 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:19.565025 10047 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:19.569146 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 273896us
W20260709 13:17:19.565728 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 292044us
W20260709 13:17:19.556020 12481 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.984s	user 0.010s	sys 0.005s
I20260709 13:17:19.534660 10203 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:19.533622 12463 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:19.562728 12211 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.001s
W20260709 13:17:19.616135 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 146043us
I20260709 13:17:19.678328 12259 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.635070 12413 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.679332 12405 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.679797 12413 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.682802 10205 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:19.684144 12413 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:19.662940 12457 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.072s	user 0.015s	sys 0.000s
I20260709 13:17:19.695539 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.696485 10203 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:19.698024 10127 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:19.697371 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:19.698132 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 212257us
I20260709 13:17:19.556849 12019 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:19.623476 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108605us
I20260709 13:17:19.630518  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:19.713128 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.548842 10198 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:19.611737 12407 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:19.618281 12378 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.989s	user 0.014s	sys 0.002s
I20260709 13:17:19.650699 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.653731 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.625180 12185 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:19.666247 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.665132 10047 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:19.519157 12435 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:17:19.700863 10054 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:19.706679 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:19.746285 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.571540 12420 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.679399 12259 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.719980 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:19.734066 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57875.6us
I20260709 13:17:19.755750 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.710741  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:19.740140 10199 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:19.750916 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:19.756922 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.699571  9951 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: config changed from index 3 to -1, term changed from 2 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65), NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) evicted, now has a pending config: VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65), VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66), VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67), NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:19.761430 10198 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:19.765774 10200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.158s	user 0.057s	sys 0.132s
W20260709 13:17:19.766207 10200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.058s	sys 0.132s
I20260709 13:17:19.691372 12481 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:19.698372 12211 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.765048  9951 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.702137 12457 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:19.802695 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47097.4us
I20260709 13:17:19.806089 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:19.767168 12365 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.001s
I20260709 13:17:19.739909 12378 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:19.762077 12523 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:19.855825 10131 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:19.759979 12019 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:17:19.859691 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53711.1us
I20260709 13:17:19.762557 12259 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.761312 12420 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.863250  9951 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:19.854367 12165 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.841526 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 446) took 1106 ms. Trace:
W20260709 13:17:19.889338 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 223642us
I20260709 13:17:19.873502 12365 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:19.870730 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 157758us
W20260709 13:17:19.868767 12521 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
W20260709 13:17:19.861356 12508 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:17:19.894908 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 158) took 1979 ms. Trace:
W20260709 13:17:19.883713 12520 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.116s	user 0.001s	sys 0.001s
I20260709 13:17:19.870231 12384 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:19.888185 12522 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.109s	user 0.001s	sys 0.001s
I20260709 13:17:19.898618 11513 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:19.898049 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 228922us
I20260709 13:17:19.898650 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 163) took 1524 ms. Trace:
W20260709 13:17:19.900295 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:19.910395  9951 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:19.914882 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:19.916432 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:19.918020 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 153958us
I20260709 13:17:19.918207  9951 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.899178 12497 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:19.898828 12536 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:19.935873  9951 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 2, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 2 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:19.896889 12439 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:17:19.893779 12165 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.919399 12019 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:19.923233 12533 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.949900  9951 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 3, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:17:19.939011 12512 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.264s	user 0.000s	sys 0.002s
W20260709 13:17:19.957703 12530 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:17:19.960366 12457 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:19.960870 12185 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:19.963774 12529 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:17:19.964813 12185 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:19.965226 12419 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.958s	user 0.013s	sys 0.001s
I20260709 13:17:19.888141 10892 rpcz_store.cc:276] 0709 13:17:18.726975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.755925 (+ 28950us) service_pool.cc:224] Handling call
0709 13:17:18.770554 (+ 14629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.780075 (+  9521us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:19.485451 (+705376us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.765903 (+280452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.778178 (+ 12275us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:19.784181 (+  6003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.807818 (+ 23637us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.833138 (+ 25320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19270,"prepare.queue_time_us":774965,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":145627,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3415,"thread_start_us":870043,"threads_started":2,"wal-append.queue_time_us":398004}
I20260709 13:17:19.969235 12165 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:19.942739 12384 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:19.971830 12525 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:17:19.934859 12435 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:19.975633 12435 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.978387 12435 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:19.980088 12532 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:19.982774 12365 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:19.944077 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:19.984098 10899 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:19.989259 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72904.1us
I20260709 13:17:19.991761 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.993335 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:19.952710  9951 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 5, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:17:19.953714 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54291.4us
W20260709 13:17:19.965364 12527 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:17:19.995837  9951 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:19.958305 12404 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:19.994418 12533 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.000260 12468 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:20.000447 12268 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:19.999362 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:20.002022 12201 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:20.009428 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 44980.1us
W20260709 13:17:20.004262 12508 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:17:20.014619 10204 raft_consensus.cc:2368] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:20.014654 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:20.026754 11702 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:20.027683 11702 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.016388 12211 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.041792 12420 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.002022 12259 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.047964 12378 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.051165 12533 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.054023 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 162) took 1712 ms. Trace:
I20260709 13:17:20.054659 10201 rpcz_store.cc:276] 0709 13:17:18.341613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.341834 (+   221us) service_pool.cc:224] Handling call
0709 13:17:18.342256 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.408892 (+ 66636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.408948 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.409806 (+   858us) log.cc:844] Serialized 26 byte log entry
0709 13:17:19.034714 (+624908us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:19.106767 (+ 72053us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.115535 (+  8768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.115645 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.053615 (+937970us) raft_consensus.cc:1685] finished
0709 13:17:20.053655 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.053907 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88477,"prepare.queue_time_us":74069,"prepare.run_cpu_time_us":1610,"prepare.run_wall_time_us":24183,"raft.queue_time_us":84142,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":12059,"thread_start_us":585068,"threads_started":1,"wal-append.queue_time_us":1216846}
I20260709 13:17:20.056486 10198 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:20.071909 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.076634 12404 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:20.023803 12464 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.004s
I20260709 13:17:20.080287 12464 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:20.085110 12448 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.177s	user 0.016s	sys 0.000s
I20260709 13:17:20.086633 10130 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:20.099972 12464 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.010905 12498 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.122320 12448 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:20.127662 11702 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.015486 12185 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:20.151605 12267 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:20.158912 12448 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.163410 12185 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:20.163523 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 148937us
I20260709 13:17:20.165922 12185 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.166592 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:20.163146 12448 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:20.163596 12463 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.163205 12407 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.174934 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:20.174669 12267 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:20.172202 12384 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:20.184358 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:20.177977 12448 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:20.195499 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:20.208637 12407 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.200001 12378 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:17:20.219527 12481 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:20.224517 12514 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.714s	user 0.006s	sys 0.002s
I20260709 13:17:20.225852 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 150) took 1091 ms. Trace:
I20260709 13:17:20.234987 10131 rpcz_store.cc:276] 0709 13:17:19.134040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.134596 (+   556us) service_pool.cc:224] Handling call
0709 13:17:20.225498 (+1090902us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540014c154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:20.225752 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585069312}
I20260709 13:17:20.236914 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.237900 10131 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.248647 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1191) took 2060 ms. Trace:
I20260709 13:17:20.229180 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.250010 10127 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.229386 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.229892 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.231873 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:19.895697 10202 rpcz_store.cc:276] 0709 13:17:17.915619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.298933 (+383314us) service_pool.cc:224] Handling call
0709 13:17:18.299372 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.299612 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:18.322238 (+ 22626us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.564822 (+1242584us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b18001555c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:17:19.894458 (+329636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.894526 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:19.894582 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.894672 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.894834 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55069,"prepare.queue_time_us":22264,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":91494,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":17383,"spinlock_wait_cycles":132415872,"thread_start_us":329130,"threads_started":2,"wal-append.queue_time_us":1579867}
I20260709 13:17:20.232110 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.227048 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.227315 10125 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:20.249297 10050 rpcz_store.cc:276] 0709 13:17:18.188495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.380796 (+192301us) service_pool.cc:224] Handling call
0709 13:17:18.414859 (+ 34063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.444222 (+ 29363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.444282 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.466117 (+ 21835us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.955161 (+489044us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:19.034457 (+ 79296us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.247797 (+1213340us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18000b8840. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:20.248100 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.248231 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.248282 (+    51us) raft_consensus.cc:1685] finished
0709 13:17:20.248388 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.248568 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10264,"prepare.queue_time_us":86814,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4879,"raft.queue_time_us":1229515,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":28651,"spinlock_wait_cycles":3959040,"thread_start_us":380772,"threads_started":1,"wal-append.queue_time_us":1072252}
I20260709 13:17:20.198725 12268 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:20.276467 10123 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:20.260957 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:20.201380 12384 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:20.281965 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:20.288617 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:20.280251 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84834.9us
I20260709 13:17:20.252507 11513 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:20.291035 11513 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.287417 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.292052 10123 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:20.288232 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.250772 12449 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:20.206166 10899 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:20.298002 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.285773 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:20.256287 12498 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.257750 12463 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.296048 10127 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:20.283620 12481 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.252718 12483 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.312173 10131 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:20.287055 12282 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:20.306924 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:20.324456 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.325505 10050 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.317034 10123 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:20.287093 12365 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:20.299999 12548 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:17:20.316808 11513 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.290362 12268 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:20.294988 12449 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:20.299937 12435 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.351132 11513 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:20.353308 12513 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.824s	user 0.003s	sys 0.014s
I20260709 13:17:20.346513 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:20.353901 12268 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.357091 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:20.337781 12533 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:20.356647 12449 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.347223 12523 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:17:20.359616 10891 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:20.360543 12378 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:17:20.363902 12509 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.853s	user 0.007s	sys 0.003s
I20260709 13:17:20.337354 12365 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.338445 12482 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.319041 11702 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:20.313894 12201 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:20.332571 12546 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:20.396967 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 154) took 1457 ms. Trace:
W20260709 13:17:20.400014 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47143.1us
I20260709 13:17:20.326900 12481 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.395757 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1190) took 2207 ms. Trace:
W20260709 13:17:20.416720 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50464.3us
W20260709 13:17:20.416777 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47729.6us
I20260709 13:17:20.434394 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:20.435169 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:20.428532 10052 rpcz_store.cc:276] 0709 13:17:18.188150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.351594 (+163444us) service_pool.cc:224] Handling call
0709 13:17:18.352187 (+   593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:18.352799 (+   612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:18.352827 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:18.353451 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:17:18.757946 (+404495us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:19.459829 (+701883us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.395325 (+935496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.395429 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.395477 (+    48us) raft_consensus.cc:1685] finished
0709 13:17:20.395490 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.395666 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":814792,"prepare.queue_time_us":77681,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":834958,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"spinlock_wait_cycles":10769152,"thread_start_us":1164382,"threads_started":3,"wal-append.queue_time_us":808042}
I20260709 13:17:20.422412 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:20.401711 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2414334459999736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:17:20.425973 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1430) took 1542 ms. Trace:
I20260709 13:17:20.333293 12483 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.351799 12435 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.380821 12498 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:17:20.418623 12482 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.418080 11702 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.445204 12448 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:20.436502 12384 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:17:20.455780 12559 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.456542 10049 rpcz_store.cc:276] 0709 13:17:18.884130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.498595 (+614465us) service_pool.cc:224] Handling call
0709 13:17:19.499047 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.257786 (+758739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.257847 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.258552 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.396338 (+137786us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:20.396447 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.396541 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.425624 (+ 29083us) raft_consensus.cc:1685] finished
0709 13:17:20.425658 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.425883 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":761011,"raft.queue_time_us":790769,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5621,"thread_start_us":135074,"threads_started":1,"wal-append.queue_time_us":163590}
I20260709 13:17:20.437302 12449 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.433583 12533 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:20.440490 12456 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.439991 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:20.439996 12267 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:17:20.485819 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:20.456705 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2288334459999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:17:20.511494 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:20.546505 10020 connection.cc:256] RPC call timeout handler was delayed by -1.170333446000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 13:17:20.547075 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1631334460000744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 13:17:20.519999 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.434268 12463 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:17:19.907034 10200 rpcz_store.cc:276] 0709 13:17:18.374756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.522457 (+147701us) service_pool.cc:224] Handling call
0709 13:17:18.523046 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.523372 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:18.607412 (+ 84040us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.898225 (+1290813us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.898292 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:19.898350 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:19.898445 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:19.898584 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13538,"prepare.queue_time_us":105402,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":122348,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":6340,"spinlock_wait_cycles":8060672,"thread_start_us":1276633,"threads_started":2,"wal-append.queue_time_us":902012}
W20260709 13:17:20.548756 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1253334460000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:17:20.549338 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1127334459999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
W20260709 13:17:20.549829 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1082334459999856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 13:17:20.418512 10056 rpcz_store.cc:276] 0709 13:17:18.939844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.520226 (+580382us) service_pool.cc:224] Handling call
0709 13:17:19.521033 (+   807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.534269 (+ 13236us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:19.568913 (+ 34644us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.394230 (+825317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:20.394311 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:20.394369 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.396658 (+  2289us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.396857 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":423136,"prepare.queue_time_us":13269,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":76,"raft.queue_time_us":484748,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":16266,"thread_start_us":304335,"threads_started":2,"wal-append.queue_time_us":291713}
I20260709 13:17:20.477170 11702 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:20.550230 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1064334459999827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 13:17:20.550743 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0893334459999551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:17:20.556824 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.438452 12201 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.558187 10056 raft_consensus.cc:2410] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:20.454471 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1428) took 1673 ms. Trace:
I20260709 13:17:20.526332 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.559043 10049 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:20.558835 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73092us
W20260709 13:17:20.557993 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:20.560703 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:20.517891 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 118005us
W20260709 13:17:20.551263 10020 connection.cc:256] RPC call timeout handler was delayed by -1.037133446000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 13:17:20.563743 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0308334460000879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
W20260709 13:17:20.564821 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0299334460000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
I20260709 13:17:20.477280 12526 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:20.436017 12546 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.480064 12561 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:20.568151 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0218334460000733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
I20260709 13:17:20.481101 12482 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.560496 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:20.573458 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.488437 12448 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:20.442991 12437 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:20.579655 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0164334460000646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:17:20.587941 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:20.563053 10055 rpcz_store.cc:276] 0709 13:17:18.781363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.460955 (+679592us) service_pool.cc:224] Handling call
0709 13:17:19.473129 (+ 12174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.473409 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:19.473770 (+   361us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.290618 (+816848us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.453924 (+163306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:20.454035 (+   111us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:20.454109 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.454217 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.454392 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":654383,"prepare.queue_time_us":9958,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":873843,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":27014,"spinlock_wait_cycles":13013376,"thread_start_us":352717,"threads_started":2,"wal-append.queue_time_us":329436}
I20260709 13:17:20.585556 12259 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:20.590359 12259 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:20.591035 12554 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:17:20.591389 12259 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.589491 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.597146 10055 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:17:20.629251 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.480839 12468 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
W20260709 13:17:20.640082 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 169644us
I20260709 13:17:20.559759 12463 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:20.573084 12526 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:20.581764 12419 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:20.589397 12448 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:20.661932 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.656962 10050 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:20.557711 12378 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:17:20.669124 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.675407 10055 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:20.678285 10050 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:20.691392 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.692575 10049 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:20.693169 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.656095 12463 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:20.716488 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.704007 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:20.683001 12540 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:17:20.694136 12419 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:20.725726 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.660029 12526 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:20.723896 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.545939 12541 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.234s
I20260709 13:17:20.742657 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.727566 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.750068 10049 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:20.741483 10050 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:20.818912 12511 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.411s	user 0.003s	sys 0.009s
I20260709 13:17:20.823102 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.825277 10050 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:20.823098 10055 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:20.828058 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:20.829206 10055 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:20.665880 12533 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.834667 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.839986 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:20.840656 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 194603us
I20260709 13:17:20.767602 12481 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.840860 10055 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:20.847288 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.848073 10055 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.850013 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1195) took 2444 ms. Trace:
I20260709 13:17:20.836882 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:20.850888 10050 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:20.855443 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 202438us
I20260709 13:17:20.544006 12559 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:20.837692 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 237688us
I20260709 13:17:20.588798 12561 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.832530 12365 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.857511 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.876989 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:20.877918 10126 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:20.858350 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.858884 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.880535 10123 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.857072 12267 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:20.837495 10049 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:20.841167 12529 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:17:20.835829 12483 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:20.850504 10122 rpcz_store.cc:276] 0709 13:17:18.400492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:18.523300 (+122808us) service_pool.cc:224] Handling call
0709 13:17:18.607818 (+ 84518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:18.881205 (+273387us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:19.345381 (+464176us) log.cc:844] Serialized 14 byte log entry
0709 13:17:19.571684 (+226303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:19.571771 (+    87us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:19.571842 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.026800 (+454958us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180043f600. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419
0709 13:17:20.844391 (+817591us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54001275d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:20.844465 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.844652 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":186103,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":147,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":43625,"spinlock_wait_cycles":560419968,"thread_start_us":675819,"threads_started":2,"wal-append.queue_time_us":415652}
I20260709 13:17:20.859040 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.880548 10131 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:20.849011 12435 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:20.558099 12456 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:20.857184 12419 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.832000 12463 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:20.984299 12546 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.000815 12267 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:20.933876 12569 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:17:20.955991 12570 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:17:21.003988 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:21.004129 12267 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.009325 12463 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:20.995450 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.011415 12456 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.011668 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.012735 10127 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.005364 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.006278 10898 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:21.008284 10129 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.990209 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1431) took 2093 ms. Trace:
I20260709 13:17:21.014532 10049 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:20.947271 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:20.949355 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.999388 12585 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.014845 10053 rpcz_store.cc:276] 0709 13:17:18.897073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.517605 (+620532us) service_pool.cc:224] Handling call
0709 13:17:19.518170 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.474795 (+956625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.474828 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.475443 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.837909 (+362466us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:20.837954 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.838037 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:20.989857 (+151820us) raft_consensus.cc:1685] finished
0709 13:17:20.989887 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:20.990115 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":741780,"raft.queue_time_us":791862,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":14408,"thread_start_us":453445,"threads_started":2,"wal-append.queue_time_us":332870}
I20260709 13:17:20.890590 10050 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:20.962291 12544 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.013661 10055 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:21.019062 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.019800 10055 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
W20260709 13:17:21.020000 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:20.954741 11702 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.006228 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:20.948287  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:21.017221 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.017463 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.028183 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:21.028949 10053 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:20.885818 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.020860 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.030615 10055 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:21.031785 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.032684 10055 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:21.034420 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 171) took 1620 ms. Trace:
I20260709 13:17:21.017926 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:21.035001 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.035770 12201 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.036532 10050 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:21.036701 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.036775 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.037300 10122 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.037909 10124 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:21.040110 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1207) took 1408 ms. Trace:
I20260709 13:17:20.954111 12559 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.040598 10130 rpcz_store.cc:276] 0709 13:17:19.632045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.632244 (+   199us) service_pool.cc:224] Handling call
0709 13:17:21.039928 (+1407684us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54001f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:21.040043 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846060800}
I20260709 13:17:21.041723 12497 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:21.042349 12497 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:21.016059 12568 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.171s	user 0.001s	sys 0.001s
I20260709 13:17:21.045012 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.046820 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.047695 10200 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:21.034793 10055 rpcz_store.cc:276] 0709 13:17:19.413626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.033365 (+1619739us) service_pool.cc:224] Handling call
0709 13:17:21.033759 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.034013 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:21.034047 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.034105 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:21.034145 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.034207 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.034346 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:21.042156 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.022164 10123 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:21.056464 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.063233 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:21.085686 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.086923 10049 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:21.040838 11702 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:21.038885 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.099740 10124 raft_consensus.cc:2368] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:21.112141 12584 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:21.112854 12584 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.113240 12584 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.121978 12173 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:21.122778 12173 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:21.018301 12544 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:21.128154 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:21.128345 12544 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.124155 12173 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.132949 12435 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.133752 12435 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:21.136507 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.138561 10052 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:17:21.131335 12173 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.023236 12585 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.149596 12585 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.150665 10897 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:21.156548 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.157320 10898 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:21.157254 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.164004 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.164973 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.174185 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.177750 10131 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:21.182626 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.179940 10898 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:21.159025 10129 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:21.181103 10127 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:21.179672 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.189324 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:21.184693 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.171514 12421 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:21.191694 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.191946 12421 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:21.192432 10130 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.192842 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.193022 12421 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.194020 12533 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:21.172055 12487 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.195220 12533 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:21.196746 12449 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.011s
I20260709 13:17:21.196455 12533 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:21.197978 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.200060 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.204018 10127 raft_consensus.cc:1217] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-5.6]   Dedup: 5.6->[]
I20260709 13:17:21.204819 10127 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:17:21.206568 12561 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.208508 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.206396 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.210496 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.185556 12542 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:21.202951 12497 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:21.214522 12571 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.507s	user 0.002s	sys 0.001s
I20260709 13:17:21.215787 12497 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:21.216486 12555 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.522s	user 0.000s	sys 0.001s
W20260709 13:17:21.216657 12448 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.516s	user 0.016s	sys 0.005s
I20260709 13:17:21.216564 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.217345 12448 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.218745 10129 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:21.219570 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1429) took 2389 ms. Trace:
I20260709 13:17:21.220047 10048 rpcz_store.cc:276] 0709 13:17:18.830091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.482203 (+652112us) service_pool.cc:224] Handling call
0709 13:17:19.482622 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.256315 (+773693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.256348 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.256957 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.446011 (+189054us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:20.446077 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.446197 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:21.215726 (+769529us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:21.219229 (+  3503us) raft_consensus.cc:1685] finished
0709 13:17:21.219252 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.219494 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":773247,"raft.queue_time_us":806735,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6382,"spinlock_wait_cycles":59155200,"thread_start_us":188584,"threads_started":1,"wal-append.queue_time_us":321113}
I20260709 13:17:21.221205 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:21.222399 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.223858 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.213126 10124 raft_consensus.cc:2376] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5. Re-sending same reply.
I20260709 13:17:21.225692 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.226747 10124 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:21.228293 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:21.228667 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.818058 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.818992 10202 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:21.661567 12419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.031s	sys 0.024s
W20260709 13:17:21.348665 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.795382 12406 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:21.242904 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 202796us
I20260709 13:17:21.288564 12437 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.824731 12406 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.360522 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1425) took 2652 ms. Trace:
W20260709 13:17:21.445007 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.836829 12476 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:21.837394 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.558s (SENT)
I20260709 13:17:21.837568 12476 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.838238 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.839905 10200 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:21.839028 10051 rpcz_store.cc:276] 0709 13:17:18.708069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.389766 (+681697us) service_pool.cc:224] Handling call
0709 13:17:19.390098 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:20.394491 (+1004393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.394553 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:20.395572 (+  1019us) log.cc:844] Serialized 26 byte log entry
0709 13:17:20.564154 (+168582us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:20.564193 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:20.564266 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:21.213774 (+649508us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:21.360028 (+146254us) raft_consensus.cc:1685] finished
0709 13:17:21.360062 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.360423 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":875996,"raft.queue_time_us":899974,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":6928,"spinlock_wait_cycles":35381504,"thread_start_us":256557,"threads_started":2,"wal-append.queue_time_us":243302}
W20260709 13:17:21.357679 12526 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.583s	user 0.019s	sys 0.011s
I20260709 13:17:21.841207 12526 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.842151 12404 raft_consensus.cc:1064] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:17:21.579041 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.843194 12546 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:21.844008 12546 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.844785 12463 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:21.845417 12463 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.846204 12546 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.847314 12583 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.847565 12546 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.847574 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.848229 12583 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.848766 10125 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:21.850252 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.850622 12583 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:21.811323 10096 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.012s	sys 0.035s
W20260709 13:17:21.853356 10096 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.012s	sys 0.035s
W20260709 13:17:21.853682 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 635620us
I20260709 13:17:21.855187 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.857700 10051 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:21.827983 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:21.190418 12544 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:21.444121 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.820865 12419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.034s	sys 0.029s
I20260709 13:17:21.830138 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:21.877592 10204 raft_consensus.cc:2376] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7. Re-sending same reply.
I20260709 13:17:21.872296 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.879333 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.830366 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1198) took 2706 ms. Trace:
I20260709 13:17:21.830659 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:21.831215 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.24328e+06us
I20260709 13:17:21.881245 10198 raft_consensus.cc:2376] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4. Re-sending same reply.
I20260709 13:17:21.881575 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.832019 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.832983 12558 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.834200 12165 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.889140 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:21.364607 10127 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
W20260709 13:17:21.678404 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.249663 10122 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:21.891165 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.891460 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2559) took 1032 ms. Trace:
I20260709 13:17:21.891812 10122 rpcz_store.cc:276] 0709 13:17:20.859010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.036356 (+177346us) service_pool.cc:224] Handling call
0709 13:17:21.891382 (+855026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:21.834722 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 169) took 2487 ms. Trace:
I20260709 13:17:21.861299 10131 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:17:21.861449 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.230243 10053 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:21.894636 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1446) took 1638 ms. Trace:
I20260709 13:17:21.256413 10130 raft_consensus.cc:2410] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:21.895001 10053 rpcz_store.cc:276] 0709 13:17:20.255892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.027799 (+771907us) service_pool.cc:224] Handling call
0709 13:17:21.894554 (+866755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:21.895558 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2564) took 1024 ms. Trace:
W20260709 13:17:21.533319 12482 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.897s	user 0.007s	sys 0.007s
I20260709 13:17:21.895929 10130 rpcz_store.cc:276] 0709 13:17:20.870914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.188643 (+317729us) service_pool.cc:224] Handling call
0709 13:17:21.895486 (+706843us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:21.889394 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:21.898506 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1443) took 1643 ms. Trace:
I20260709 13:17:21.902297 10050 rpcz_store.cc:276] 0709 13:17:20.254949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.051847 (+796898us) service_pool.cc:224] Handling call
0709 13:17:21.087611 (+ 35764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.087843 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:21.101523 (+ 13680us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.890803 (+789280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.890862 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:21.890917 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.890986 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.898437 (+  7451us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":634883,"prepare.queue_time_us":23388,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":637090,"thread_start_us":153769,"threads_started":1}
I20260709 13:17:21.904508 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:21.852474 12462 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:21.907624 10200 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:21.908272 10130 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:21.892933 10054 rpcz_store.cc:276] 0709 13:17:19.347860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.752041 (+404181us) service_pool.cc:224] Handling call
0709 13:17:19.754885 (+  2844us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:20.394748 (+639863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:20.394785 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:20.395572 (+   787us) log.cc:844] Serialized 52 byte log entry
0709 13:17:20.642600 (+247028us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:20.845056 (+202456us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.834062 (+989006us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.834288 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.834403 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:21.834458 (+    55us) raft_consensus.cc:1685] finished
0709 13:17:21.834473 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.834647 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1514777,"prepare.queue_time_us":84992,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":127,"raft.queue_time_us":1556252,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":10483,"spinlock_wait_cycles":18176,"thread_start_us":473734,"threads_started":3,"wal-append.queue_time_us":343200}
I20260709 13:17:21.911028 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.912245 10205 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:21.337445 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.368186 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:21.918181 10898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.022s	sys 0.036s
I20260709 13:17:21.913897 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:21.341334 12435 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:21.915309 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:21.918651 10898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.022s	sys 0.036s
I20260709 13:17:21.866075 12583 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:21.874536 12481 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.880831 10047 rpcz_store.cc:276] 0709 13:17:19.123648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.733153 (+609505us) service_pool.cc:224] Handling call
0709 13:17:19.733516 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:19.733771 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:19.750796 (+ 17025us) log.cc:844] Serialized 14 byte log entry
0709 13:17:20.544201 (+793405us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.829670 (+1285469us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b180028fd80. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:21.829898 (+   228us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.829967 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:21.830035 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.830126 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.830282 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":488621,"prepare.queue_time_us":216982,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":114,"raft.queue_time_us":1748139,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":21245,"spinlock_wait_cycles":87654784,"thread_start_us":389892,"threads_started":3,"wal-append.queue_time_us":204284,"wal-append.run_cpu_time_us":3868,"wal-append.run_wall_time_us":1754964}
I20260709 13:17:21.920456 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.921769 10198 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:21.884891 12463 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.926117 10125 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 3.
I20260709 13:17:21.928795 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.927734 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:21.929827 10200 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:21.930706 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:21.927265 10055 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:21.914142 12560 raft_consensus.cc:2764] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:17:21.935655 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.900233 12419 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:21.943332 10900 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.947062 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:21.919783 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:21.951144 10122 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:21.889026 12165 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.900355 12421 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:17:21.966176 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.927812 12406 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:21.914673 12462 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.942330 12483 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:21.961598 12419 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.913640 12482 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.977468 12419 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:21.973889 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.978259 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1228) took 1432 ms. Trace:
I20260709 13:17:21.974841 12561 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:17:21.975415 12590 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:17:21.979770 10196 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:21.941629 12448 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:21.983530 10205 raft_consensus.cc:2410] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:21.977679 12482 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:21.960981 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:21.932585 12481 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:21.931584 12583 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:21.996508 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:21.981348 12560 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:21.988734 12173 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.003s
I20260709 13:17:21.998576 10894 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.998778 12448 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:21.977772 12404 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:22.045740 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.990988 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:22.000473 12201 raft_consensus.cc:2764] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:17:22.046602 12584 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.930s	user 0.017s	sys 0.000s
I20260709 13:17:22.069558 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:22.005196 12463 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:22.048801 10892 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.850s	user 0.011s	sys 0.002s
I20260709 13:17:22.067420 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:22.078337 10053 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.049579 10899 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:22.061715 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.087864 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.088719 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.089816 10051 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:17:22.090256 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.091924 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.52799e+06us
I20260709 13:17:22.089582 10054 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:22.096392 10095 connection.cc:256] RPC call timeout handler was delayed by -2.7375735679997888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
I20260709 13:17:22.097064 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:22.101138 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 233733us
W20260709 13:17:22.105136 10095 connection.cc:256] RPC call timeout handler was delayed by -2.260573567999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24480000000000002s for a response
I20260709 13:17:22.105422 10048 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:22.106528 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.102571 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.103206 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106840us
I20260709 13:17:22.112408 10053 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:22.112589 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.050673 12448 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.115272 10051 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:22.110903 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.103793 10019 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.664s (SENT)
W20260709 13:17:22.125968 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.720s (SENT)
I20260709 13:17:22.126101 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.126899 10019 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.725s (SENT)
W20260709 13:17:22.127827 10019 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.746s (SENT)
I20260709 13:17:22.130779 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.132513 10048 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:22.129354 10019 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.804s (SENT)
I20260709 13:17:22.132819 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.134125 10019 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.811s (SENT)
W20260709 13:17:22.134938 10019 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.827s (SENT)
I20260709 13:17:22.135514 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.143713 12448 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.088467 10892 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:22.161432 10020 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.162981 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.164294 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 255729us
I20260709 13:17:22.088507 12584 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:22.090834 12463 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:22.162221 10019 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.172s (SENT)
W20260709 13:17:22.172468 10019 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.175s (SENT)
I20260709 13:17:22.153373 12201 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.154870 12560 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:22.157490 12483 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:22.161160 12559 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:22.183877 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:22.191874 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:22.165382 12365 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:22.173950 12482 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.180306 12492 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:22.171517 12599 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:22.222867 12584 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:22.226303 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.227305 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.228186 12592 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.238s	user 0.001s	sys 0.001s
I20260709 13:17:22.228639 12481 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:22.236583 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0690638660000786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3109s for a response
I20260709 13:17:22.209903 12583 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.235136 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:22.209066 12406 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.213083 12437 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:22.311926 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 149047us
I20260709 13:17:22.312095 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:22.312950 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.237576 12589 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
I20260709 13:17:22.316838 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:22.308367 12487 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.113s	user 0.009s	sys 0.006s
I20260709 13:17:22.320957 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:22.316941 10900 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:22.333251 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:22.317251 12583 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:17:22.311429 12595 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.316s	user 0.002s	sys 0.000s
I20260709 13:17:22.327665 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.248836 12365 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:22.333956 12594 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.286s	user 0.000s	sys 0.002s
I20260709 13:17:22.328088 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.355824 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:22.356506 10053 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:22.330650 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 146911us
I20260709 13:17:22.333828 10051 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:22.335575 10203 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:22.238938 12584 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:22.323227 12559 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:22.320947 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.356974 10126 raft_consensus.cc:2376] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5. Re-sending same reply.
I20260709 13:17:22.376362 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2577) took 1357 ms. Trace:
W20260709 13:17:22.376526 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55716.3us
I20260709 13:17:22.324044 12563 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:22.320223 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 194233us
I20260709 13:17:22.336599 12405 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.323331 12602 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.334296 12201 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:22.401993 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:22.330899 12609 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:22.348204 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:22.349180 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:22.348878 12365 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:22.319947 12492 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:17:22.351032 10894 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:22.364378 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.427083 12492 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:22.427642 10053 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:22.427868 12492 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:22.377645 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.422667 12563 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:22.429486 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51977.3us
I20260709 13:17:22.429145 12492 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.430409 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.377053 10126 rpcz_store.cc:276] 0709 13:17:21.018812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.086055 (+ 67243us) service_pool.cc:224] Handling call
0709 13:17:22.355609 (+1269554us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:22.376228 (+ 20619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201962368}
I20260709 13:17:22.377490 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.385342 12483 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.384941 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.237825 12599 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.367954 12584 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.383014 12559 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:22.392762 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.434886 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:22.438002 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2573) took 1489 ms (client timeout 1908 ms). Trace:
W20260709 13:17:22.438236 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.438514 10123 rpcz_store.cc:269] 0709 13:17:20.949029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.090061 (+141032us) service_pool.cc:224] Handling call
0709 13:17:22.392466 (+1302405us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:22.437910 (+ 45444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192963328}
I20260709 13:17:22.432384 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.400633 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1223) took 2177 ms (client timeout 1955 ms). Trace:
W20260709 13:17:22.439792 10128 rpcz_store.cc:269] 0709 13:17:20.223061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.231337 (+  8276us) service_pool.cc:224] Handling call
0709 13:17:22.400200 (+2168863us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54003d5754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:22.400440 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938975488}
I20260709 13:17:22.402849 10129 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.387444 12405 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.412149 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1211) took 2150 ms. Trace:
W20260709 13:17:22.455160 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70300.2us
I20260709 13:17:22.456923 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.454665 10055 rpcz_store.cc:276] 0709 13:17:20.262196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.054629 (+792433us) service_pool.cc:224] Handling call
0709 13:17:21.055184 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.885194 (+830010us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:21.949243 (+ 64049us) log.cc:844] Serialized 14 byte log entry
0709 13:17:22.156601 (+207358us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.157172 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.157793 (+   621us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:22.157846 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.157926 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:22.411809 (+253883us) raft_consensus.cc:1685] finished
0709 13:17:22.411834 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.412049 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":802760,"raft.queue_time_us":828037,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":28804,"thread_start_us":170630,"threads_started":1,"wal-append.queue_time_us":368528}
I20260709 13:17:22.461163 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.414451 10124 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:22.337565 12583 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:22.420161 12602 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:22.470582 12482 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.366343 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1214) took 2103 ms. Trace:
I20260709 13:17:22.430114 12563 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.370814 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:22.358618 12448 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.431684 10051 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
W20260709 13:17:22.488472 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 253487us
I20260709 13:17:22.436956 12560 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:22.435293 12584 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.433633 12185 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:17:22.434221 12609 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.447821 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:22.449401 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:22.449831 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.453428 12558 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.006s
I20260709 13:17:22.500638 10201 raft_consensus.cc:2376] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:22.501427 12558 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.502465 12558 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:22.467937 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.467962 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.469553 12583 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.503773 12560 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.486934 12482 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.507601 12482 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.502915 12558 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:22.512161 12558 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.472680 12602 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.473459 12601 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:22.416744 12201 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:22.476306 12612 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.444483 12559 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:22.482671 10049 rpcz_store.cc:276] 0709 13:17:20.263258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.088670 (+825412us) service_pool.cc:224] Handling call
0709 13:17:21.089036 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.886644 (+797608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:21.886683 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:21.914382 (+ 27699us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.173504 (+259122us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:22.210892 (+ 37388us) log.cc:844] Serialized 14 byte log entry
0709 13:17:22.211163 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.211246 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:22.365968 (+154722us) raft_consensus.cc:1685] finished
0709 13:17:22.366001 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:22.366232 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":800066,"prepare.queue_time_us":37711,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":3148,"raft.queue_time_us":801163,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":35678,"thread_start_us":211191,"threads_started":1,"wal-append.queue_time_us":427294}
W20260709 13:17:22.405036 12604 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:17:22.492393 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:22.566133 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73880.5us
I20260709 13:17:22.502907 10201 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 13:17:22.431022 12492 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:22.529387 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61681.7us
I20260709 13:17:22.567210 10095 leader_election.cc:400] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:22.567988 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.528208 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:22.532961 10129 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:22.530378 12544 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.654s	user 0.015s	sys 0.000s
W20260709 13:17:22.531657 12546 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.002s
I20260709 13:17:22.506916 12463 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.432688 12483 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:22.573817 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:22.577358 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 218890us
I20260709 13:17:22.562148 12201 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.581137 10202 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:22.571755 12448 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:22.571499 12614 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.572567 12487 raft_consensus.cc:2764] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:22.630033 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:22.630595 12599 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:22.572847 12601 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:22.633148 12462 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.632987 12601 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:22.644079 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63863.7us
I20260709 13:17:22.646227 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:22.584352 12546 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:22.656586 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 206862us
I20260709 13:17:22.587493 12610 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:22.684696 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:22.685438 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54225.7us
I20260709 13:17:22.657298 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:22.592391 12483 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:22.624930 12448 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:22.627524 12614 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.691468 10020 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.694425 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:22.700263 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 132368us
I20260709 13:17:22.701419 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.698167 12614 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.687759 12185 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:22.644745 12523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.989s	user 0.025s	sys 0.049s
I20260709 13:17:22.655292 12609 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:22.708115 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 180029us
I20260709 13:17:22.703025 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:22.693152 12462 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.692449 12448 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.696682 12560 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:22.717272 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.697188 10895 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:22.719163 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.706220 12487 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:22.721416 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:22.717044 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59815.3us
I20260709 13:17:22.730559 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:22.733224 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:22.724995 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.718544 12612 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:17:22.726876 12609 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:22.728474 12463 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:22.737542 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:22.710670 12523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.065s	user 0.026s	sys 0.049s
I20260709 13:17:22.759919 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.716075 12628 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.760380 12560 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.757449 12483 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.768107 12476 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:22.771369 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:22.775215 12626 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:22.765350 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:22.721841 12544 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:17:22.769251 12476 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:22.767805 12560 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.808730 12560 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.783718 12637 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:22.812162 12637 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.812433 12634 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:22.813000 12637 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:22.780249 12628 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.814504 12628 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.810730 12638 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.777345 12487 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:22.755944 12609 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:22.792021 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71865.2us
I20260709 13:17:22.736606 12462 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:22.835040 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 114994us
I20260709 13:17:22.764922 12625 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:22.807309 12476 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.824491 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:22.764506 12463 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:22.805348 12419 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:22.840420 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:22.841811 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.849860 12419 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.853822 12419 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:22.850167 12463 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.834229 12609 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.862377 10201 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:22.876361 12601 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:22.878605 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.842752 12476 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:22.887545 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47277.5us
I20260709 13:17:22.907825 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:22.909677 12558 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.889693 12643 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:22.911154 12643 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.911437 12637 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:22.911648 12643 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:22.912061 12637 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:22.912596 12583 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.858964 12633 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:22.891764 10897 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:17:22.916988 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 461) took 1048 ms. Trace:
I20260709 13:17:22.917457 10897 rpcz_store.cc:276] 0709 13:17:21.868686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.875226 (+  6540us) service_pool.cc:224] Handling call
0709 13:17:22.891084 (+1015858us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:22.891331 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.916644 (+ 25313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.916910 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5641984}
I20260709 13:17:22.834599 12487 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:22.920099 10899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 157) took 1700 ms (client timeout 1593 ms). Trace:
I20260709 13:17:22.920755 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.921641 12487 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:22.839679 12638 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:17:22.899348 12586 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:22.895320 12601 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:22.903012 12419 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.839196 12625 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:22.912563 12637 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:22.892347 12630 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:22.926607 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 122667us
I20260709 13:17:22.924322 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.933216 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.936715 10129 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:22.928208 12481 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:22.928001 12583 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:22.931305 12625 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:22.898689 12560 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.924541 12201 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.951663 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:22.959187 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 237869us
I20260709 13:17:22.960794 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.961474 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.952674 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.962221 10055 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:22.962283 10051 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:22.956597 10050 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:22.962284 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.952893 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.964923 10123 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:22.969784 12583 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.932341 12185 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:22.970906 12583 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.972295 12185 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:22.975991 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:22.976933 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:22.976934 10049 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:22.978921 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.936120 12630 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:22.980111 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 220312us
I20260709 13:17:22.966384 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.984931 12487 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:22.985164 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109597us
I20260709 13:17:22.943100 12627 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:22.986435 12601 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:22.984918 10020 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.985598 12487 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:22.988072 12487 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:22.954326 12615 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:22.991644 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.988670 12601 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:22.980778 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:22.994468 12615 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:22.936178 12544 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:17:23.017699 10123 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:23.018472 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:22.951187 12448 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.067822 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:22.939466 12482 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:22.954121 12633 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.964876 12481 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:23.054831 12419 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:22.931663 12586 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.028959 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:23.020222 12492 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:23.026286 12583 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:22.920801 10899 rpcz_store.cc:269] 0709 13:17:21.220179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.220686 (+   507us) service_pool.cc:224] Handling call
0709 13:17:22.919828 (+1699142us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54003f2ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:22.919979 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888184576}
I20260709 13:17:23.074271 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.075683 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.081429 12448 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:23.069708 12559 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:23.083628 12642 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:17:23.084096 12633 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.085464 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1468) took 1156 ms. Trace:
I20260709 13:17:23.085769 12481 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.087345 12560 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.087934 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.088968 12201 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:23.071472 12584 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.004s
W20260709 13:17:23.093751 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75412.8us
W20260709 13:17:23.093232 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64376.3us
I20260709 13:17:23.116026 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.060410 12630 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.044725 12628 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.136080 10129 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:23.137171 12628 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.138788 12482 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:23.125049 12165 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:23.141530 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1242) took 1253 ms. Trace:
I20260709 13:17:23.141924 10127 rpcz_store.cc:276] 0709 13:17:21.888033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.009855 (+121822us) service_pool.cc:224] Handling call
0709 13:17:22.010332 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.141966 (+131634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.142003 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:22.142061 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:22.142096 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.141279 (+999183us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002ed1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:23.141325 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.141465 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302123904}
I20260709 13:17:23.143607 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.144672 12630 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.144870 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.145622 10127 raft_consensus.cc:2368] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:23.146157 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.146481 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.110302 12559 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.147202 10053 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.106638 10205 rpcz_store.cc:276] 0709 13:17:21.928712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.771144 (+842432us) service_pool.cc:224] Handling call
0709 13:17:22.771547 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.776718 (+  5171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.776764 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.791452 (+ 14688us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.977950 (+186498us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:22.977998 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.978080 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.085160 (+107080us) raft_consensus.cc:1685] finished
0709 13:17:23.085189 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.085383 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4595,"raft.queue_time_us":37016,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"thread_start_us":160002,"threads_started":1,"wal-append.queue_time_us":237374}
I20260709 13:17:23.149047 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.094355 12639 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:23.099189 12635 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:22.930537 12614 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.119606 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:23.164602 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45139.9us
I20260709 13:17:23.164732 10126 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:23.098361 12627 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:23.074079 12637 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.116834 12558 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.118777 12584 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.181012 12637 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.123284 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.184695 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.184993 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.186067 10128 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:23.125929 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.187723 10047 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:23.188978 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1467) took 1281 ms. Trace:
I20260709 13:17:23.126125 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.189595 10201 rpcz_store.cc:276] 0709 13:17:21.916452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.568049 (+651597us) service_pool.cc:224] Handling call
0709 13:17:22.568529 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:22.869654 (+301125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.188599 (+318945us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54003598d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:23.188881 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675087104}
I20260709 13:17:23.190411 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1456) took 1326 ms. Trace:
I20260709 13:17:23.190976 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.191039 10204 rpcz_store.cc:276] 0709 13:17:21.863464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.888691 (+ 25227us) service_pool.cc:224] Handling call
0709 13:17:23.125968 (+1237277us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:23.190293 (+ 64325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13398144}
I20260709 13:17:23.196548 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.129670 12643 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.141342 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.206589 10049 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.076545 12615 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.093097 12476 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.158071 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.216740 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.217738 10128 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:23.219187 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.138659 12625 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.165426 10020 leader_election.cc:400] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:23.222797 10020 leader_election.cc:403] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:23.223640 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58333.7us
I20260709 13:17:23.224736 10020 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.174679 12627 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.160943 12635 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.176242 12185 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.178395 12560 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:23.178656 12614 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:23.198422 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.191046 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.198011 10204 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:23.199087 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:23.204852 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.123297 12201 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:23.182646 12584 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.241636 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.246670 12185 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.104167 12563 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:23.249293 12185 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.248848 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.249987 12560 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:23.250459 12560 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.250921 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.251773 10127 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:23.252619 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.217442 12448 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:17:23.238771 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.254447 12448 consensus_queue.cc:1243] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:23.233604 12648 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:17:23.233572 12635 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.258328 12615 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.259526 12201 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:23.260839 12492 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:23.243935 12586 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.232079 12559 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:23.260637 12563 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:23.209770 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.268770 12476 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:23.270018 12633 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.271348 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1444) took 1442 ms. Trace:
I20260709 13:17:23.271744 10202 rpcz_store.cc:276] 0709 13:17:21.829304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.871810 (+ 42506us) service_pool.cc:224] Handling call
0709 13:17:23.271105 (+1399295us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b540012a054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:23.271272 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612461440}
I20260709 13:17:23.273303 10202 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.261057 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:23.307798 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46827.2us
I20260709 13:17:23.315304 12476 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:23.265909 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66990us
I20260709 13:17:23.266047 12626 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:23.266194 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.329728 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1442) took 2102 ms. Trace:
I20260709 13:17:23.330298 10196 rpcz_store.cc:276] 0709 13:17:21.227335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.227824 (+   489us) service_pool.cc:224] Handling call
0709 13:17:21.248854 (+ 21030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.984686 (+735832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.329419 (+1344733us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54001fe854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:23.329626 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679207296}
I20260709 13:17:23.331543 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.332306 10051 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:23.277333 10123 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.323489 12626 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.334563 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.331064 12585 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:17:23.336032 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:23.336854 10895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 160) took 1462 ms (client timeout 1549 ms). Trace:
I20260709 13:17:23.331211 12173 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:17:23.337481 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:23.337393 10895 rpcz_store.cc:269] 0709 13:17:21.874650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.880425 (+  5775us) service_pool.cc:224] Handling call
0709 13:17:23.336639 (+1456214us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b540053c0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:23.336778 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681466240}
I20260709 13:17:23.330756 10047 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:23.338979 10047 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:23.335161 12626 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.339948 10047 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.341341 10047 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.333004 10049 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:23.168421 12463 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.345732 12463 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:23.346534 12437 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:23.242408 12625 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.280637 12405 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:23.354244 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.356354 12611 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:23.357968 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.284190 12601 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:23.286237 12588 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.313s	user 0.005s	sys 0.001s
I20260709 13:17:23.291273 10122 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:23.290212 10204 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:23.358867 10124 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.363692 10128 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:23.379638 10131 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:23.379511 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.380412 10054 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:23.376137 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.381618 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.382534 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.370221 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.365823 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.383899 10196 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.296574 10126 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:21.978663 10052 rpcz_store.cc:276] 0709 13:17:20.545668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.155818 (+610150us) service_pool.cc:224] Handling call
0709 13:17:21.186730 (+ 30912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:21.187047 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:21.827400 (+640353us) log.cc:844] Serialized 14 byte log entry
0709 13:17:21.974456 (+147056us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:21.974503 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:21.974597 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:21.974707 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:21.978165 (+  3458us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3066,"prepare.queue_time_us":638581,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":62,"raft.queue_time_us":7694,"thread_start_us":143520,"threads_started":1}
I20260709 13:17:23.389784 12601 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:23.390452 10051 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.393541 10054 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [4.2-7.3]
I20260709 13:17:23.403414 10055 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.256119 10053 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:23.408921 10049 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.297114 12639 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:23.297545 12635 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.300730 12526 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:23.301553 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.304243 12365 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:23.304942 12643 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:23.305963 10892 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 343) took 2964 ms (client timeout 1612 ms). Trace:
I20260709 13:17:23.322623 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1471) took 1337 ms. Trace:
I20260709 13:17:23.443986 10202 rpcz_store.cc:276] 0709 13:17:21.985001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.272665 (+1287664us) service_pool.cc:224] Handling call
0709 13:17:23.273081 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:23.322207 (+ 49126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.322536 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:23.236850 12609 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:23.281996 12651 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:23.394929 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.483901 10199 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:23.400236 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.401993 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:23.485754 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 83928.4us
I20260709 13:17:23.295655 12560 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.358796 12165 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:17:23.498253 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.413914 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.414400 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.414002 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.294443 12487 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.476312 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:23.480379 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:23.377221 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:23.503739 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.520193 10896 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:23.502403 12483 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.017s	sys 0.056s
W20260709 13:17:23.523003 12620 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.013s	sys 0.053s
W20260709 13:17:23.525882 12620 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.013s	sys 0.053s
I20260709 13:17:23.267701 12563 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.499307 12445 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:23.538239 10897 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:23.535823 12445 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.540239 12445 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:23.541265 12487 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:23.526440 12483 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.017s	sys 0.057s
I20260709 13:17:23.554107 12635 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:23.560854 12560 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.500835 12601 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.572359 12601 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.384596 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.447913 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:23.449008 10122 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:23.422905 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:23.404173 10125 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:23.253713 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.432497 10129 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:23.439456 10130 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:23.421324 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:23.416411 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58105.1us
W20260709 13:17:23.347019 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.491807 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1450) took 1637 ms. Trace:
I20260709 13:17:23.480918 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.230878 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.521930 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:23.473228 10892 rpcz_store.cc:269] 0709 13:17:20.341964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:20.350285 (+  8321us) service_pool.cc:224] Handling call
0709 13:17:23.305675 (+2955390us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b540064f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:23.305815 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845872384}
I20260709 13:17:23.526760 10895 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.681154 10125 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.680895 10127 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:23.693507 10202 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:23.657255 10053 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:23.699478 10201 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:23.655853 10052 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:23.658670 10054 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:23.714787 10891 raft_consensus.cc:2376] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:23.654311 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.659200 10056 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:23.630025 10020 connection.cc:256] RPC call timeout handler was delayed by -2.3909463560000859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:17:23.669322 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.657611 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.702029 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:23.654062 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.654987 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.673348 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1464) took 1797 ms. Trace:
I20260709 13:17:23.689630 10124 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:23.678138 10123 raft_consensus.cc:2376] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8. Re-sending same reply.
I20260709 13:17:23.679754 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:23.672709 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:23.664602 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1465) took 1787 ms. Trace:
I20260709 13:17:23.723261 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.673445 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1461) took 1798 ms. Trace:
I20260709 13:17:23.770124 10050 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.769773 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:23.712949 10865 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.528s (SENT)
I20260709 13:17:23.718104 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.692221 10199 rpcz_store.cc:276] 0709 13:17:21.853937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.854097 (+   160us) service_pool.cc:224] Handling call
0709 13:17:23.394709 (+1540612us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:23.491682 (+ 96973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13276928}
I20260709 13:17:23.718142 12165 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:23.703442 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 205166us
W20260709 13:17:23.676653 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 228762us
I20260709 13:17:23.797542 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.797201 12463 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:17:23.798046 12483 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:23.799155 12483 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.799721 10047 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:23.800352 10047 pending_rounds.cc:85] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:17:23.800781 10047 pending_rounds.cc:107] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:23.801192 10047 raft_consensus.cc:2889] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:23.716518 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:23.803237 12560 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:23.804024 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.804314 12560 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.805984 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.583716 12609 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.808159 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.809397 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.810292 10056 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:23.809880 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.784477 12583 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.790788 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.791954 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:23.822892 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:23.823947 12630 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:23.824991 12630 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:23.827948 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.828658 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.829238 10891 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:23.829806 10202 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:23.832747 10896 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:17:23.833950 12463 consensus_queue.cc:1243] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:23.834885 12463 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:23.835563 10054 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.771414 10198 rpcz_store.cc:276] 0709 13:17:21.875264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.926831 (+ 51567us) service_pool.cc:224] Handling call
0709 13:17:21.939146 (+ 12315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:21.947485 (+  8339us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:17:22.174112 (+226627us) log.cc:844] Serialized 14 byte log entry
0709 13:17:22.653200 (+479088us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:23.349825 (+696625us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b48001e0258. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:17:23.360392 (+ 10567us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:17:23.360982 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.361184 (+   202us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:23.672022 (+310838us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.672599 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.672747 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.672864 (+   117us) raft_consensus.cc:1685] finished
0709 13:17:23.672877 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.673160 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7752,"prepare.queue_time_us":436694,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":108,"raft.queue_time_us":55939,"raft.run_cpu_time_us":19352,"raft.run_wall_time_us":1401939,"spinlock_wait_cycles":26954368,"thread_start_us":949032,"threads_started":3,"wal-append.queue_time_us":533528}
I20260709 13:17:23.840590 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.840899 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.841847 10053 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:23.835497 12463 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:23.842365 10049 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.798930 10051 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:23.844362 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:23.845046 10051 raft_consensus.cc:2376] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:23.846153 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:23.819641 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 148797us
I20260709 13:17:23.843770 12463 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.848064 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:23.853453 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 151895us
I20260709 13:17:23.862437 10128 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:23.864589 12421 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:23.865317 12421 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.865552 12635 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:23.865849 12421 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:23.866484 12635 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.856479 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.868077 10205 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.868208 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:23.870118 10204 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:23.870659 12635 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:23.871145 12609 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:23.871680 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.871820 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.872044 12445 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.873638 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.873827 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.875406 12615 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:23.876145 12615 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:23.876888 12615 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.877756 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:23.879284 12583 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:23.880921 12445 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.820109 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.870483 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:23.857566 10051 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:23.859745 10198 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:23.879004 10900 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:23.802910 12483 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.899960 10896 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:23.765440 10203 rpcz_store.cc:276] 0709 13:17:21.876136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.491929 (+615793us) service_pool.cc:224] Handling call
0709 13:17:22.492613 (+   684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.529448 (+ 36835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.529485 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.552198 (+ 22713us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.386934 (+834736us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18004113a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:17:23.664298 (+277364us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:23.671605 (+  7307us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.672149 (+   544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.672412 (+   263us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.672543 (+   131us) raft_consensus.cc:1685] finished
0709 13:17:23.672700 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.673039 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":672,"prepare.queue_time_us":76254,"raft.queue_time_us":88854,"raft.run_cpu_time_us":478,"raft.run_wall_time_us":4910,"spinlock_wait_cycles":2942336,"thread_start_us":276692,"threads_started":1,"wal-append.queue_time_us":972414}
I20260709 13:17:23.911095 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:23.768744 10200 rpcz_store.cc:276] 0709 13:17:21.876364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.500228 (+623864us) service_pool.cc:224] Handling call
0709 13:17:22.500700 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.551519 (+ 50819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.551577 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.904891 (+353314us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b48001e0258. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:17:23.139929 (+235038us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:17:23.140519 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.344289 (+203770us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:23.344350 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.344444 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.663675 (+319231us) raft_consensus.cc:1685] finished
0709 13:17:23.663723 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.664179 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28277,"raft.queue_time_us":69411,"raft.run_cpu_time_us":2380,"raft.run_wall_time_us":332709,"spinlock_wait_cycles":16176256,"thread_start_us":203314,"threads_started":1,"wal-append.queue_time_us":349241}
I20260709 13:17:23.913872 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.891366 12560 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:23.919957 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:23.923164 10900 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:23.938118 10050 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:23.936488 10053 raft_consensus.cc:2376] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5. Re-sending same reply.
I20260709 13:17:23.918257 12628 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:23.943316 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.826727 12630 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:23.924142 12445 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:23.951324 10204 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:23.950953 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.955768 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.958714 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.961050 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.961846 10896 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:23.963449 12165 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:23.965514 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.966621 10052 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:23.962859 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:23.968505 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 255974us
I20260709 13:17:23.969928 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.943427 12639 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.980783 12481 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:23.981500 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:23.981743 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.982717 10052 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:23.983268 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.984704 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:23.985455 10200 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:23.975283 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 105022us
I20260709 13:17:23.984045 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:23.992062 10196 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:23.916229 12609 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:17:23.981948 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:24.012732 10050 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:23.996282 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:23.980994 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:24.016062 12626 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.612s	user 0.015s	sys 0.009s
I20260709 13:17:24.016813 12626 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.012544 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:24.022015 10203 raft_consensus.cc:2368] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:17:24.021152 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:24.031329 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:24.037154 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:24.043493 10054 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:17:24.013095 12463 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.042680 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:24.052043 10055 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:24.055613 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 116953us
I20260709 13:17:24.055727 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:24.050993 10048 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:24.081934 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 51297us
W20260709 13:17:24.089208 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 158235us
I20260709 13:17:24.091188 10049 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:24.064009 12560 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.095247 10203 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.071218 12620 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:23.980767 12639 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.101579 10052 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 5: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:24.103930 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 122575us
I20260709 13:17:24.109793 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 169) took 2060 ms. Trace:
I20260709 13:17:24.110776 10203 rpcz_store.cc:276] 0709 13:17:22.040949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.030364 (+1989415us) service_pool.cc:224] Handling call
0709 13:17:24.085311 (+ 54947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.100773 (+ 15462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.101033 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:24.116022 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 54920.9us
I20260709 13:17:24.122361 10053 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:24.112857 12406 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:24.136469 10204 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 13:17:24.142343 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1480) took 1202 ms. Trace:
I20260709 13:17:24.142880 10204 rpcz_store.cc:276] 0709 13:17:22.939518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.064954 (+1125436us) service_pool.cc:224] Handling call
0709 13:17:24.091227 (+ 26273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:24.141957 (+ 50730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.142265 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:24.097081 12626 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:24.127848 12583 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:24.161427 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.128317 12481 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:17:24.163861 12601 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.008s
I20260709 13:17:24.165119 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1469) took 2213 ms. Trace:
I20260709 13:17:24.165731 10201 rpcz_store.cc:276] 0709 13:17:21.951542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.254901 (+1303359us) service_pool.cc:224] Handling call
0709 13:17:23.255281 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:23.768661 (+513380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.164713 (+396052us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54002ddcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:24.165020 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940203392}
W20260709 13:17:24.167337 12655 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:17:24.165143 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1452) took 2288 ms. Trace:
W20260709 13:17:24.169404 12671 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:17:24.168046 10197 rpcz_store.cc:276] 0709 13:17:21.854667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:21.854799 (+   132us) service_pool.cc:224] Handling call
0709 13:17:23.147446 (+1292647us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:23.147672 (+   226us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:23.148162 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.148191 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:23.148908 (+   717us) log.cc:844] Serialized 52 byte log entry
0709 13:17:23.468799 (+319891us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:23.905509 (+436710us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.063727 (+158218us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.115341 (+ 51614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.115425 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.135641 (+ 20216us) raft_consensus.cc:1685] finished
0709 13:17:24.135660 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.143474 (+  7814us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79693,"prepare.queue_time_us":76620,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":123,"raft.queue_time_us":90585,"raft.run_cpu_time_us":1116,"raft.run_wall_time_us":21288,"spinlock_wait_cycles":14777600,"thread_start_us":644641,"threads_started":2,"wal-append.queue_time_us":361020}
I20260709 13:17:24.171892 12481 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:24.172348 12481 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.173714 10895 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:24.173421 12481 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.174333 10199 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:24.174681 10894 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.153506 12406 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.176633 12406 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.186656 10204 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:24.135267 12560 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:24.196810 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.197552 10203 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.197691 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:24.136930 12609 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:17:24.201280 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 120125us
I20260709 13:17:24.200731 12481 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:24.231312 10197 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:24.203153 12627 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:24.199177 12583 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:17:24.268216 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:24.270488 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:24.283667 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71184.1us
I20260709 13:17:24.268705 12620 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.287835 12636 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.289435 12643 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.299190 12583 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:24.300062 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 148095us
I20260709 13:17:24.300964 12583 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:24.301765 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.304039 10200 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:24.289534 12670 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:17:24.294721 12445 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:24.319629 12627 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.331981 10205 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:24.301487 12583 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:24.315402 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:24.333364 10056 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.011s
I20260709 13:17:24.332723 10900 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:24.333735 10196 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:24.333863 10056 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:24.335029 10048 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:17:24.334898 12626 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:24.336843 12620 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:24.331936 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:24.295375 12643 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:24.351661 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49538.8us
I20260709 13:17:24.354574 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:24.355651 12579 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:24.357919 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:24.359032 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:24.359859 12637 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:24.375049 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1479) took 1435 ms. Trace:
I20260709 13:17:24.360495 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:24.363198 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:24.380373 10050 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:17:24.366055 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54451.7us
I20260709 13:17:24.331943 12445 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.378796 10196 rpcz_store.cc:276] 0709 13:17:22.939214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.071962 (+1132748us) service_pool.cc:224] Handling call
0709 13:17:24.333433 (+261471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:24.334312 (+   879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.374720 (+ 40408us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540035a7d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:24.374937 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535379328}
I20260709 13:17:24.371420 10898 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:24.381655 12672 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.096s	user 0.001s	sys 0.001s
W20260709 13:17:24.385619 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 208842us
W20260709 13:17:24.391247 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 135799us
I20260709 13:17:24.372053 12579 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.408830 12563 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.409825 10019 leader_election.cc:400] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
W20260709 13:17:24.411101 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 114625us
I20260709 13:17:24.413759 12165 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.376238 12173 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.347774 12437 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:17:24.376423 12561 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:24.436228 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 201442us
I20260709 13:17:24.385366 12445 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:24.386765 12627 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:24.404663 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72242.1us
I20260709 13:17:24.441143 12482 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:24.442615 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.445914 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:24.449752 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:24.511345 10056 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:24.450966 12544 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:24.452107 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91761.8us
I20260709 13:17:24.452872 12583 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.455659 12643 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.457551 10895 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:24.458479 12620 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.432133 12165 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.469288 12173 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:24.472487 10900 raft_consensus.cc:1217] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-5.7]   Dedup: 4.6->[5.7-5.7]
I20260709 13:17:24.540380 10900 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 13:17:24.479449 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:24.402503 12609 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:24.447741 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:24.422771 12637 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:24.423321 12579 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:24.519447 10051 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.005s
I20260709 13:17:24.550004 10051 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:24.550482 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 71265.1us
W20260709 13:17:24.494962 12671 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:17:24.528869 10047 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:24.512707 12639 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:24.542112 12421 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.675s	user 0.016s	sys 0.000s
W20260709 13:17:24.567320 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119767us
I20260709 13:17:24.568250 12636 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:24.568794 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:24.570266 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 124528us
I20260709 13:17:24.570418 12165 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.571476 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:24.527671 12482 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.573336 12483 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:24.574277 12483 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.574386 12482 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:24.575250 12365 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.575934 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 168) took 2585 ms. Trace:
I20260709 13:17:24.568919 12636 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.537452 12561 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.545372 12609 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.572094 12165 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:24.546711 12637 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:24.576370 10198 rpcz_store.cc:276] 0709 13:17:21.990750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.023850 (+2033100us) service_pool.cc:224] Handling call
0709 13:17:24.107990 (+ 84140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.292819 (+184829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.292854 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.292926 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:24.292968 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.575647 (+282679us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:24.575696 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.575870 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370967808}
I20260709 13:17:24.493657 12563 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:24.583597 10864 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.041s (SENT) [suppressed 36 similar messages]
W20260709 13:17:24.599887 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 151113us
W20260709 13:17:24.603930 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 171956us
W20260709 13:17:24.611930 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 68917.6us
I20260709 13:17:24.612548 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1491) took 1167 ms. Trace:
I20260709 13:17:24.613091 10203 rpcz_store.cc:276] 0709 13:17:23.445590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.224631 (+779041us) service_pool.cc:224] Handling call
0709 13:17:24.261933 (+ 37302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:24.307950 (+ 46017us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:24.372946 (+ 64996us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.612045 (+239099us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.612116 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:24.612168 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.612255 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.612431 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62826,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":67002,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":5959,"thread_start_us":184217,"threads_started":2,"wal-append.queue_time_us":164374}
I20260709 13:17:24.578327 12630 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.588685 12639 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.619464 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:24.554576 12421 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.589792 12609 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:24.601315 12583 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:24.617270 10893 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.813s	user 0.016s	sys 0.002s
I20260709 13:17:24.634171 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:24.624991 12643 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:24.643970 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75368.1us
I20260709 13:17:24.649154 12690 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:24.651520 12690 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:24.650986 12627 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.652170 12690 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.652742 12421 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:24.654409 12620 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:24.654691 10202 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:24.654735 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 73605.1us
I20260709 13:17:24.658128 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:24.663408 12636 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.664726 12563 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:24.665230 12636 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:24.635011 12639 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:24.668180 12560 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.668681 12639 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.645035 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:24.675485 12637 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.680130 12637 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.676743 10892 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:24.615448 12561 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:24.639541 10893 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:24.677421 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1269) took 1187 ms. Trace:
I20260709 13:17:24.697060 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:24.697060 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:24.673066 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:24.683472 12686 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:24.685029 10056 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:24.685391 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:24.686697 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:24.687961 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:24.688160 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1487) took 1253 ms. Trace:
I20260709 13:17:24.701798 10196 raft_consensus.cc:2376] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
W20260709 13:17:24.689730 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:24.702808 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.676028 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:24.701936 10202 rpcz_store.cc:276] 0709 13:17:23.435166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.948979 (+513813us) service_pool.cc:224] Handling call
0709 13:17:24.687931 (+738952us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:24.688090 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438634240}
I20260709 13:17:24.642616 12630 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:24.660115 12615 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.006s
I20260709 13:17:24.712724 12615 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.697389 10126 rpcz_store.cc:276] 0709 13:17:23.361706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.437851 (+ 76145us) service_pool.cc:224] Handling call
0709 13:17:23.438244 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:23.472140 (+ 33896us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.472191 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:23.702105 (+229914us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b2c0011b9a8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:17:23.703795 (+  1690us) log.cc:844] Serialized 52 byte log entry
0709 13:17:24.361460 (+657665us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b180044fce0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1
0709 13:17:24.361634 (+   174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:24.361991 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.362391 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.529586 (+167195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.536350 (+  6764us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.542159 (+  5809us) raft_consensus.cc:1685] finished
0709 13:17:24.542189 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.548825 (+  6636us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":252643,"prepare.queue_time_us":3663,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":6942,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":16401,"spinlock_wait_cycles":18115712,"thread_start_us":225834,"threads_started":2,"wal-append.queue_time_us":232066}
W20260709 13:17:24.721882 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:24.724520 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48579.9us
I20260709 13:17:24.727756 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:24.727864 10054 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:17:24.731658 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85012.5us
I20260709 13:17:24.716444 12560 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:24.653338 12690 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.746541 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:24.747393 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 74402us
W20260709 13:17:24.748323 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63689us
I20260709 13:17:24.746771 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:24.751150 10199 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:24.751724 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:24.752691 10204 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:24.760147 10047 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:24.761487 10053 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.764142 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:24.762382 10205 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:24.761931 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:24.769820 10196 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:17:24.738682 10893 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 170) took 1246 ms (client timeout 1576 ms). Trace:
I20260709 13:17:24.770010 10050 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:24.759466 12686 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:17:24.774178 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53655us
I20260709 13:17:24.775274 10052 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:24.759509 12696 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:17:24.777542 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:24.778505 12686 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:24.785084 10049 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:24.778023 12463 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.706s	user 0.016s	sys 0.001s
I20260709 13:17:24.794866 10056 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:24.789403 10053 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:24.784231 12627 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.802736 12603 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.802933 12630 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.809252 10047 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:24.815236 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:24.772688 10893 rpcz_store.cc:269] 0709 13:17:23.477317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.481786 (+  4469us) service_pool.cc:224] Handling call
0709 13:17:24.716157 (+1234371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:24.819828 12682 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.820531 12697 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:24.820971 12682 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.823369 12687 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.824164 12687 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.827261 12658 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:24.827809 12658 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.828202 12658 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:24.828516 12561 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.821475 12697 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:24.826812 12687 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:24.812618 12615 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.845638 12560 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.793550  9953 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 5, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:24.849364 12560 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.851850 12560 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.852401 10056 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:17:24.855787 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.857911 12544 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:24.859579 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:24.811408 12463 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.814713 12628 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.806633 12688 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:24.877830 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:24.878968 10049 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:24.804123 12421 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 13:17:24.862988 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:24.800465 12686 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:24.875008 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 121778us
I20260709 13:17:24.865450 12445 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.798280 12627 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:24.863842 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113185us
I20260709 13:17:24.841327 12561 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.894071 10204 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:24.868820 12628 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:24.889588 12421 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:24.905166 10198 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:24.880183 12544 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:24.893848 12687 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:24.910744 12628 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.892699 12620 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.876541 12463 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.923534 12579 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:24.912746 12561 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.928048 12694 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:17:24.930646 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:24.931686 12481 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:24.932061 10049 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:24.932176 12626 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:24.932538 12620 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.933319 12680 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:24.933985 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1856043059999593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2663s for a response
I20260709 13:17:24.934916 12686 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:24.946658 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87238.7us
I20260709 13:17:24.908988 12544 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:24.961685 12560 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:24.962561 12560 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:24.893095 12688 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.946328 12561 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.974529 12637 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.986682 12560 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.987476 12637 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.987905 12615 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:24.988996 12615 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.959472 12680 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:24.997870 12658 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:24.931514 12487 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:17:24.944059 12698 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:24.943567 12603 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:24.981096 12628 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.018360 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.020160 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.020888 10205 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:25.018951 10053 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:25.013636 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83864us
I20260709 13:17:25.022109 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:25.000156 12487 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:25.013963 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 240192us
I20260709 13:17:25.026549 12487 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:25.027846 12487 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.028401 12688 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:24.952791 12697 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.029621 12487 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.031644 12705 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:25.032295 12705 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:25.033986 12603 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.012024 12698 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.036311 12630 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:25.036708 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:25.037166 12698 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:25.037101 12630 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.033486 12705 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.038769 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:25.013549 12561 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.040019 12627 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:25.040323 10128 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:25.029663 12583 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.005963 12682 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:24.999078 12680 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.028502 12579 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.000941 12544 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.048267 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.107812 12680 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:25.108402 12544 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.107897 12583 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.110581 12583 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:25.017041 12703 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:25.111435 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:25.111336 12703 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.113662 12703 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:25.114012 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:25.114432 10125 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:25.114645 10051 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.113636 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:25.118530 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.060320 10130 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:17:25.127501 10130 pending_rounds.cc:85] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:25.127713 12630 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:25.062345 12463 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.062623 12626 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:25.128343 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.128827 12560 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:25.129200 10127 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:25.063434 12637 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.068871 10202 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.130657 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.131309 10127 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:25.081460 12705 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.081607 10198 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:25.081925 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.084090 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:25.085268 12561 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.085897 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:25.081615 12620 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.091579 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.099959 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:25.102238 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.153115 10125 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:17:25.106132 10893 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:25.155702 10129 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:25.156389 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:25.157200 10198 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:25.157369 10129 pending_rounds.cc:85] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 1
I20260709 13:17:25.157871 10129 pending_rounds.cc:107] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:17:25.158236 10129 raft_consensus.cc:2889] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.159699 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.049121 12639 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.120221 12563 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.162092 12561 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.162670 12639 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.165311 12639 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.171439 12635 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:25.172008 12635 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:25.173010 12635 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.174608 12635 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:25.162093 10200 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:25.098093 12481 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.181687 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.182246 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:25.182329 12481 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.185034 12481 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:25.185694 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.122635 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.120865 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:25.187476 10128 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.188123 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.188480 12658 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.124928 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.126417 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:25.189520 10900 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:25.127851 10130 pending_rounds.cc:107] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:25.190538 10130 raft_consensus.cc:2889] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.053314 12445 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.131892 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.061808 10131 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:25.132799 12560 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:25.193696 12560 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:25.194280 12560 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.195569 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.196444 10053 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:25.196483 12560 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.113559 12694 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:25.076409 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.136034 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:25.122004 10203 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:25.144122 10126 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:25.148598 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:25.208904 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.209204 10124 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:25.209646 10051 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:25.151300 10122 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:25.154731 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.210664 10196 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:25.157532 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:25.182822 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:25.211664 10047 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:25.133267 12615 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.132135 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.187510 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.214941 10202 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:25.209769 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61294.2us
W20260709 13:17:25.190961 12609 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.548s	user 0.004s	sys 0.007s
I20260709 13:17:25.216634 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:25.224563 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:25.225189 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.218020 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.220104  9953 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:25.230872 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.234740 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.244511 10056 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:25.250417 10900 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:25.252146 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:25.254218 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:25.262038 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.264094 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:25.197242 12560 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:25.253517 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:25.266623 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.267584 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:25.268653 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.207700 12694 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:25.269171 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111895us
I20260709 13:17:25.275038 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:25.226002 12620 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.223726 12601 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:25.290145 10205 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:25.299968 10126 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:25.293141 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:25.305557 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87662.9us
I20260709 13:17:25.311798 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:25.316205 10049 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:25.322059 10055 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:25.276170 12561 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:25.325861 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:25.284771 12481 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:25.234203 12690 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:17:25.243242 12609 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.302000 12601 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.284555 12686 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:25.334699 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:25.284204 12636 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:25.326838 12561 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:25.343402 12687 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.380378 12639 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.455524 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 323460us
I20260709 13:17:25.459939 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:25.381356 12690 consensus_queue.cc:1243] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:17:25.405402 12672 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:17:25.329962 12699 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:25.407716 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.317034 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:25.413444 12712 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 13:17:25.584966 10127 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:25.309686 12620 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.586447 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.587899 12563 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:25.588127 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 253562us
I20260709 13:17:25.588476 12620 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.470151 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.589699 10125 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:25.467895 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.406585 12694 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.590961 12711 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:25.591707 10122 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:25.277632 12626 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:17:25.465178  9953 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 6, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.396379  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:25.457046 10131 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:25.380364 12601 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.453024 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.596194 10124 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:25.301806 12560 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.399537 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63576.4us
I20260709 13:17:25.453909 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:25.456260 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.449072 10055 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:25.379472 12406 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:17:25.348997 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.405064 12689 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
W20260709 13:17:25.497305 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.534873 10128 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:25.545579 12715 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:17:25.540426 12702 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:17:25.546795 12713 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 13:17:25.547289 12716 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:17:25.387517 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55580.1us
I20260709 13:17:25.632910 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:25.350245 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.389622 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:25.384465 10047 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:25.565443 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:25.567751 12639 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.560304 12561 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:25.566293 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 180176us
W20260709 13:17:25.572502 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 117392us
I20260709 13:17:25.567806 10895 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:25.580331 12687 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:25.631861  9953 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.457348 10123 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:25.454934 12609 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.602895 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.531697 10198 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:25.615587 12694 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:25.604701 10130 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:25.578802 12658 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.629582 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.628726 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.650442 12728 raft_consensus.cc:2764] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:17:25.591305 12721 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.690090 12487 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:25.641649 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:25.692317 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58177.8us
I20260709 13:17:25.693524 12636 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:25.637172 10048 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:25.645004 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:25.697700 10126 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:25.593065 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:25.647840 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 82468.6us
I20260709 13:17:25.699980 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.701428 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:25.592078 12699 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:25.704921 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:25.650532 10124 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:25.709173 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47978.4us
I20260709 13:17:25.710978 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.579892 12686 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:25.662366  9953 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.664321 12560 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:25.664644 10892 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:17:25.665475 10127 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:25.613343 10893 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:25.666275 10125 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:25.667658 12579 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.666958 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.677773 12658 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:25.732576 12658 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.679248  9951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 218) took 20975 ms (client timeout 15000 ms). Trace:
I20260709 13:17:25.683194 12687 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.683939 12705 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260709 13:17:25.690750 12487 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:25.704825 12699 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.702646 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:25.748342 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.630204 12481 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:25.729387 12720 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:17:25.729951 12706 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:25.730772 12165 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:25.738350 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:25.688863 12639 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:25.716236 12718 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:25.716820 10122 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:25.749538 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47236.3us
I20260709 13:17:25.719210 12603 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:25.724412 12697 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:25.737722 12579 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:25.758970 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:25.750370  9953 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 6, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.735603 12620 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:25.771831 12635 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.597s	user 0.016s	sys 0.004s
I20260709 13:17:25.721276 12609 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:17:25.792552 10020 leader_election.cc:400] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5.
I20260709 13:17:25.786404 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:25.798427 12722 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:25.799294 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.800150 12626 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:17:25.801674 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:25.805971 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67720.4us
W20260709 13:17:25.831182 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:25.775023 12560 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:25.840099 10201 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:25.819231  9953 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.690657 12721 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.848232 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46654.4us
I20260709 13:17:25.728869 12690 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:17:25.869755 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 110937us
I20260709 13:17:25.877381  9953 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.842386 12635 raft_consensus.cc:2764] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:25.849001 12728 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:25.848343 12688 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:25.893769 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 163809us
I20260709 13:17:25.880271  9953 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index -1 to 3, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added, no longer has a pending config: VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65), VOTER e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66), VOTER d51ad94710c348889d417f7f00032948 (127.8.183.67), NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:25.901213  9953 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 6, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.901822 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:25.905607  9953 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 6, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.849174 12718 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.911993 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:25.873142 12603 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:25.921602 10050 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:17:25.917040 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 181121us
W20260709 13:17:25.923377 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 81909.2us
I20260709 13:17:25.918807  9953 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 3 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.932298 10051 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:25.909071 12688 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.886785 12487 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.951798 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52712.9us
W20260709 13:17:25.958866 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57191.4us
I20260709 13:17:25.961097 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.962970 10125 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:17:25.963711 10125 pending_rounds.cc:85] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:25.963996 10125 pending_rounds.cc:107] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:25.964291 10125 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.846866 12722 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 13:17:25.882817 12601 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:25.916219 12626 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:17:25.979319 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:25.942483 12688 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:25.939441 12718 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:25.931430 12609 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:17:25.995635 10124 raft_consensus.cc:1217] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[4.2-8.4]   Dedup: 4.2->[6.3-8.4]
I20260709 13:17:25.842543 12686 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.000588 10124 pending_rounds.cc:85] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:25.965639 12658 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:26.011749 10124 pending_rounds.cc:107] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.3
W20260709 13:17:26.000499 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 214243us
I20260709 13:17:25.965863 12732 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:25.940174 12635 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:17:25.953048 12482 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:17:26.046618 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67399us
I20260709 13:17:26.047528 10124 raft_consensus.cc:2889] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.887923 12481 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:25.735363  9951 rpcz_store.cc:269] 0709 13:17:04.544407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:05.375699 (+831292us) service_pool.cc:224] Handling call
0709 13:17:07.561496 (+2185797us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:17:25.519874 (+17958378us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:20.708423 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:20.956201 (+247778us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:20.956280 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:17:20.956310 (+    30us) tablet.cc:662] Decoding operations
0709 13:17:21.032115 (+ 75805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:21.032217 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:21.032248 (+    31us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:17:21.033838 (+  1590us) tablet.cc:701] Row locks acquired
0709 13:17:21.033892 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:17:21.034142 (+   250us) write_op.cc:270] Start()
0709 13:17:21.088331 (+ 54189us) write_op.cc:276] Timestamp: P: 1783603041034055 usec, L: 0
0709 13:17:21.088365 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:17:21.223747 (+135382us) log.cc:844] Serialized 7312 byte log entry
0709 13:17:22.630951 (+1407204us) op_driver.cc:464] REPLICATION: finished
0709 13:17:22.776221 (+145270us) write_op.cc:301] APPLY: starting
0709 13:17:22.777453 (+  1232us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:22.781800 (+  4347us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:22.781999 (+   199us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:22.784625 (+  2626us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:22.785062 (+   437us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:17:22.785114 (+    52us) write_op.cc:312] APPLY: finished
0709 13:17:22.785595 (+   481us) log.cc:844] Serialized 125 byte log entry
0709 13:17:22.785832 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:22.786086 (+   254us) write_op.cc:454] Released schema lock
0709 13:17:22.786178 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:22.852954 (+ 66776us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:22.859893 (+  6939us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":312,"mutex_wait_us":762424,"reported_tablets":31,"spinlock_wait_cycles":10634624,"child_traces":[["op",{"apply.queue_time_us":144869,"apply.run_cpu_time_us":5901,"apply.run_wall_time_us":60068,"num_ops":18,"prepare.queue_time_us":499112,"prepare.run_cpu_time_us":23746,"prepare.run_wall_time_us":1104723,"raft.queue_time_us":400489,"raft.run_cpu_time_us":14525,"raft.run_wall_time_us":88251,"replication_time_us":1542364,"thread_start_us":777055,"threads_started":4,"wal-append.queue_time_us":737649,"wal-append.run_cpu_time_us":52696,"wal-append.run_wall_time_us":1869815}]]}
I20260709 13:17:26.050045 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:26.050452 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52102.7us
I20260709 13:17:26.032626 12627 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:26.010872 12705 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.033804 12722 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.070453 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80382.5us
W20260709 13:17:26.074959 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 163077us
I20260709 13:17:26.000401 12639 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 13:17:26.049021 12603 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.091439 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:26.071147 12560 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:26.098937 12732 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:17:26.101888 12720 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:26.102125 12165 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:26.053175 10052 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:26.102860 12739 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:26.084964 12705 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.111016 12560 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:26.140089 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.126884 12635 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:26.136305 12579 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.126389 12165 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.165526 12544 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:26.167280 12729 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.134042 12718 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.156582 12560 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.134353 12721 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.087903 12722 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.102928 12639 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:17:26.172627 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:26.143960 12689 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:26.198027 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:26.206260 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 152349us
W20260709 13:17:26.207155 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 96284us
I20260709 13:17:26.123333 12720 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.170611 12635 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:26.247285 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.250792 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.251935 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:26.211295 12732 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:17:26.227972 12694 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.001s
I20260709 13:17:26.254348 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:26.256430 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99123.1us
W20260709 13:17:26.255957 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 416198us
I20260709 13:17:26.195914 12544 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:26.267480 12601 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.240870 12658 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.195457 12737 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.277652 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.226259 12609 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.229352 12731 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:26.280450 12699 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.290827 12726 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 13:17:26.259811 12689 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.267204 12706 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.310590 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56372.6us
I20260709 13:17:26.310814 12706 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.311034 12601 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:26.312635 12711 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:26.313288 12711 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.306030 12609 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.286864 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:26.313292 12706 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:26.315550 12609 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:26.314352 12711 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:26.310217 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:26.316593 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.319298 12689 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.319561 12750 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.320298 12750 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:26.320770 12750 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.322489 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:26.323401 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:26.324570 12579 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.324781 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:26.325239 12579 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.326342 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.327982 12741 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.331804 12741 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.332166 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:26.308905 12699 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.333405 12165 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.335796 12731 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:26.336553 12699 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:26.337074 12731 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.328847 12744 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:26.329694 12579 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.241720 12603 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.339272 12737 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.341766 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:26.332376 12741 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.327445 12544 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.257669 12732 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.272792 12735 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.349112 10056 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:26.348582 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.272166 12747 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:26.225838 12729 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:26.348598 12736 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 13:17:26.295957 12563 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:26.342000 12737 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.369443 10052 raft_consensus.cc:606] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:26.376807 10052 raft_consensus.cc:614] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:26.375686 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:26.375839 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124872us
I20260709 13:17:26.377599 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:26.381537 10056 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:26.357755 12744 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:26.394079 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 302828us
W20260709 13:17:26.379813 12743 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.171s	user 0.002s	sys 0.000s
I20260709 13:17:26.370654 12482 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.370373 12165 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.400477 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74194.5us
I20260709 13:17:26.409902 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.418479 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:26.419425 10122 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:26.420778 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.421514 10202 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:26.424315 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.425215 10123 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:26.426142 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:26.427314 10893 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:26.428647 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.428846 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:26.430742 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:26.432602 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:26.435253 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.436127 10128 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:26.439016 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:26.439445 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.405079 12744 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.408147 12544 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.406208 12627 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.443928 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.445254 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.459805 10197 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:26.446323 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:26.447767 10095 connection.cc:256] RPC call timeout handler was delayed by -1.5841092400000889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15760000000000002s for a response
I20260709 13:17:26.461510 10200 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:26.448041 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.409024 12563 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:26.449455 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.444573 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:26.373883 12735 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.415088 12482 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.478595 12482 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.478943 12687 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.432372 12165 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.479532 12687 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.481156 12165 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.481710 12687 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.484452 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.451994 12744 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:26.494454 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 55570us
I20260709 13:17:26.453819 12699 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.534839 10020 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:26.559473 10020 leader_election.cc:403] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:17:26.475574 12706 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.541501 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72691.9us
I20260709 13:17:26.390583 12635 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.476135 12747 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.569974 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:26.548311 12561 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.889s	user 0.017s	sys 0.000s
W20260709 13:17:26.535912 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50144.9us
I20260709 13:17:26.566018 10122 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:26.565824 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.492790 12165 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.568701 10890 tablet_service.cc:1558] Processing DeleteTablet for tablet 806f35e66c234d569cbafd8b3981518f with delete_type TABLET_DATA_TOMBSTONED (TS db052bef7536453fa2d7f5a00e314a18 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:58686
I20260709 13:17:26.494971 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:26.556034 10201 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:26.492331 12705 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:26.568763 12635 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.547256 12636 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:26.604314 10048 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:26.471375 12627 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.601814 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:26.548485 12639 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:26.452248 12658 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:26.653146 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51465.4us
I20260709 13:17:26.635831 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:25.604496 12680 consensus_queue.cc:1175] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 7 status { last_received { term: 7 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:26.655028 10055 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:26.647294 12754 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:26.646889 12636 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:26.576004 12706 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.567680 12699 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.688371 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:26.647009 12639 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:26.691721 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:26.629981 12561 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.699908 12615 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:26.706708 12682 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.706374 12711 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:26.685567 12579 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:26.715863 12720 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:26.716058 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:26.717958 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.723954 12437 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:26.726501 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:26.648499 12722 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:26.673187 12721 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:26.735379 12620 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.633443 12165 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:26.736668 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 45037.3us
I20260709 13:17:26.682489 12758 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.713097 10896 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:26.748353 10055 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:26.727627 12636 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:26.815207 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:26.738328 12757 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:17:26.647408 12563 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.731052 12639 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:26.827134 10900 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:26.827765 10122 raft_consensus.cc:2376] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10. Re-sending same reply.
I20260709 13:17:26.828536 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:26.768905 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 234208us
I20260709 13:17:26.729015 10048 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:26.734532 12615 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.855581 10020 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:26.736423 12754 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:26.820884 12437 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:26.861685 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128975us
I20260709 13:17:26.833560 12721 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.847019 12620 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.749995 12682 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.833721 12579 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.896354 10205 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:26.837390 12764 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.460s
I20260709 13:17:26.850194 12705 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:26.971907 12763 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:17:27.009476 12487 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:26.973266 12784 tablet_replica.cc:333] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:17:27.012110 12699 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:26.909971 12721 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:26.815704 12747 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.967936 12731 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:27.014437 12771 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:17:27.016738 12579 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.019376 12735 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:26.893770 12768 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:27.021035 12165 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.021579 12764 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:27.022423 12780 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:27.023602 12437 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:26.968461 12738 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:26.895449 12750 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.009s
I20260709 13:17:27.031471 12776 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:27.084360 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.012324 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1297) took 1080 ms. Trace:
W20260709 13:17:27.033388 12765 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.458s	user 0.001s	sys 0.001s
I20260709 13:17:27.033951 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:27.034464 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.012545 10205 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:27.033241 10896 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:27.034137 10200 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:26.945814 12770 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:27.046260 10196 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:27.047325 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1302) took 1124 ms. Trace:
W20260709 13:17:27.019894 12766 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:17:27.094728 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:26.973483 12740 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.052613 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:27.053866 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 489412us
W20260709 13:17:27.094401 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 238899us
I20260709 13:17:27.065013 12784 raft_consensus.cc:2243] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:27.044759 12560 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.001s
I20260709 13:17:26.864535 12758 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.110165 10055 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82960478409442549781547254b9d8ea"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:26.973615 12761 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.324s
I20260709 13:17:27.050422 12731 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.086176 10124 rpcz_store.cc:276] 0709 13:17:25.931724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.932490 (+   766us) service_pool.cc:224] Handling call
0709 13:17:25.955779 (+ 23289us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:26.670023 (+714244us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b180044fa40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa277ffcbfb 00007fa27800764b
0709 13:17:26.670231 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:26.670266 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:26.671061 (+   795us) log.cc:844] Serialized 52 byte log entry
0709 13:17:26.877082 (+206021us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:26.879395 (+  2313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.879557 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.011856 (+132299us) raft_consensus.cc:1685] finished
0709 13:17:27.011878 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.012211 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1100,"raft.queue_time_us":116789,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":2431,"spinlock_wait_cycles":52127616,"thread_start_us":260495,"threads_started":2,"wal-append.queue_time_us":314329}
I20260709 13:17:27.091770 10054 rpcz_store.cc:276] 0709 13:17:25.912150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.919104 (+  6954us) service_pool.cc:224] Handling call
0709 13:17:25.946639 (+ 27535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:25.957639 (+ 11000us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:26.122205 (+164566us) log.cc:844] Serialized 14 byte log entry
0709 13:17:26.271362 (+149157us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.368874 (+ 97512us) log.cc:844] Serialized 556 byte log entry
0709 13:17:26.369186 (+   312us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:26.369243 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:26.369375 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.036046 (+666671us) raft_consensus.cc:1685] finished
0709 13:17:27.036070 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.036209 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61753,"prepare.queue_time_us":279240,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":7107,"raft.queue_time_us":131519,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":10745,"thread_start_us":238667,"threads_started":3,"wal-append.queue_time_us":166320}
I20260709 13:17:27.098032 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:27.026499 12705 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:27.109254 12697 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.325s	user 0.018s	sys 0.000s
I20260709 13:17:27.111558 10055 raft_consensus.cc:606] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:27.133773 10055 raft_consensus.cc:614] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:27.114017 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.070816 12780 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.117290 12740 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:26.727516 12762 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Committing config change with OpId 6.5: config changed from index 3 to 5, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:27.119728 12759 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:27.121109 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:27.164753 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 43732us
I20260709 13:17:27.165918 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:27.166611 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:27.167434 10051 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:27.123008 12753 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:27.124507 12741 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.000s
I20260709 13:17:27.124241 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.110101 12706 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.125821 12682 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.127501 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:27.128069 10895 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:27.132797 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:27.137202 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:27.145972 12775 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:17:27.095490 12776 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:27.216506 12615 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.142903 12777 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.223767 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:27.173529 12784 pending_rounds.cc:70] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Trying to abort 1 pending ops.
I20260709 13:17:27.097079 12750 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:27.085067 12764 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.136545 12731 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.161342 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.168604 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:27.108381 12561 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.171881 10896 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:27.099687 12686 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.168164 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:27.192530 10051 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:27.085465 12735 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:17:27.203689 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:27.204667 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67525.7us
W20260709 13:17:27.255468 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87414.7us
I20260709 13:17:27.154026 12697 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:27.213094 12717 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.213434 12734 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:27.215354 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:27.215466 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:27.218547 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 91166.6us
I20260709 13:17:27.108043 12770 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.139132 12780 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:27.172422 12437 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.149405 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:27.171689 12626 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:27.252013 12635 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.002s
I20260709 13:17:27.234454 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.117313 12560 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.224112 12777 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:27.210820 12544 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.003s
I20260709 13:17:27.242689 10197 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:27.242759 10049 raft_consensus.cc:2410] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:17:27.245144 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:27.246398 12746 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:27.223942 12615 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.247093 12639 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.141048 12769 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:17:27.252921 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.301476 10898 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:27.302681 10893 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:27.207258 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 188) took 1038 ms. Trace:
I20260709 13:17:27.307650 10198 rpcz_store.cc:276] 0709 13:17:26.168804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.173340 (+  4536us) service_pool.cc:224] Handling call
0709 13:17:26.224950 (+ 51610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:26.272768 (+ 47818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.272804 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:26.272856 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:26.272890 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.206773 (+933883us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:27.206873 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.207155 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109355904}
I20260709 13:17:27.308626 12639 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.309653 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.271957 12737 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:17:27.255625 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:27.256487 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:27.229105 12794 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:27.312464 12740 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:27.316113 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60585.3us
I20260709 13:17:27.317121 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:27.317663 12729 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.319075 12697 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:27.319988 12697 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:27.318593 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:27.323407 12740 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:27.277211 12777 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.324743 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:27.325870 12740 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:27.329243 10123 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:27.262362 12686 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:27.336189 12688 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.298s	user 0.012s	sys 0.004s
I20260709 13:17:27.223841 12791 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.340528 12791 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:27.267215 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.282770 12764 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.343554 10899 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:27.229259 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.345345 12791 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.261030 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:27.270684 10200 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:27.274262 10125 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:27.200078 12741 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.209738 12620 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.282725 12560 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.286752 12732 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.264861 12717 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:27.388500 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.396296 10899 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:27.399693 12772 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.704s	user 0.000s	sys 0.005s
I20260709 13:17:27.403223 12620 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.404194 12794 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:27.403710 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:27.405869 12739 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:27.406502 12739 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:27.289490 12792 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:17:27.408044 12739 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.409132 12741 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.410288 12732 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:27.264310 12748 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.292203 10124 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:27.412424 12748 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.296277 10047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87da05694091470284f66a8181207e2f"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:27.413235 12748 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.413599 10047 raft_consensus.cc:606] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:27.414181 10047 raft_consensus.cc:614] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:27.275552 12561 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:17:27.226985 12784 pending_rounds.cc:77] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Aborting op as it isn't in flight: id { term: 6 index: 5 } timestamp: 7305638059583778816 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "806f35e66c234d569cbafd8b3981518f" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } } new_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } } }
I20260709 13:17:27.415800 12784 raft_consensus.cc:2917] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Aborting config change with OpId 6.5: Aborted: Op aborted
I20260709 13:17:27.416355 12784 raft_consensus.cc:2889] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:17:27.416788 12784 raft_consensus.cc:2272] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Raft consensus is shut down!
I20260709 13:17:27.299575 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.270833 12734 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.419490 12734 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.311002 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.420683 10198 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:27.398419 12717 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.337009 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:27.444527 10196 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:27.338868 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.237202 12790 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:27.346312 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1314) took 1015 ms. Trace:
W20260709 13:17:27.350051 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89094.5us
I20260709 13:17:27.468532 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:27.219202 12776 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.388213 12627 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:27.320169 12746 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:27.389549 10204 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:17:27.477458 10204 pending_rounds.cc:85] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 4
I20260709 13:17:27.477901 10204 pending_rounds.cc:107] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:17:27.467269 10053 rpcz_store.cc:276] 0709 13:17:26.330957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.337439 (+  6482us) service_pool.cc:224] Handling call
0709 13:17:26.337903 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.719208 (+381305us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b180040dc20. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa277ffcbfb 00007fa27800764b
0709 13:17:26.719366 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.719393 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.732912 (+ 13519us) log.cc:844] Serialized 26 byte log entry
0709 13:17:27.062063 (+329151us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:27.062105 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.062191 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:27.345553 (+283362us) raft_consensus.cc:1685] finished
0709 13:17:27.345578 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:27.346118 (+   540us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32674,"raft.queue_time_us":94313,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5609,"spinlock_wait_cycles":14057216,"thread_start_us":336979,"threads_started":2,"wal-append.queue_time_us":501295}
I20260709 13:17:27.389303 12688 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:27.391585 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 135181us
I20260709 13:17:27.398429 12737 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.090404 12754 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.332552 12758 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:27.371930 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:27.563026 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 191198us
I20260709 13:17:27.464339 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:27.565706 10900 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:27.400204 12791 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:27.473814 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:27.478307 10204 raft_consensus.cc:2889] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.372813 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:27.311546 12750 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.480314 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:27.393528 12764 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:27.472609 12790 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:27.489007 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:27.295789 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:27.467738 12744 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.001s
I20260709 13:17:27.496660 10203 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:27.553817 10196 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:27.610482 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:27.488863 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:27.946971 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 382346us
W20260709 13:17:27.938997 10863 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.851s (SENT)
I20260709 13:17:27.953138 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:27.957319 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:27.641376 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:27.962452 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 473612us
I20260709 13:17:27.494630 12563 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:27.564250 12718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.034s	sys 0.037s
W20260709 13:17:27.954933 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 474682us
I20260709 13:17:27.971431 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:27.964515 10204 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:27.479354 12717 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:27.968327 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:27.977250 12717 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.513916 12737 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.980074 12737 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.980540 12717 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:27.892645 12762 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 13:17:27.981305 12717 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:27.889014 12636 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.008s	user 0.011s	sys 0.003s
I20260709 13:17:27.981798 12717 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.982191 12636 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.983001 12800 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:27.983707 12800 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.984246 12800 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.888407 12747 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.524513 12686 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:27.498278 12741 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:27.866963 12776 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:27.621851 12785 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.856204 12758 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:27.967486 12739 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.002861 10202 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:27.849030 12688 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:27.982221 12717 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:27.975265 12735 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:27.976173 12718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.103s	sys 0.128s
I20260709 13:17:27.607080 12620 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:27.562734 12406 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.441s
I20260709 13:17:27.966797 12639 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:28.011781 10202 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:17:28.043701 12639 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:28.044265 12639 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:28.045377 10202 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:27.974963 12706 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:28.047308 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:28.049623 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:28.009591 12776 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:27.969389 12790 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:28.006343 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 189) took 1595 ms (client timeout 2003 ms). Trace:
W20260709 13:17:28.059957 10201 rpcz_store.cc:269] 0709 13:17:26.403658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.409406 (+  5748us) service_pool.cc:224] Handling call
0709 13:17:27.998318 (+1588912us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400217fd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:27.998450 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333477760}
I20260709 13:17:27.623525 12682 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.385s
I20260709 13:17:28.047281 12706 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:28.005343 12721 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.944s	user 0.012s	sys 0.004s
I20260709 13:17:28.008818 12747 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.018272 12739 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:28.018915 12797 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:17:28.018936 12799 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:17:28.028023 12746 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.028760 10199 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:28.012509 12758 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.081482 12747 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.017885 12735 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.062855 12682 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.028434 12737 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.028599 12785 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.010417 12764 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 13:17:28.022025 12487 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.978s	user 0.008s	sys 0.004s
I20260709 13:17:27.993906 12636 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:28.008306 12784 ts_tablet_manager.cc:1916] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:28.083204 10200 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:27.642325 12791 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.062911 12706 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:28.038223 12780 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.000s
I20260709 13:17:28.058784 12776 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:28.096693 12615 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.001s
I20260709 13:17:28.093073 12785 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.071895 12750 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.088322 12735 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:28.107949 10895 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:28.093420 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:28.083045 12758 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.086227 12739 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.011642 12686 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:27.970969 12744 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.140237 12791 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.114174 12421 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.090498 12682 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.142196 12744 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.142979 10126 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:28.143450 10893 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:28.151563 12639 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.144164 12421 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.122071 12780 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.154666 12639 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.155669 12780 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.150350 12682 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.160248 12737 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:28.160650 12800 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.011826 12741 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.214529 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:28.241326 12741 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.088028 12721 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.171813 10130 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:28.244467 10204 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:17:28.246187 12731 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.982s	user 0.005s	sys 0.009s
I20260709 13:17:28.247560 12731 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:28.248977 12777 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.924s	user 0.015s	sys 0.001s
I20260709 13:17:28.249984 12731 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.249605 12777 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.246781 12721 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:28.190337 12748 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.776s	user 0.013s	sys 0.000s
W20260709 13:17:28.195989 12803 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 13:17:28.254534 12800 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:28.205605 12793 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.875s	user 0.011s	sys 0.009s
I20260709 13:17:28.135627 12750 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.273525 12750 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:28.275638 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1300) took 2286 ms. Trace:
I20260709 13:17:28.276182 10127 rpcz_store.cc:276] 0709 13:17:25.988861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.011980 (+ 23119us) service_pool.cc:224] Handling call
0709 13:17:26.045414 (+ 33434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:26.893776 (+848362us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b180043f660. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa277ffcbfb 00007fa27800764b
0709 13:17:26.893983 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:26.894023 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:26.945532 (+ 51509us) log.cc:844] Serialized 26 byte log entry
0709 13:17:27.070101 (+124569us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:27.070148 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.070235 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.204634 (+1134399us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:28.275335 (+ 70701us) raft_consensus.cc:1685] finished
0709 13:17:28.275361 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.275527 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17895,"raft.queue_time_us":78659,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":9382,"spinlock_wait_cycles":57788160,"thread_start_us":223980,"threads_started":2,"wal-append.queue_time_us":330592}
I20260709 13:17:28.212149 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:28.216305 12732 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.805s	user 0.012s	sys 0.002s
I20260709 13:17:28.280237 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 482) took 1027 ms. Trace:
I20260709 13:17:28.280747 10894 rpcz_store.cc:276] 0709 13:17:27.252752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.254539 (+  1787us) service_pool.cc:224] Handling call
0709 13:17:28.211834 (+957295us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:28.280146 (+ 68312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40229248}
W20260709 13:17:28.226591 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2625) took 1381 ms (client timeout 1614 ms). Trace:
I20260709 13:17:28.231174 12705 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:28.238243 12734 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.005s
I20260709 13:17:28.238181 12790 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.252691 12717 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.253535 12777 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:28.246646 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:28.200268 12804 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:17:28.190929 10895 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:28.255995 12748 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.258718 12544 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.260555 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:28.322955 10197 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:28.262830 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:28.324291 10201 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:28.264303 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:28.266750 12706 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:28.268883 12735 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:28.321084 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 316320us
I20260709 13:17:28.321094 12421 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:28.288462 10126 rpcz_store.cc:269] 0709 13:17:26.844874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.069975 (+225101us) service_pool.cc:224] Handling call
0709 13:17:28.226231 (+1156256us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000a00d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:28.226463 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408023936}
W20260709 13:17:28.315589 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 69031.3us
I20260709 13:17:28.319377 12705 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:28.327437 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 838472us
I20260709 13:17:28.332370 10202 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:28.332265 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:28.328586 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 356425us
I20260709 13:17:28.334808 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:28.334020 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 395057us
I20260709 13:17:28.354979 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:28.368959 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:28.370808 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:28.371459 10048 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:28.317001 12734 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.380631 10128 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:28.382721 12748 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:28.340780 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:28.425923 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:28.328711 12544 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.336941 12705 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:28.343456 12794 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:28.363289 12750 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.560297 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:28.347719 12686 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.177s
W20260709 13:17:28.598925 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 477731us
I20260709 13:17:28.561398 10127 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:28.348055 12785 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.201s
I20260709 13:17:28.354771 12731 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.195s
I20260709 13:17:28.550261 12705 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:28.655253 12747 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.655645 12620 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.052632 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 361) took 1567 ms. Trace:
I20260709 13:17:29.121266 12620 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:29.121178 10899 rpcz_store.cc:276] 0709 13:17:27.464312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.478946 (+ 14634us) service_pool.cc:224] Handling call
0709 13:17:27.486168 (+  7222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:27.486637 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:27.486672 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:27.491961 (+  5289us) log.cc:844] Serialized 26 byte log entry
0709 13:17:27.831917 (+339956us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:27.832067 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:27.832202 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:28.381300 (+549098us) raft_consensus.cc:1685] finished
0709 13:17:28.381336 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:28.862284 (+480948us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"thread_start_us":209753,"threads_started":1,"wal-append.queue_time_us":638898}
W20260709 13:17:29.075573 12817 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:17:29.124307 12609 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:28.826885 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:29.125254 12609 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:29.126042 10197 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:29.127707 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:29.128118 12609 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:29.031075 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:29.128952 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:29.129078 10128 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:28.784272 10126 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:28.570225 12794 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:28.593169 10891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.010s
I20260709 13:17:28.566977 12718 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:28.569870 12750 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:28.828058 12812 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:28.597548 12773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.598s	user 0.017s	sys 0.029s
W20260709 13:17:28.627403 12754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.040s	sys 0.050s
I20260709 13:17:29.076066 12770 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:29.030782 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:29.034713 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:29.102568 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:29.103672 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 759138us
W20260709 13:17:29.106720 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 460993us
W20260709 13:17:29.087888 12801 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.751s	user 0.002s	sys 0.000s
W20260709 13:17:29.109479 10865 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.007s	sys 0.035s
I20260709 13:17:29.081712 10196 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:29.117836 12813 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:29.071339 12165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.013s	sys 0.037s
I20260709 13:17:29.119524 12741 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:29.094059 12790 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:29.076946 12717 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:29.130451 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2602) took 1975 ms. Trace:
I20260709 13:17:29.094419 12706 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:29.489744 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:29.488739 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:29.228837 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 186) took 2510 ms (client timeout 2003 ms). Trace:
I20260709 13:17:29.491159 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1524) took 1206 ms. Trace:
I20260709 13:17:29.491909 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2601) took 2374 ms. Trace:
I20260709 13:17:29.490998 10128 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:29.492655 10865 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.010s	sys 0.059s
I20260709 13:17:29.491771 10197 rpcz_store.cc:276] 0709 13:17:28.284254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.129874 (+845620us) service_pool.cc:224] Handling call
0709 13:17:29.491061 (+361187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:29.490983 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1311) took 1153 ms (client timeout 1537 ms). Trace:
W20260709 13:17:29.493356 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.29848e+06us
W20260709 13:17:29.493590 10124 rpcz_store.cc:269] 0709 13:17:28.337896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.338039 (+   143us) service_pool.cc:224] Handling call
0709 13:17:29.490901 (+1152862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:29.494464 10865 connection.cc:256] RPC call timeout handler was delayed by -1.2682626850000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
I20260709 13:17:29.492672 10196 rpcz_store.cc:276] 0709 13:17:27.117148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.080858 (+1963710us) service_pool.cc:224] Handling call
0709 13:17:29.081426 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:29.491492 (+410066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:29.491845 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:29.181627 12806 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.219413 12806 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.220525 10196 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:30.221323 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1525) took 1936 ms. Trace:
I20260709 13:17:30.221680 10196 rpcz_store.cc:276] 0709 13:17:28.284502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.219906 (+1935404us) service_pool.cc:224] Handling call
0709 13:17:30.220311 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.221046 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.221262 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:30.222369 12806 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:29.490056 12773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.522s	user 0.028s	sys 0.033s
I20260709 13:17:29.491218 12812 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.320107 12812 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:29.517393 12809 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.324064 12809 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:29.515218 12813 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:29.610598 12705 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.006s
W20260709 13:17:29.491040 10125 rpcz_store.cc:269] 0709 13:17:26.718364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.214796 (+496432us) service_pool.cc:224] Handling call
0709 13:17:29.228450 (+2013654us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:29.228711 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34407296}
I20260709 13:17:30.330658 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:29.785537 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:29.141002 10891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.003s	sys 0.011s
W20260709 13:17:29.588933 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.987s (SENT) [suppressed 12 similar messages]
W20260709 13:17:30.467917 12802 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 2.334s	user 0.007s	sys 0.009s
W20260709 13:17:30.011580 10123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.015s	sys 0.030s
I20260709 13:17:30.012596 12827 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.469697 10123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.016s	sys 0.030s
I20260709 13:17:30.469887 12827 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.061882 12620 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.071648 10205 rpcz_store.cc:276] 0709 13:17:27.154852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.965609 (+810757us) service_pool.cc:224] Handling call
0709 13:17:27.966002 (+   393us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:27.969286 (+  3284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:27.969328 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:27.976900 (+  7572us) log.cc:844] Serialized 78 byte log entry
0709 13:17:28.335769 (+358869us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:28.335845 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:28.335965 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:29.130154 (+794189us) raft_consensus.cc:1685] finished
0709 13:17:29.130179 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:29.130362 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3807,"raft.queue_time_us":9236,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":9680,"thread_start_us":217784,"threads_started":1,"wal-append.queue_time_us":465557}
I20260709 13:17:30.076598 12832 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:30.077666 12833 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:30.077811 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:17:30.079819 10095 leader_election.cc:343] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Tablet error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:30.475759 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 395960us
W20260709 13:17:29.489871 12165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.016s	sys 0.053s
W20260709 13:17:29.325145 12421 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.000s	sys 0.084s
I20260709 13:17:30.092453 12735 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:29.233628 12777 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.480389 12421 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.955s	user 0.002s	sys 0.097s
I20260709 13:17:30.480476 12777 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.140733 12750 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:29.350438 12544 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:30.166301 12784 ts_tablet_manager.cc:1929] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.5
I20260709 13:17:30.170957 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:29.181393 12810 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.483822 12784 log.cc:1199] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Deleting WAL directory at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0/minicluster-data/ts-3-root/wals/806f35e66c234d569cbafd8b3981518f
W20260709 13:17:30.164178 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.006s	sys 0.009s
W20260709 13:17:29.233794 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.025s	sys 0.044s
W20260709 13:17:30.484720 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.010s	user 0.043s	sys 0.053s
I20260709 13:17:30.484442 12810 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.485154 12840 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:30.485641 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1328) took 2146 ms. Trace:
I20260709 13:17:30.486492 12777 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.260547 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:30.275862 10898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.600s	user 0.004s	sys 0.009s
W20260709 13:17:30.254599 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2628) took 3408 ms (client timeout 2200 ms). Trace:
W20260709 13:17:29.324880 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.024s	sys 0.059s
W20260709 13:17:30.285210 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1507) took 3559 ms (client timeout 1987 ms). Trace:
I20260709 13:17:30.486992 12810 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.290915 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2603) took 3128 ms. Trace:
I20260709 13:17:30.294339 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1316) took 1644 ms. Trace:
I20260709 13:17:29.173125 12718 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:30.487702 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.122s	user 0.031s	sys 0.070s
I20260709 13:17:30.488492 10897 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:30.484040 10129 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:17:30.489442 10897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 485) took 3234 ms (client timeout 1906 ms). Trace:
I20260709 13:17:30.489724 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1320) took 1274 ms. Trace:
W20260709 13:17:30.487797 10898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.837s	user 0.005s	sys 0.009s
W20260709 13:17:29.502003 12754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.486s	user 0.040s	sys 0.063s
W20260709 13:17:30.306690 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.056s	user 0.030s	sys 0.030s
W20260709 13:17:30.488780 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 2.12343e+06us
W20260709 13:17:30.484483 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.006s	sys 0.010s
I20260709 13:17:30.485702 12840 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.486377 10053 rpcz_store.cc:276] 0709 13:17:28.339364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.340008 (+   644us) service_pool.cc:224] Handling call
0709 13:17:28.372582 (+ 32574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.373134 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:28.373175 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:28.396793 (+ 23618us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.485178 (+2088385us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:30.485241 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.485320 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.485365 (+    45us) raft_consensus.cc:1685] finished
0709 13:17:30.485377 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.485540 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11242,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":81567,"thread_start_us":2010051,"threads_started":1,"wal-append.queue_time_us":1081471}
I20260709 13:17:30.492779 12840 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:30.492935 10863 connection.cc:256] RPC call timeout handler was delayed by -1.9664438679999421s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
I20260709 13:17:30.328243 12705 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.495499 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:17:30.490104 10897 rpcz_store.cc:269] 0709 13:17:27.255398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.258284 (+  2886us) service_pool.cc:224] Handling call
0709 13:17:30.260224 (+3001940us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:30.489339 (+229115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10493184}
I20260709 13:17:30.335879 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 191) took 1160 ms. Trace:
I20260709 13:17:30.496518 12686 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:29.132088 12609 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.497354 12686 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:30.487931 10130 rpcz_store.cc:269] 0709 13:17:26.845878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.244764 (+398886us) service_pool.cc:224] Handling call
0709 13:17:30.254309 (+3009545us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:30.254470 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1959660928}
I20260709 13:17:30.498055 12609 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.499439 12544 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:30.499715 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 489) took 2545 ms. Trace:
I20260709 13:17:30.500087 12544 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.500908 10129 rpcz_store.cc:276] 0709 13:17:29.215065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.170559 (+955494us) service_pool.cc:224] Handling call
0709 13:17:30.489641 (+319082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:30.501302 12609 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.342494 12813 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.343998 12809 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.350963 10199 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:30.503141 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2614) took 2163 ms (client timeout 1670 ms). Trace:
I20260709 13:17:30.503351 12165 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.503489 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.490635 10204 rpcz_store.cc:276] 0709 13:17:27.162650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.972995 (+810345us) service_pool.cc:224] Handling call
0709 13:17:27.973314 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.245038 (+271724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.290355 (+2045317us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54002de1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:30.290793 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622862720}
W20260709 13:17:30.503839 10199 rpcz_store.cc:269] 0709 13:17:28.340005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.371360 (+ 31355us) service_pool.cc:224] Handling call
0709 13:17:30.502992 (+2131632us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:30.496476 10205 raft_consensus.cc:606] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:30.357012 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 190) took 3193 ms. Trace:
I20260709 13:17:30.505245 10205 raft_consensus.cc:614] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:30.359709 12829 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.365156 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 190) took 1227 ms. Trace:
I20260709 13:17:30.365993 12812 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.506852 12829 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.509696 12829 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:30.500099 10898 rpcz_store.cc:276] 0709 13:17:27.954556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.954703 (+   147us) service_pool.cc:224] Handling call
0709 13:17:27.994143 (+ 39440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.378656 (+384513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:28.378715 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:28.453605 (+ 74890us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.490774 (+2037169us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:30.490819 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.490902 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.490956 (+    54us) raft_consensus.cc:1685] finished
0709 13:17:30.490970 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.491104 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130065,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":7326,"thread_start_us":2214769,"threads_started":2,"wal-append.queue_time_us":939966}
I20260709 13:17:30.509858 12812 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.367350 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 189) took 1235 ms. Trace:
I20260709 13:17:30.504051 12165 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.376019 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1519) took 2111 ms. Trace:
I20260709 13:17:30.376880 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1319) took 1232 ms. Trace:
I20260709 13:17:30.510910 10893 raft_consensus.cc:1803] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: voting while tombstoned based on last-logged opid 6.5
W20260709 13:17:30.487743 10203 rpcz_store.cc:269] 0709 13:17:26.726040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.726170 (+   130us) service_pool.cc:224] Handling call
0709 13:17:30.284907 (+3558737us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54001883d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:30.285083 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064309504}
I20260709 13:17:30.511968 10893 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:30.374356 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:30.490298 12718 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:30.513118 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 138880us
W20260709 13:17:30.513995 10169 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:30.381642 12798 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 3.050s	user 0.015s	sys 0.041s
I20260709 13:17:29.165362 12747 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.387815 12717 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.516650 12165 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:30.517416 12717 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.518613 10202 rpcz_store.cc:276] 0709 13:17:28.264247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.264815 (+   568us) service_pool.cc:224] Handling call
0709 13:17:28.265201 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:28.337699 (+ 72498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.375713 (+2038014us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b54001d0454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:30.375943 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153854208}
I20260709 13:17:30.502342 12544 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:30.505463 10055 rpcz_store.cc:276] 0709 13:17:27.163655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.169168 (+  5513us) service_pool.cc:224] Handling call
0709 13:17:27.220578 (+ 51410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:27.612788 (+392210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:27.612827 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:27.612878 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:27.612912 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.356636 (+2743724us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54002e95d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:30.356698 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.356877 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901413120}
I20260709 13:17:30.400256 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:30.496605 10128 rpcz_store.cc:276] 0709 13:17:29.175204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.334835 (+1159631us) service_pool.cc:224] Handling call
0709 13:17:30.335277 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.335486 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.335522 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.335574 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:30.335616 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.335679 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.335808 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:30.400570 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.506896 10131 rpcz_store.cc:276] 0709 13:17:29.137465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.138217 (+   752us) service_pool.cc:224] Handling call
0709 13:17:30.364159 (+1225942us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:30.364468 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.364719 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.364747 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.364796 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:30.364843 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.364909 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.365067 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010100480}
I20260709 13:17:30.402525 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:30.532339 10051 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:29.132572 12785 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 1.283s
I20260709 13:17:30.432286 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:29.254205 12734 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:30.440466 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:30.539994 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.540376 10896 raft_consensus.cc:1803] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: voting while tombstoned based on last-logged opid 6.5
I20260709 13:17:30.541115 10896 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Advancing to term 7
W20260709 13:17:30.440833 10895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 490) took 2473 ms (client timeout 2776 ms). Trace:
I20260709 13:17:30.441542 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:30.442564 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.446367 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.550403 10050 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:30.447085 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.329s	user 0.017s	sys 0.029s
I20260709 13:17:30.448779 12794 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.462697 10197 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:17:30.472776 12827 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:30.475899 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 398159us
W20260709 13:17:30.563874  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800035ae0 after lost signal to thread 8925
W20260709 13:17:30.564424  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800035c40 after lost signal to thread 8928
I20260709 13:17:30.564940 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:30.509697 12686 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:30.565929  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800035a60 after lost signal to thread 9988
I20260709 13:17:30.484001 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.248678 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.520898 12717 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:30.580583 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 188) took 1449 ms (client timeout 1681 ms). Trace:
W20260709 13:17:30.581264 10126 rpcz_store.cc:269] 0709 13:17:29.131487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.170557 (+ 39070us) service_pool.cc:224] Handling call
0709 13:17:30.248523 (+1077966us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:30.580492 (+331969us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624461824}
W20260709 13:17:30.282675 12755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.980s	user 0.022s	sys 0.036s
I20260709 13:17:29.353461 12731 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.582736 10894 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:30.583331 12731 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:30.585048 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 492) took 1975 ms (client timeout 2590 ms). Trace:
W20260709 13:17:30.585574 10894 rpcz_store.cc:269] 0709 13:17:28.609828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.627013 (+ 17185us) service_pool.cc:224] Handling call
0709 13:17:30.584754 (+1957741us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54001e5854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:30.584968 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400887168}
I20260709 13:17:30.586171 12731 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:30.491181 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.031s	sys 0.030s
I20260709 13:17:30.307683 12753 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.587193 12753 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:30.587819 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.588678 10899 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:17:30.549187 12437 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:30.590236 12437 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.496042 12705 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.591635 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.334964 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 186) took 2363 ms. Trace:
I20260709 13:17:30.591948 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.592785 12437 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:30.593221 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:30.593416 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:30.593591 12800 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.593767 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.594430 10126 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:30.594272 12800 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.594521 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:30.507457 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.595029 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.595245 12718 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:30.373392 10168 connection.cc:256] RPC call timeout handler was delayed by -2.5019402100001571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
W20260709 13:17:30.596292 10168 connection.cc:256] RPC call timeout handler was delayed by -2.4794402100001207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
I20260709 13:17:30.502148 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.596705 12734 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.374609 12721 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.597175 10129 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:30.597294 12734 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.597592 12721 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.598358 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.565730 12747 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.390724 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.600227 10047 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.395167 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.601614 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:30.395591 10200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2611) took 3029 ms (client timeout 1591 ms). Trace:
I20260709 13:17:30.602279 10052 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:30.602578 10200 rpcz_store.cc:269] 0709 13:17:27.366280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:27.479993 (+113713us) service_pool.cc:224] Handling call
0709 13:17:30.395363 (+2915370us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:30.395504 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284333440}
I20260709 13:17:30.511435 10125 rpcz_store.cc:276] 0709 13:17:29.144636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.331942 (+1187306us) service_pool.cc:224] Handling call
0709 13:17:30.332363 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.332844 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.332881 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.333474 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.373799 (+ 40325us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:30.373846 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.373919 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.376582 (+  2663us) raft_consensus.cc:1685] finished
0709 13:17:30.376611 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.376807 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51896,"raft.queue_time_us":3685,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":52067,"thread_start_us":39818,"threads_started":1,"wal-append.queue_time_us":39376}
I20260709 13:17:30.488276 10127 rpcz_store.cc:276] 0709 13:17:28.649975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.650273 (+   298us) service_pool.cc:224] Handling call
0709 13:17:28.783438 (+133165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:29.187874 (+404436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:29.187925 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:29.187995 (+    70us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:29.188040 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.293946 (+1105906us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54001270d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:30.294021 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.294246 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929264512}
W20260709 13:17:30.525424 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2720316229999753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:17:30.511385 10122 rpcz_store.cc:276] 0709 13:17:29.136628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.139312 (+  2684us) service_pool.cc:224] Handling call
0709 13:17:30.366299 (+1226987us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:30.366539 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.366781 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.366827 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.366892 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:30.366946 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.367037 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.367252 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8654720}
W20260709 13:17:30.431293 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 935763us
I20260709 13:17:30.530959 10056 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:30.608867 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:30.541863 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.546834 10900 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:30.548128 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 959143us
I20260709 13:17:30.559126 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1523) took 2275 ms. Trace:
I20260709 13:17:30.611840 10197 rpcz_store.cc:276] 0709 13:17:28.283975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.461923 (+2177948us) service_pool.cc:224] Handling call
0709 13:17:30.462426 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.558669 (+ 96243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.559033 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:30.559773 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:30.562114 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.445s	user 0.018s	sys 0.029s
I20260709 13:17:30.613269 10055 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.614365 10048 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:30.564111 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.607393 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:30.567680 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:30.575938  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800036d40 after lost signal to thread 10080
W20260709 13:17:30.616398  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800036dc0 after lost signal to thread 10081
W20260709 13:17:30.596985 10168 connection.cc:256] RPC call timeout handler was delayed by -2.16018401600013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 13:17:30.617147 10168 connection.cc:256] RPC call timeout handler was delayed by -2.0242840160001379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:17:30.616227 10205 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:30.618170  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080031ee60 after lost signal to thread 10155
W20260709 13:17:30.618608  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080030cd00 after lost signal to thread 10156
I20260709 13:17:30.549412 12706 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:30.595889 10124 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:30.596220 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:30.631157 10130 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.596866 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.632719 10053 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:30.485795 12784 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58686 (request call id 0) took 4030 ms. Trace:
I20260709 13:17:30.517647 12165 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:30.643930 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.645011 10125 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:30.534511 12735 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.616580 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:30.582823 12755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.281s	user 0.023s	sys 0.036s
I20260709 13:17:30.654403 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:30.653927 10128 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:30.599016 10129 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:30.629530  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08003d92a0 after lost signal to thread 10229
I20260709 13:17:30.599650 12734 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:30.638998 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71945.4us
I20260709 13:17:30.639480 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:30.639765 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1398603489999459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:17:30.673349 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:30.640558 10168 connection.cc:256] RPC call timeout handler was delayed by -1.5385798209999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 13:17:30.674554 10168 connection.cc:256] RPC call timeout handler was delayed by -1.3666798209999342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:17:30.675421 10050 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:30.389308 12764 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.655500  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08003d9480 after lost signal to thread 10230
I20260709 13:17:30.553236 12829 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:30.676616 10051 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:30.681054 10019 leader_election.cc:400] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:30.719199 10019 leader_election.cc:403] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:30.684172  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800302fe0 after lost signal to thread 10924
I20260709 13:17:30.690349 10896 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 7 LEARNER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:17:30.720131  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800380380 after lost signal to thread 10925
I20260709 13:17:30.683914 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:30.720778  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800272fc0 after lost signal to thread 12406
I20260709 13:17:30.687003 10127 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:30.686549 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:30.728449  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080016a120 after lost signal to thread 12563
W20260709 13:17:30.728899  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080031ef80 after lost signal to thread 12603
I20260709 13:17:30.592026 12601 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:17:30.652644 12165 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.744238 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.640128 12705 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:30.630106 12747 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:30.644392 12777 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.746210 10900 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:30.685295 12686 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:30.653064 12813 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:30.755291 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:30.749264 10898 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:17:30.730494 10893 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:30.546175 10895 rpcz_store.cc:269] 0709 13:17:27.967075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.049199 (+ 82124us) service_pool.cc:224] Handling call
0709 13:17:30.440534 (+2391335us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:30.440704 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104416640}
W20260709 13:17:30.751396 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 150518us
I20260709 13:17:30.680215 12718 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.653158 12731 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:30.713646 12609 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:30.719362 12734 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.643493 12706 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.769294 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.730324 12754 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:30.770923 10047 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:30.753031 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.780526 10127 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:30.743902 12785 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:30.639181 12784 rpcz_store.cc:276] 0709 13:17:26.494592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.568219 (+ 73627us) service_pool.cc:224] Handling call
0709 13:17:26.964047 (+395828us) ts_tablet_manager.cc:1151] Deleting tablet 806f35e66c234d569cbafd8b3981518f
0709 13:17:26.972748 (+  8701us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:17:28.237610 (+1264862us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:30.166056 (+1928446us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:30.485624 (+319568us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26119,"tablet-delete.queue_time_us":257388,"thread_start_us":91643,"threads_started":1}
I20260709 13:17:30.776089 10130 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:30.774713 12718 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.730104 12741 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:17:30.749902 12806 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.779246 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.683290 12812 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:17:30.795626 10131 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:30.757915 12705 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:30.729350  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800378460 after lost signal to thread 12615
I20260709 13:17:30.777778 12620 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:30.725350 12809 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:17:30.731353 12829 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.729992 12764 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.748625 12165 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.880460 12830 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:30.881295 12830 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:30.771628 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 277994us
I20260709 13:17:30.883203 12837 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.883945 12837 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.801767 12620 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.885387 12822 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:30.885701 12620 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:30.886353 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.886988 12847 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.753710 12833 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.754271 12686 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:30.787563 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:30.797868  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08003277a0 after lost signal to thread 12627
W20260709 13:17:30.897202  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800324a60 after lost signal to thread 12682
I20260709 13:17:30.683766 12735 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.797116 12731 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:30.815680 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:30.814419 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:30.853698 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.824990 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:30.908103 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:30.908897 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:30.909878 10201 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:30.818626 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.816244 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:30.823186 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 183799us
I20260709 13:17:30.882788 12764 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:30.912261 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:30.913549 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:30.817499 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.829599 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:30.811550 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130751us
I20260709 13:17:30.918221 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.919036 10047 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:30.808849 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:30.812958 12851 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:30.758791 12810 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:17:30.784857 12721 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:30.886513 12816 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:30.712626 12759 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:17:30.887737 10054 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:30.592360 10891 rpcz_store.cc:276] 0709 13:17:27.971648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:28.054459 (+ 82811us) service_pool.cc:224] Handling call
0709 13:17:28.089935 (+ 35476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:28.090746 (+   811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:28.090801 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:28.091806 (+  1005us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.334515 (+2242709us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:30.334565 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.334650 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:30.334699 (+    49us) raft_consensus.cc:1685] finished
0709 13:17:30.334714 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:30.334863 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77141,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":10334,"thread_start_us":2242058,"threads_started":1,"wal-append.queue_time_us":1361874,"wal-append.run_cpu_time_us":2764,"wal-append.run_wall_time_us":1074397}
I20260709 13:17:30.787086 12601 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.891203 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 121080us
W20260709 13:17:30.897565  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08002727e0 after lost signal to thread 12688
I20260709 13:17:30.914755 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.792904 12544 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:30.753453 12777 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.815021 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.792200 12754 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:30.753854 12747 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.835953 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.783229 12734 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.771492 12813 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:30.915624 10203 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:30.910758 10197 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:30.923272 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.901417 12686 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:30.887284 10895 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:30.971581 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.941737 12853 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:30.979203 10199 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:30.803879 12806 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:30.979487  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080030dfc0 after lost signal to thread 12737
I20260709 13:17:30.980472 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:30.899631 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:30.911178 12833 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:30.935956 12764 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:30.992372 10052 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:30.986352 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:30.895788 12847 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.029917 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.030706 10199 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:30.993491 12544 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:30.977002 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.009670 10896 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:30.989838 10047 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
W20260709 13:17:31.008896  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080038c820 after lost signal to thread 12739
W20260709 13:17:31.032634  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800364fe0 after lost signal to thread 12744
W20260709 13:17:31.032985  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080038c6c0 after lost signal to thread 12745
I20260709 13:17:31.033111 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.033334 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:30.930872 12794 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 13:17:31.020385 12841 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
W20260709 13:17:31.020876 12843 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:17:31.040400 12847 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.004962  9939 catalog_manager.cc:5028] TS db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): tablet 806f35e66c234d569cbafd8b3981518f (table test-workload [id=726aeb71715d4dd79dd17dca8cc5b2f1]) successfully deleted
I20260709 13:17:31.040875 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.041559  9939 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 165088us
I20260709 13:17:31.041818 12813 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:31.042353 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.042721 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1317) took 1953 ms. Trace:
I20260709 13:17:31.044435 12747 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:31.043555 10123 rpcz_store.cc:276] 0709 13:17:29.089854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:29.108373 (+ 18519us) service_pool.cc:224] Handling call
0709 13:17:29.114162 (+  5789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.470645 (+1356483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.470690 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.471527 (+   837us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.678263 (+206736us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:30.678327 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:30.678449 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.042462 (+364013us) raft_consensus.cc:1685] finished
0709 13:17:31.042489 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.042648 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45294,"raft.queue_time_us":395344,"raft.run_cpu_time_us":4180,"raft.run_wall_time_us":873416,"thread_start_us":1516506,"threads_started":2,"wal-append.queue_time_us":280673}
I20260709 13:17:31.036926 12706 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.045889 12753 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.024602 12777 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:30.979470 12731 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:17:31.047305 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.047837 12731 consensus_queue.cc:1243] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 6.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:31.048650 12806 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.048688 12731 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.049431 12731 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:30.991171 12601 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.025645 12718 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.011816 12609 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:17:31.033265  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800201ea0 after lost signal to thread 12749
I20260709 13:17:31.051787 12601 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:31.052266  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080034c4c0 after lost signal to thread 12758
W20260709 13:17:31.052635  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080034c080 after lost signal to thread 12762
I20260709 13:17:31.052577 10200 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:31.053249  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800345480 after lost signal to thread 12776
W20260709 13:17:31.053586  9985 debug-util.cc:398] Leaking SignalData structure 0x7b080034be60 after lost signal to thread 12780
W20260709 13:17:31.053936  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc160 after lost signal to thread 12789
W20260709 13:17:31.054402  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2640 after lost signal to thread 12791
W20260709 13:17:31.054818  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1ce0 after lost signal to thread 12803
W20260709 13:17:31.019773 12844 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:17:31.036919 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:31.055090  9985 debug-util.cc:398] Leaking SignalData structure 0x7b08002d1020 after lost signal to thread 12804
I20260709 13:17:31.055608 10130 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 2. (index mismatch)
I20260709 13:17:31.055699 10201 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:31.027654 12800 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.057152 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.058737 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.038141 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:31.060274 10202 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:31.061376 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.061861 12850 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.038339 12794 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.063354 12794 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:31.063931 12544 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.064293 12794 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.065376 12544 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:31.065521 12794 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:30.984172 12816 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.039950 12734 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.066831 12809 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.067346 12809 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:31.067579 10201 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:31.068398 12809 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.069402 10201 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:31.053720 12833 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.037313 12754 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.035233 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.071347 12829 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.071534 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:31.028365 12853 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:31.072912 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.074438 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.062872 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.008862 12705 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.042052 12735 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.074023 12853 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.076867 10202 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:31.034941 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.078260 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:31.050495 12731 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.082734 10126 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:31.105170 10052 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:31.116874 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.118000 10050 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:31.118377 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.118980 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.120002 10052 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:31.154284 10200 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:31.005666 12721 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:31.161661 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.197603 10052 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:17:31.162367 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45600.1us
I20260709 13:17:31.205137 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.207732 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.208936 10052 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 3.
I20260709 13:17:31.210644 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.170217 12764 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:31.173118 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 251475us
W20260709 13:17:31.176160 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 498776us
I20260709 13:17:31.173353 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.188040 10055 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.002s
I20260709 13:17:31.221714 10055 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:31.188862 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 110694us
I20260709 13:17:31.223587 12717 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.224506 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:31.189584 12735 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:31.191039 12850 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:31.193498 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 405993us
W20260709 13:17:31.087634 12840 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.006s
W20260709 13:17:31.164036 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 477591us
I20260709 13:17:31.240635 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:31.241406 10865 connection.cc:256] RPC call timeout handler was delayed by -2.2723058859999128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:17:31.119050 12830 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:31.230700 10055 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:31.241815 10129 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:31.215003 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.236557 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:31.242009 10865 connection.cc:256] RPC call timeout handler was delayed by -1.5338867529999334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:17:31.243309 10865 connection.cc:256] RPC call timeout handler was delayed by -1.5293867529999261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 13:17:31.243625 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:31.215304 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.243942 10865 connection.cc:256] RPC call timeout handler was delayed by -1.3826867529999163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 13:17:31.244561 10865 connection.cc:256] RPC call timeout handler was delayed by -1.3421867529998508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
I20260709 13:17:31.244522 10052 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:31.236426 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.246047 10052 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:31.247293 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.243938 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:31.249948 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:31.252730 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.260074 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:31.237802 12731 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.286895 10055 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:31.235677 12735 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.281136 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:31.224503 12717 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.288414 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:31.286566 12830 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:31.288863 10048 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:31.281147 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:31.295310 10049 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:31.322347 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.326138 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:31.252146 12829 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:31.349481 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.327795 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.290280 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:31.294958 12735 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:31.294409 12620 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.399727 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.596256 10055 raft_consensus.cc:2376] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7. Re-sending same reply.
I20260709 13:17:31.405857 12816 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.327025 12829 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.431500 12753 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.247668 12840 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.412894 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.402449 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 468496us
I20260709 13:17:31.412578 12601 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:31.403578 10052 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:31.323218 12854 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:31.401557 10048 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:31.437793 12827 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:31.616806 12827 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:31.618005 12827 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.412839 12764 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:31.494063 12165 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.005s
I20260709 13:17:31.621718 12165 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.624900 12165 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:31.414968 12850 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.253155 12872 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 13:17:31.407133 10049 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:31.411137 12858 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:31.603513 12876 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:17:31.636132 10053 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:31.627545 12759 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.638352 12759 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.627324 12854 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.642268 12865 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.643061 12872 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:31.644733 12872 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.605149 12620 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:31.612015 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.003s	sys 0.010s
I20260709 13:17:31.628509 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:31.647938 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.003s	sys 0.010s
I20260709 13:17:31.633424 10203 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:31.648442 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 717885us
W20260709 13:17:31.651716 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 358230us
W20260709 13:17:31.655428 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 374373us
W20260709 13:17:31.655787 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:31.656595 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.662101 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:31.612634 12829 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.667874 10198 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:17:31.667958 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.669476 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:31.669694 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.669891 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:31.670938 10199 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:17:31.671598 12877 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.672343 12877 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.673372 12877 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:31.674721 12544 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.609s	user 0.006s	sys 0.006s
I20260709 13:17:31.675545 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.658918 12601 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.677587 12601 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:31.669509 12686 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
I20260709 13:17:31.643376 12865 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.680658 12686 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.642481 12854 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.688822 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:31.682652 12865 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.688779 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:31.665980 12794 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.600s	user 0.004s	sys 0.012s
I20260709 13:17:31.657209 12827 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:31.696956 12848 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:31.701994 12848 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.664492 12868 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
W20260709 13:17:31.704231 12882 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:17:31.705868 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 192) took 1068 ms. Trace:
I20260709 13:17:31.706199 12721 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.686641 12885 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:17:31.704308 12868 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:31.687436 12421 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:17:31.692384 12816 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 13:17:31.697975 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45324us
I20260709 13:17:31.708418 12721 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.708534 12868 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:31.713347 10203 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:31.706423 10892 rpcz_store.cc:276] 0709 13:17:30.637305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.640114 (+  2809us) service_pool.cc:224] Handling call
0709 13:17:31.070901 (+430787us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edff8c 00007fa274edfef6 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:31.071067 (+   166us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:31.220333 (+149266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.227018 (+  6685us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:31.411079 (+184061us) log.cc:844] Serialized 52 byte log entry
0709 13:17:31.658307 (+247228us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:31.658411 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.658564 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.705434 (+ 46870us) raft_consensus.cc:1685] finished
0709 13:17:31.705465 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:31.705771 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17103,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":183,"spinlock_wait_cycles":16739072,"thread_start_us":94654,"threads_started":1,"wal-append.queue_time_us":243162}
I20260709 13:17:31.718973 12868 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.720609 12868 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:31.724279 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:31.646272 12872 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.699929 12686 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.708702 12848 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.715711 12735 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:31.736959 10055 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:31.736881 12858 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:17:31.739871 12883 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.740283 12816 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:31.740655 12883 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.740698 12544 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:31.741585 12544 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.742769 12544 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.752384 10198 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:31.753185 12735 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:31.770848 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54477.5us
I20260709 13:17:31.741782 12883 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:31.775738 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47922us
I20260709 13:17:31.778651 12832 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:31.779289 12832 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:31.780259 12832 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.781442 12832 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.781757 12829 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:31.777701 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.782984 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:31.783517 10049 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:31.783761 10894 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:31.784337 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 499) took 1011 ms. Trace:
I20260709 13:17:31.784757 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.784709 10894 rpcz_store.cc:276] 0709 13:17:30.773269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.801706 (+ 28437us) service_pool.cc:224] Handling call
0709 13:17:31.782739 (+981033us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:31.784272 (+  1533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9936256}
I20260709 13:17:31.785612 10049 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
W20260709 13:17:31.789877 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:31.790664 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.786839 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.799374 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.803604 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.808036 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:31.810966 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.837541 12809 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.824918 12837 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.794847 12892 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:17:31.919450 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1353) took 1389 ms. Trace:
I20260709 13:17:31.911765 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.920410 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:31.925938 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.898519 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 174656us
I20260709 13:17:31.881646 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:31.905680 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:31.833576 12855 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.835s	user 0.003s	sys 0.016s
I20260709 13:17:31.935978 10048 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:31.936478 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:31.899806 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.950081 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.946467 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:31.974349 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.941118 10052 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:31.920607 12851 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.950s	user 0.013s	sys 0.005s
W20260709 13:17:31.916301 12891 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:17:31.973778 10203 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:31.932145 10049 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:31.975047 10198 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:31.924337 10053 rpcz_store.cc:276] 0709 13:17:30.530045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.682904 (+152859us) service_pool.cc:224] Handling call
0709 13:17:30.733043 (+ 50139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.636937 (+903894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.919018 (+282081us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400369054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:31.919342 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076170496}
I20260709 13:17:31.976675 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.977761 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.979050 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:31.863931 12872 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:31.920084 12829 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:31.909453 12717 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:31.870769 12854 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.097218 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1329) took 1687 ms. Trace:
I20260709 13:17:31.830783 12883 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:32.003435 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.102885 10122 rpcz_store.cc:276] 0709 13:17:30.410228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.797250 (+387022us) service_pool.cc:224] Handling call
0709 13:17:30.797779 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.798104 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:31.194155 (+396051us) log.cc:844] Serialized 14 byte log entry
0709 13:17:31.287826 (+ 93671us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.048077 (+760251us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b1800024220. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:32.048497 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.095839 (+ 47342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.095924 (+    85us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:32.095995 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.096100 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.097131 (+  1031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":872742,"prepare.queue_time_us":122235,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":811988,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":21630,"spinlock_wait_cycles":30673024,"thread_start_us":358877,"threads_started":2,"wal-append.queue_time_us":324490}
I20260709 13:17:32.092779 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:31.976374 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:32.102226 10020 connection.cc:256] RPC call timeout handler was delayed by -2.132632982999894s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2566s for a response
I20260709 13:17:32.003610 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:32.072124 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:32.106068 10020 connection.cc:256] RPC call timeout handler was delayed by -2.0938045269999748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:17:32.071245 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:31.984272 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:32.119648 10020 connection.cc:256] RPC call timeout handler was delayed by -2.0857045269999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:17:32.007855 12759 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:32.120191 10020 connection.cc:256] RPC call timeout handler was delayed by -1.8382045270000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1875s for a response
W20260709 13:17:32.030490 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.069572 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:32.121889 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.122434 10020 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:32.122773 10048 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:32.105492 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.124043 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:32.016279 12898 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:17:32.131224 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:32.091315 12620 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.122869 10197 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:32.133303 10048 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:31.980316 12764 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:32.140429 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.080651 12884 ts_tablet_manager.cc:933] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.136338 12759 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:32.095494 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 193) took 1438 ms. Trace:
I20260709 13:17:32.154402 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.114679 12883 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:31.990098 12851 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:32.134095 10049 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:32.142719 12620 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.164911 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.166342 10049 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:32.159237 12759 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:32.170529 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.171981 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.172857 10052 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:32.173434 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.174468 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.143162 12764 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:32.167090 12620 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.176542 10048 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:32.181589 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.185180 10053 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:32.183599 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.187602 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.187964 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.188531 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:31.886055 12721 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.190456 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.191895 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.212188 10048 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:17:32.213593 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.215490 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.225139 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:32.225318 10053 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
I20260709 13:17:32.225865 10048 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.226868 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1535) took 1151 ms. Trace:
I20260709 13:17:32.228034 10053 raft_consensus.cc:606] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.229138 10053 raft_consensus.cc:614] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:32.227710 10052 rpcz_store.cc:276] 0709 13:17:31.075659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.212583 (+1136924us) service_pool.cc:224] Handling call
0709 13:17:32.213953 (+  1370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.226431 (+ 12478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.226459 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.226523 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:32.226561 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.226635 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.226793 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.233359 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1537) took 1072 ms. Trace:
I20260709 13:17:32.241091 10052 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:32.234092 10048 rpcz_store.cc:276] 0709 13:17:31.161489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.225158 (+1063669us) service_pool.cc:224] Handling call
0709 13:17:32.225576 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.233019 (+  7443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.233284 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:32.246230 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1354) took 1715 ms. Trace:
I20260709 13:17:32.256476 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1344) took 1579 ms. Trace:
I20260709 13:17:32.265259 10130 rpcz_store.cc:276] 0709 13:17:30.677256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.924974 (+247718us) service_pool.cc:224] Handling call
0709 13:17:30.925248 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.056167 (+130919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.256046 (+1199879us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540015ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:32.256375 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298751360}
W20260709 13:17:32.241597 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.270699 10049 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:32.272950 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:32.273993 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1504) took 1356 ms. Trace:
I20260709 13:17:32.265019 10051 rpcz_store.cc:276] 0709 13:17:30.530406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.739462 (+209056us) service_pool.cc:224] Handling call
0709 13:17:30.767793 (+ 28331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.027293 (+259500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.027337 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.027382 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:31.027417 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.245964 (+1218547us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:32.246011 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.246158 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596160768}
I20260709 13:17:32.272673 12865 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:32.251968  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:32.282796 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.284476 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.274816 10056 rpcz_store.cc:276] 0709 13:17:30.917340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.918103 (+   763us) service_pool.cc:224] Handling call
0709 13:17:30.918414 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.932756 (+ 14342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.932809 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:30.932892 (+    83us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:30.932957 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.273564 (+1340607us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b54002350d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:32.273644 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.273887 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32164608}
W20260709 13:17:32.293244 10169 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.715s (SENT)
I20260709 13:17:32.295361 10056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46784
I20260709 13:17:32.296515 10056 raft_consensus.cc:606] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Received request to transfer leadership
I20260709 13:17:32.297253 10056 raft_consensus.cc:614] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:32.298316 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.302665 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:32.245498 12840 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.306007 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:32.314213 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:32.280942 12865 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:32.211599 12759 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.323168 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.326368 10049 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.318603 12850 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:32.337210 12850 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.337772 12850 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:32.299803 12837 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:17:32.340194 12837 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:32.340794 12837 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:32.341922 12837 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.240263 12883 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.153540 10897 rpcz_store.cc:276] 0709 13:17:30.656758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.669932 (+ 13174us) service_pool.cc:224] Handling call
0709 13:17:30.773057 (+103125us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:30.883288 (+110231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:30.883329 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:30.883957 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:17:30.983926 (+ 99969us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:30.997116 (+ 13190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:31.004172 (+  7056us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:31.832722 (+828550us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:32.072319 (+239597us) raft_consensus.cc:1685] finished
0709 13:17:32.072346 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.095391 (+ 23045us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6469,"raft.queue_time_us":77708,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":13810,"spinlock_wait_cycles":3795584,"thread_start_us":99421,"threads_started":1,"wal-append.queue_time_us":110870}
I20260709 13:17:32.309432 12877 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:32.351549 12877 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:32.352228 12877 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.234900 12764 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.318887 12896 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:32.322346 12854 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:32.320309 12829 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:32.385135 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 195) took 1632 ms. Trace:
I20260709 13:17:32.386049 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1522) took 1328 ms. Trace:
I20260709 13:17:32.385805 12865 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.388212 12865 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:32.388487 10019 connection.cc:256] RPC call timeout handler was delayed by -2.9934621419999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
I20260709 13:17:32.389010 12717 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.389628 12717 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.391072 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 194) took 1618 ms. Trace:
I20260709 13:17:32.385686 10127 rpcz_store.cc:276] 0709 13:17:30.752752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.943416 (+190664us) service_pool.cc:224] Handling call
0709 13:17:30.995778 (+ 52362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:31.012308 (+ 16530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:31.012376 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:31.012441 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:31.012491 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.384831 (+1372340us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b540014b254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:32.384885 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.385044 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37333248}
I20260709 13:17:32.392045 12717 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:32.392516 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.06639e+06us
I20260709 13:17:32.360739 12896 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.393509 12896 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:32.393682 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.391566 10898 rpcz_store.cc:276] 0709 13:17:30.772517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.936341 (+163824us) service_pool.cc:224] Handling call
0709 13:17:30.985419 (+ 49078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:30.985719 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:31.235287 (+249568us) log.cc:844] Serialized 14 byte log entry
0709 13:17:31.331204 (+ 95917us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.390475 (+1059271us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18002a8bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:17:32.390677 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.390740 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:32.390806 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.390882 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.391006 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68335,"prepare.queue_time_us":609554,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":161,"raft.queue_time_us":51663,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":8937,"spinlock_wait_cycles":6954240,"thread_start_us":282326,"threads_started":2,"wal-append.queue_time_us":219382}
I20260709 13:17:32.395354  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:32.393994 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:32.396895 12860 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.397776 12860 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:32.399346 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 856282us
I20260709 13:17:32.402067 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:32.402456  9953 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 115) took 16898 ms (client timeout 15000 ms). Trace:
I20260709 13:17:32.406971 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.409142 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:32.386664 10050 rpcz_store.cc:276] 0709 13:17:31.057591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.118722 (+ 61131us) service_pool.cc:224] Handling call
0709 13:17:32.385809 (+1267087us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002e8e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:32.385934 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2430025600}
I20260709 13:17:32.411307 10049 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:32.377099 12837 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.415151 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:32.415997 10894 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:32.420025 10199 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:32.419562 12717 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.427361 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.431916 10052 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:32.428153 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:32.434309 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:32.428817 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.436084 10050 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:32.437741 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.438984 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.440959 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.442149 10050 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:32.444787 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:32.446785 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.448168 10050 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:32.449113 12859 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:32.449702 10897 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:17:32.402988  9953 rpcz_store.cc:269] 0709 13:17:15.612022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:15.612481 (+   459us) service_pool.cc:224] Handling call
0709 13:17:32.402366 (+16789885us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:26.443982 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:26.880592 (+436610us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:26.895333 (+ 14741us) write_op.cc:435] Acquired schema lock
0709 13:17:26.902366 (+  7033us) tablet.cc:662] Decoding operations
0709 13:17:27.247286 (+344920us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:27.247427 (+   141us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:27.247459 (+    32us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:17:27.268719 (+ 21260us) tablet.cc:701] Row locks acquired
0709 13:17:27.268782 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:17:27.269076 (+   294us) write_op.cc:270] Start()
0709 13:17:27.269296 (+   220us) write_op.cc:276] Timestamp: P: 1783603047268993 usec, L: 0
0709 13:17:27.269321 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:27.301901 (+ 32580us) log.cc:844] Serialized 4772 byte log entry
0709 13:17:30.256825 (+2954924us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:17:31.020894 (+764069us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b18003cf9a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa27ba3b41d 00007fa27ba3d71a 00007fa27ba3d6a8 00007fa27ba3d628 00007fa27ba3d5e3 00007fa27ba3c7fc 00007fa27de6f626 00007fa27de69dd8 00007fa278010310
0709 13:17:31.020973 (+    79us) op_driver.cc:464] REPLICATION: finished
0709 13:17:31.239748 (+218775us) write_op.cc:301] APPLY: starting
0709 13:17:31.261233 (+ 21485us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:31.417019 (+155786us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:31.417081 (+    62us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:31.429928 (+ 12847us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:31.430499 (+   571us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:17:31.430577 (+    78us) write_op.cc:312] APPLY: finished
0709 13:17:31.493050 (+ 62473us) log.cc:844] Serialized 89 byte log entry
0709 13:17:31.641788 (+148738us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa27800132a 00007fa277f86f14 00007fa277f86ec3 00007fa277f86e38 00007fa277f86df3 00007fa277f860dc 00007fa277fafe16 00007fa277f97cbb 00007fa277f8239d 00007fa277f85d75
0709 13:17:31.692662 (+ 50874us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:31.698535 (+  5873us) write_op.cc:454] Released schema lock
0709 13:17:31.716511 (+ 17976us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:31.754862 (+ 38351us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:31.754902 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1490500,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":157687,"apply.run_cpu_time_us":12898,"apply.run_wall_time_us":525841,"num_ops":12,"prepare.queue_time_us":219896,"prepare.run_cpu_time_us":71351,"prepare.run_wall_time_us":4333198,"raft.queue_time_us":13587,"raft.run_cpu_time_us":72077,"raft.run_wall_time_us":1341261,"replication_time_us":3476737,"spinlock_wait_cycles":1198075008,"thread_start_us":3732500,"threads_started":5,"wal-append.queue_time_us":3165522,"wal-append.run_cpu_time_us":16752,"wal-append.run_wall_time_us":1044799}]]}
I20260709 13:17:32.452160 10898 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:17:32.453266 10002 auto_rebalancer.cc:893] Removing replace marker failed: Replica 01bfaf1ad56a4deea886bbc64dec466c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:17:32.453846 10002 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:17:32.454210 10002 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:17:32.459445 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:32.460381 10056 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:32.461534 12721 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:32.467778 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 502) took 1219 ms. Trace:
I20260709 13:17:32.468447 10893 rpcz_store.cc:276] 0709 13:17:31.248817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.252373 (+  3556us) service_pool.cc:224] Handling call
0709 13:17:32.057886 (+805513us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:32.058141 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.058627 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:32.226149 (+167522us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.392765 (+166616us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b18000266e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:32.467208 (+ 74443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:32.467336 (+   128us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:32.467396 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:32.467493 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:32.467687 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51373,"prepare.queue_time_us":172628,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":54870,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":17139,"spinlock_wait_cycles":142945152,"thread_start_us":200206,"threads_started":2,"wal-append.queue_time_us":102708}
I20260709 13:17:32.449659 12859 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:32.437948 12812 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:32.482192 12859 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.431488 12705 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.478827 12717 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.480204 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:32.425565 12858 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:32.492990 12858 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.493508 12858 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:32.492264 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:32.498566 10200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1528) took 1684 ms (client timeout 1793 ms). Trace:
W20260709 13:17:32.499091 10200 rpcz_store.cc:269] 0709 13:17:30.813865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.814021 (+   156us) service_pool.cc:224] Handling call
0709 13:17:32.498321 (+1684300us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001bfb54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:32.498491 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333531008}
I20260709 13:17:32.501399 12851 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:32.502187 12851 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.502715 12851 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:32.503327 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:32.486610 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:32.488564 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1345) took 1811 ms (client timeout 1989 ms). Trace:
I20260709 13:17:32.504752 12764 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:32.505158 10124 rpcz_store.cc:269] 0709 13:17:30.677601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:30.745837 (+ 68236us) service_pool.cc:224] Handling call
0709 13:17:32.488365 (+1742528us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54002d1754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:32.488495 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098496256}
W20260709 13:17:32.507097 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.508651 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:32.509784 12914 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.110s	user 0.002s	sys 0.000s
I20260709 13:17:32.510162 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.510578 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.511849 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.513789 10122 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:17:32.515904 12741 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:32.516083 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 423416us
I20260709 13:17:32.517261 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:32.519344 12827 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:32.519731 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.520601 10125 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.521163 12620 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:32.521420 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.521981 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.521898 12620 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.522481 10126 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:32.520102 12827 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.523455 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.524394 10131 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.524621 12620 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:32.525086 12827 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:32.525199 12878 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.519078 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:32.454377 12877 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.526801 12878 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:32.527468 12878 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:32.527575 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.528002 12878 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.528404 10123 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 4.
I20260709 13:17:32.529481 12830 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.529800 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.532802 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:32.534245 12913 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.141s	user 0.000s	sys 0.002s
I20260709 13:17:32.534068 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.534363 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.535027 10125 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:32.535578 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.536098 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.536351 10130 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:32.536635 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1549) took 1126 ms. Trace:
I20260709 13:17:32.536809 10125 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:32.537144 10055 rpcz_store.cc:276] 0709 13:17:31.410815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.656443 (+245628us) service_pool.cc:224] Handling call
0709 13:17:32.536395 (+879952us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b5400086954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:32.536564 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138779008}
I20260709 13:17:32.537559 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.538826 10054 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 2. (term mismatch)
I20260709 13:17:32.539324 10054 pending_rounds.cc:85] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 1
I20260709 13:17:32.539037 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.539584 10054 pending_rounds.cc:107] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:32.539899 10054 raft_consensus.cc:2889] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.540175 10128 raft_consensus.cc:2410] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:17:32.530983 12830 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:17:32.541498 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.542234 12877 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:32.542567 10131 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:32.540236 10054 pending_rounds.cc:107] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:17:32.544001 10054 raft_consensus.cc:2889] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.545637 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:32.546104 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.546804 10123 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:17:32.547305 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57498.2us
I20260709 13:17:32.548152 12896 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.548698 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.549316 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.548210 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.550410 12892 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:32.550726 12731 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.551008 10122 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:32.551469 12840 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.551069 12892 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.552093 12840 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.553918 12892 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:32.554955 10127 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:32.555572 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.555807 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:32.557658 10199 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:32.563084 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 169470us
I20260709 13:17:32.564725 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:32.565794 10052 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:32.578816 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:32.583160 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 138454us
I20260709 13:17:32.584092 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:17:32.589411 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.582326 12753 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:32.585374 12809 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:32.592514 12753 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.592952 12753 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.583402 12609 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:17:32.585605 12741 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:32.596953 12741 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.597424 12741 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:32.603600 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:32.604769 12884 tablet_copy_client.cc:323] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:17:32.605222 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.607362 12896 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:32.610673 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:32.619289 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.610965 10205 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:17:32.617089 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.623653 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:32.631913 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:32.643777 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.644863 10127 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:32.585727 12822 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.654981 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.657999 10131 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.662292 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.576987 12731 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.662292 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.665220 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.665767 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.666111 10123 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 3.
I20260709 13:17:32.663481 10127 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.641474 12809 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:17:32.667380 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 290034us
I20260709 13:17:32.643446 12794 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:32.669210 10131 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:32.668974 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.662282 12822 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:32.653020 12620 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:32.683995 10123 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.697168 12809 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:32.698266 12809 consensus_queue.cc:1243] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:32.698314 10894 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:17:32.663511 12717 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:32.699507 12717 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:32.641633 12609 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:17:32.700539 12717 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:32.701220 12717 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.646974 12832 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:17:32.704053 12832 consensus_queue.cc:1243] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:32.699508 12794 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:32.704811 10055 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:17:32.704743 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.704768 12794 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:32.706110 10131 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:32.648651 12827 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:17:32.694415 12872 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:32.726444 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.727257 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.728163 10131 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:32.728343 10128 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.721237 12717 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.717393 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.735646 10050 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.660961 12810 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:17:32.745687 10047 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:32.747730 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.748778 10125 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:32.687438 12620 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:32.753077 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:32.724336 12822 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:32.759037 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:32.760088 10052 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:32.766186 10125 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:32.766849 10125 pending_rounds.cc:85] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 3
I20260709 13:17:32.767184 10125 pending_rounds.cc:107] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:17:32.767534 10125 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.761335 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:32.738921 12794 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.806113 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:32.900300 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:32.864900 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.937307 10131 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.767566 12620 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:32.802968 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:32.949764 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 358838us
I20260709 13:17:32.952491 10130 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:17:32.930823 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.953156 10130 pending_rounds.cc:85] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:32.954274 10130 pending_rounds.cc:107] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:32.954735 10130 raft_consensus.cc:2889] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:32.956609 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:32.954063 10125 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:17:32.788872 10128 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:32.961519 12832 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.863327 12850 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 13:17:32.961668 12822 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.776223 12877 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.187s
I20260709 13:17:32.749567 12717 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:32.962592 12741 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:32.865135 12753 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:32.934507 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 323909us
W20260709 13:17:32.852533 12909 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:17:32.978971 10129 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:33.005717 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:32.855943 12922 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:32.844643 10052 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:32.981263 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 327378us
I20260709 13:17:32.990072 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:32.976679 12850 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.016233 10896 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:33.017104 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:32.991073 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:33.020123 12827 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.021711 12753 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.991629 10895 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:32.883440 12759 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:17:33.024582 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.024838 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:32.968111 12859 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:32.991940 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:33.028769 12859 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:32.992985 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:32.992123 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:32.993937 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:33.031560 12810 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:32.984476 12717 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:33.022559 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.033353 12717 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.032212 12810 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.034545 12810 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:33.034608 12717 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:33.031863 10891 raft_consensus.cc:2410] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:17:33.032153 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.026015 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:33.026139 12741 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:32.882803 12878 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.005250 12883 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.061348 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.003568 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:33.063464 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 321480us
I20260709 13:17:33.035753 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:33.021711 12840 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:33.067932 12883 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.068584 12856 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.069308 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.070379 12883 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:33.029143 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet 806f35e66c234d569cbafd8b3981518f from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:17:33.071396 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet 806f35e66c234d569cbafd8b3981518f from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f
I20260709 13:17:33.000864 12858 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:33.067098 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62309.8us
I20260709 13:17:33.070405 10123 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:32.981824 10130 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:32.995219 12832 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:33.083989 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:33.085870 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.048401 12922 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:33.088932 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:33.093439 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.093684 10122 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:33.050984 12731 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:33.096463 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:33.110417 12877 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:33.111050 12877 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:33.111979 12877 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:33.076938 12850 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:33.125402 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.104990 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:33.126374 10052 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:33.116562 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:33.084044 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.128367 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:33.128943 10123 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:33.131012 10054 raft_consensus.cc:2368] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
W20260709 13:17:33.134006 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.134153 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:33.124631 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 339389us
I20260709 13:17:33.135502 10051 raft_consensus.cc:2376] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7. Re-sending same reply.
I20260709 13:17:33.141187 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:33.099050 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.142884 10202 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:33.129666 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:33.146536 10202 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:33.150786 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.155928 10130 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:17:33.159384 10202 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:17:33.336009 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.541s (SENT)
I20260709 13:17:33.161509 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:33.337395 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:33.337934 10056 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:33.175678 10125 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:33.293865 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162442us
I20260709 13:17:33.338560 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.252418 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:33.341710 10052 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:33.340041 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.343293 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:33.321035 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.320794 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.344094 10130 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:33.344620 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.344559 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:33.161123 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:33.346480 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.349408 10130 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:17:33.321826 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:33.136579 12920 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 13:17:33.324074 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:33.354339 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:33.118037 12858 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:33.358807 10123 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:33.428977 12930 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:33.357631 10130 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:17:33.315037 12851 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.812s	user 0.018s	sys 0.002s
W20260709 13:17:33.355048 12910 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:17:33.160522 12856 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:33.425400 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:33.351183 12922 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.355698 10125 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 4. Current term is 6. Ops: [4.3-4.3]
I20260709 13:17:33.428831 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:33.360015 12753 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:33.262094 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:33.429837  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:33.259847 12165 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.428834 12731 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.440681 12827 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:33.296653 12860 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.259651 12609 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.448643 12942 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:33.454974 12741 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:33.426095 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 102125us
I20260709 13:17:33.707162 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:33.708971 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:33.444744 10056 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:17:33.447544 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:33.448415 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94168.3us
I20260709 13:17:33.451085 10052 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:33.449343 10048 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:33.711467 10096 leader_election.cc:400] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:33.360149 12940 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:33.435776 12764 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.930s	user 0.020s	sys 0.002s
I20260709 13:17:33.465170 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:33.494238 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.724758 10127 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:33.430013 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76216.1us
I20260709 13:17:33.452075 12851 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.691797 12609 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.711834 10096 leader_election.cc:403] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:33.468952 12856 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:33.672837 12620 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.711s	user 0.025s	sys 0.000s
I20260709 13:17:33.665370 12827 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:33.728354 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.455094 12928 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:33.731405 10049 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:33.662734 12731 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:33.754513 10131 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.806s	user 0.016s	sys 0.000s
I20260709 13:17:33.788547 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:33.656720 12165 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:33.807199 10056 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:33.816964 10131 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:33.731914 12878 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:33.800479 12827 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:33.439450 10130 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:17:33.839098 12609 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:33.840755 12832 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:33.844892 12165 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:33.845727 12856 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:33.847904 12809 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:33.852182 12936 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.409s	user 0.000s	sys 0.001s
W20260709 13:17:33.856473 12943 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:17:33.857378 12754 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:33.774833 12764 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:33.866374 10130 pending_rounds.cc:85] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
I20260709 13:17:33.868654 12609 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:33.730897 12940 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:33.869347 12921 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.558s	user 0.001s	sys 0.001s
I20260709 13:17:33.825138 12930 raft_consensus.cc:2764] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:17:33.713295 12942 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:33.867851 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:33.906944 10052 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:17:33.914598 10130 pending_rounds.cc:107] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:17:33.871770 12731 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:33.800971 12620 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:33.931336 10130 raft_consensus.cc:2889] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:33.936640 10125 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:33.820545 12928 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:33.914801 12754 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:33.969498 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:33.930145 12165 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:33.971933 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 263090us
I20260709 13:17:33.976779 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:33.976794 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:33.936343 12620 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:33.957767 12764 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:33.981110 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 269784us
I20260709 13:17:33.983805 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:33.986289 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:33.968401 12847 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.976040 12601 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
W20260709 13:17:34.274888 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.000s	sys 0.014s
W20260709 13:17:34.275902 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.000s	sys 0.015s
W20260709 13:17:34.145066 12951 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
W20260709 13:17:34.288863 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:33.999329 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.006s	sys 0.014s
W20260709 13:17:34.291816 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.022s	sys 0.061s
I20260709 13:17:33.992082 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:33.993394 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 507) took 1435 ms. Trace:
W20260709 13:17:34.253850 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 269953us
I20260709 13:17:33.978040 12544 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:33.994750 12922 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:34.012033 12731 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.029366 12851 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.293340 12953 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:34.239424 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.014s
I20260709 13:17:34.118125 12940 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:34.298791 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.008s	sys 0.019s
I20260709 13:17:34.155942 12928 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:34.156461 12942 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:34.253994 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 277157us
W20260709 13:17:34.302012 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 854470us
I20260709 13:17:33.978365 12165 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.303355 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:34.304345 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:33.990757 12764 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:34.293073 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.308497 12949 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:34.309767 12949 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.078224 12812 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:34.311319 12759 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.296640 12922 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.011411 12741 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:34.298050 12731 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.021613 10214 tablet_copy_source_session.cc:215] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:33.994560 12847 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.137449 12794 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.335453 12940 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:34.003515 12754 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.336731 12940 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.337853 12754 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.339244 12961 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.340636 12961 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.344887 12958 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.345861 12731 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.348069 12764 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.349419 10214 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:50726 (request call id 208) took 1372 ms. Trace:
I20260709 13:17:34.350076 10214 rpcz_store.cc:276] 0709 13:17:33.013209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.028756 (+ 15547us) service_pool.cc:224] Handling call
0709 13:17:34.349040 (+1320284us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:34.354745 12717 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.309s	user 0.017s	sys 0.001s
I20260709 13:17:34.327939 12812 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.293870 10893 rpcz_store.cc:276] 0709 13:17:32.556430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.562680 (+  6250us) service_pool.cc:224] Handling call
0709 13:17:32.642519 (+ 79839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:32.714189 (+ 71670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:32.724169 (+  9980us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:32.851352 (+127183us) log.cc:844] Serialized 26 byte log entry
0709 13:17:33.251698 (+400346us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:33.261944 (+ 10246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:33.293390 (+ 31446us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:33.978348 (+684958us) raft_consensus.cc:1685] finished
0709 13:17:33.978387 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:33.991327 (+ 12940us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23461,"raft.queue_time_us":61133,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":193314,"threads_started":1,"wal-append.queue_time_us":578171}
I20260709 13:17:34.335494 12759 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.324187 12922 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:34.172487 12810 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.137s	user 0.021s	sys 0.002s
I20260709 13:17:34.346153 12958 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:34.408095 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 81310.5us
I20260709 13:17:34.399248 12544 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.412835 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1565) took 1283 ms. Trace:
I20260709 13:17:34.413030 12544 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.414135 12958 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.416675 12544 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:34.400046 12731 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:34.006667 12830 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:34.418989 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 426844us
I20260709 13:17:34.420884 10893 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:34.358991 12955 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:34.351855 12953 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.334672 12847 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.423539 12848 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.430558 10891 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:17:34.419092 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2628) took 2407 ms. Trace:
I20260709 13:17:34.402235 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:34.406244 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.407442 12165 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:34.436290 12953 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:34.436568 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:34.437930 12953 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:34.408250 12601 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:34.408941 12812 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:34.411873 12759 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.400419 10053 pending_rounds.cc:85] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 1
I20260709 13:17:34.410421 12810 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.353363 12959 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:34.422140 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2638) took 1680 ms (client timeout 2007 ms). Trace:
I20260709 13:17:34.438473 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:34.407608 12963 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.420783 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:34.444759 10053 pending_rounds.cc:107] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:17:34.454034 12963 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.441651 12601 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:34.448810 10205 rpcz_store.cc:269] 0709 13:17:32.741486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.742297 (+   811us) service_pool.cc:224] Handling call
0709 13:17:34.421468 (+1679171us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000b3954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:34.421791 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420988288}
I20260709 13:17:34.433455 10198 rpcz_store.cc:276] 0709 13:17:32.011512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.011760 (+   248us) service_pool.cc:224] Handling call
0709 13:17:32.102031 (+ 90271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.103443 (+  1412us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:32.114371 (+ 10928us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.247219 (+132848us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b1800284bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:32.527392 (+280173us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b1800284bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:17:32.722121 (+194729us) log.cc:844] Serialized 14 byte log entry
0709 13:17:32.722732 (+   611us) log.cc:844] Serialized 14 byte log entry
0709 13:17:34.417401 (+1694669us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b1800284bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa277bbad70 00007fa277bae293 00007fa277ba3cea 00007fa277ba423a 00007fa27800ea2a 00007fa27801e0bf 00007fa27801e038
0709 13:17:34.417981 (+   580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:34.418105 (+   124us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:34.418238 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.418421 (+   183us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.418763 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1426137,"prepare.queue_time_us":20412,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":175,"raft.queue_time_us":9101,"raft.run_cpu_time_us":615,"raft.run_wall_time_us":17513,"spinlock_wait_cycles":72983936,"thread_start_us":227380,"threads_started":2,"wal-append.queue_time_us":109325}
I20260709 13:17:34.450893 10053 raft_consensus.cc:2889] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:34.459354 12601 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:34.461790 12620 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:34.462546 12620 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:34.463037 12620 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.469875 10126 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:34.465356 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2643) took 1867 ms (client timeout 2202 ms). Trace:
I20260709 13:17:34.473639 12609 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:34.469491 12959 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.475803 12856 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.475617 12953 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.476462 12856 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.480100 12949 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:34.470558 10122 rpcz_store.cc:269] 0709 13:17:32.554854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.555252 (+   398us) service_pool.cc:224] Handling call
0709 13:17:34.353474 (+1798222us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:34.400713 (+ 47239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1221758848}
W20260709 13:17:34.483758 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63226.9us
I20260709 13:17:34.484284 12961 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.485154 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:34.486109 12810 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:34.487373 12827 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.602s	user 0.020s	sys 0.000s
I20260709 13:17:34.487491 12858 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.474128 12741 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.488699 12858 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.488042 12827 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.489831 10128 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:34.490772 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1379) took 1631 ms. Trace:
I20260709 13:17:34.491827 12827 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:34.494596 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 190663us
I20260709 13:17:34.478639 12959 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.413501 10048 rpcz_store.cc:276] 0709 13:17:33.129112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.723057 (+593945us) service_pool.cc:224] Handling call
0709 13:17:33.723564 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:33.723848 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:33.971419 (+247571us) log.cc:844] Serialized 14 byte log entry
0709 13:17:34.410656 (+439237us) log.cc:844] Serialized 14 byte log entry
0709 13:17:34.411177 (+   521us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:34.411352 (+   175us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:34.411576 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.411745 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.412163 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":288136,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":124,"raft.queue_time_us":37002,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":412,"thread_start_us":574038,"threads_started":2,"wal-append.queue_time_us":359985}
I20260709 13:17:34.422183 12950 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:17:34.511948 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 209031us
I20260709 13:17:34.492853 10128 rpcz_store.cc:276] 0709 13:17:32.859755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.087061 (+227306us) service_pool.cc:224] Handling call
0709 13:17:34.488903 (+1401842us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:34.489409 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.490356 (+   947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.490646 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88480000}
I20260709 13:17:34.522734 12950 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:34.523653 12950 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:34.525710 12950 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.531805 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1566) took 1402 ms. Trace:
I20260709 13:17:34.531558 12827 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:34.532927 12827 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:34.533423 12827 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:34.535617 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:34.538367 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:34.540587 10130 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.542143 12544 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:34.545549 10122 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:17:34.546118 10122 pending_rounds.cc:85] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:17:34.546402 10122 pending_rounds.cc:107] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.5
W20260709 13:17:34.515450 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 47936.3us
I20260709 13:17:34.545220 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:34.542810 12544 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:34.492779 12856 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.546684 10122 raft_consensus.cc:2889] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:34.538504 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:34.532316 10056 rpcz_store.cc:276] 0709 13:17:33.129374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.856694 (+727320us) service_pool.cc:224] Handling call
0709 13:17:33.857201 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.246804 (+389603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.246929 (+   125us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.248431 (+  1502us) log.cc:844] Serialized 26 byte log entry
0709 13:17:34.318389 (+ 69958us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:34.318454 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.326418 (+  7964us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.531342 (+204924us) raft_consensus.cc:1685] finished
0709 13:17:34.531363 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.531610 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205707,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":150,"thread_start_us":457814,"threads_started":2,"wal-append.queue_time_us":184326}
I20260709 13:17:34.553030 12544 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:34.556881 12609 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:34.534811 12827 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:34.562705 10020 leader_election.cc:343] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Tablet error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Illegal state: cannot vote while shut down
I20260709 13:17:34.557523 12810 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:34.577849 12955 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:34.578717 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1507) took 3660 ms. Trace:
I20260709 13:17:34.579097 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:34.579487 12896 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:34.580487 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:34.580304 12165 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:34.581478 12856 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.584141 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:34.585438 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:34.611805 12959 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.612984 12959 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.573509 12940 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.589573 10047 rpcz_store.cc:276] 0709 13:17:30.918367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.046701 (+128334us) service_pool.cc:224] Handling call
0709 13:17:32.495777 (+1449076us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:32.496031 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:32.746538 (+250507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.578050 (+1831512us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b540010c2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:34.578482 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581681664}
W20260709 13:17:34.624330 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:34.625067 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.628111 12896 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:34.631908 12609 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.642877 12950 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.643651 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1568) took 1513 ms. Trace:
I20260709 13:17:34.644125 10052 rpcz_store.cc:276] 0709 13:17:33.129916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.961963 (+832047us) service_pool.cc:224] Handling call
0709 13:17:34.247149 (+285186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.401478 (+154329us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:34.401512 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.411138 (+  9626us) log.cc:844] Serialized 26 byte log entry
0709 13:17:34.506849 (+ 95711us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:34.506882 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.506940 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:34.643211 (+136271us) raft_consensus.cc:1685] finished
0709 13:17:34.643252 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:34.643436 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79885,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":30633,"thread_start_us":176422,"threads_started":2,"wal-append.queue_time_us":163441}
W20260709 13:17:34.648885 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79257.4us
I20260709 13:17:34.651407 12955 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.653954 12955 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.655537 10054 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:34.655274 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 71379.9us
I20260709 13:17:34.655915 12856 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.655063 12940 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:34.667862 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129619us
I20260709 13:17:34.655772 12810 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.675805 12963 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.677217 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:34.723886 12963 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:34.730479 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59606.7us
I20260709 13:17:34.686736 12966 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.689028 12810 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.691391 12717 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:34.692162 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.521s (SENT) [suppressed 47 similar messages]
W20260709 13:17:34.698796 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.706097 10122 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
W20260709 13:17:34.844887 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 152752us
I20260709 13:17:34.706674 12856 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.707624 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:34.707952 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:34.709038 12970 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:34.709518 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77707.9us
I20260709 13:17:34.655989 12959 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.771193 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:34.779078 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:34.780737 12963 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:34.784885 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2656) took 1801 ms (client timeout 1647 ms). Trace:
I20260709 13:17:34.855393 10130 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:34.788897 12946 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.800s	user 0.004s	sys 0.000s
I20260709 13:17:34.791496 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:34.794720 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 207) took 1491 ms. Trace:
W20260709 13:17:34.797493 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1552) took 2368 ms (client timeout 1541 ms). Trace:
I20260709 13:17:34.800186 12848 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:34.802256 10198 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:34.802461 10205 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:34.804519 12601 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.805722 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:34.806885 10123 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:34.808398 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:34.811928 12884 ts_tablet_manager.cc:1916] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 13:17:34.737524 12827 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:34.785741 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:34.847805 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.863893 12963 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.865881 10127 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:34.870929 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 92065us
I20260709 13:17:34.871157 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:34.871862 12930 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:34.872469 12812 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:34.873063 12856 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.899863  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:34.898509 10204 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:34.855646 10126 rpcz_store.cc:269] 0709 13:17:32.984000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.022160 (+ 38160us) service_pool.cc:224] Handling call
0709 13:17:34.784553 (+1762393us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002ef254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:34.784788 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3299697536}
I20260709 13:17:34.874392 12860 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:34.880722 10049 rpcz_store.cc:269] 0709 13:17:32.428453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:32.434805 (+  6352us) service_pool.cc:224] Handling call
0709 13:17:34.796777 (+2361972us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400086e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:34.797166 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969239808}
I20260709 13:17:34.880286 10125 rpcz_store.cc:276] 0709 13:17:33.303369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.449663 (+146294us) service_pool.cc:224] Handling call
0709 13:17:33.449905 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:33.939582 (+489677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.794160 (+854578us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540014b254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:34.794498 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1118722816}
I20260709 13:17:34.908818 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:34.855245 12810 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.856482 12741 raft_consensus.cc:2764] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:17:34.898458 12812 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.900362 12856 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.901223 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:34.873723 12934 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:34.884899 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 109597us
W20260709 13:17:34.905946 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:34.875015 12620 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.885591 12858 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:34.915038 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 129592us
I20260709 13:17:34.889523 12959 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:34.930490 12717 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.931314 12934 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.932322 12812 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.914552 12860 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.933154 12601 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:34.933321 12717 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.933919 12934 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.937151 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2666) took 1772 ms. Trace:
I20260709 13:17:34.937809 10123 rpcz_store.cc:276] 0709 13:17:33.164855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:33.435709 (+270854us) service_pool.cc:224] Handling call
0709 13:17:33.439877 (+  4168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:34.886909 (+1447032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.936707 (+ 49798us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:34.937033 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3142368000}
I20260709 13:17:34.938688 12544 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:34.941856 12544 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.930562 10891 raft_consensus.cc:2410] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:34.960230 12904 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:34.961753 12620 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:34.910216 12955 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:34.974478 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:34.975298 12860 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:34.976635 10050 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:34.887926 12966 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.943856 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:34.924284 12970 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:34.962112 10054 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:34.978437 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162107us
W20260709 13:17:35.073316 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56261.4us
I20260709 13:17:35.041122 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.037259 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.061757 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.039803 10199 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:35.128623 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43626.8us
W20260709 13:17:34.998867 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.031476 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:34.945616 12959 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:35.109488 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:34.978636 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.129858 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.161460 10056 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:35.162905 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:35.077440 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 176414us
I20260709 13:17:35.063592 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:35.039595 12977 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:17:35.084368 12922 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.010s
W20260709 13:17:35.190564 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60819.6us
I20260709 13:17:35.078706 12856 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.027792 12930 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:35.040042 12963 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:35.191502 10049 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:34.980971 12904 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:35.170375 12959 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:35.202885 10051 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:35.182188 12860 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:35.077240 12934 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:17:35.214699 10048 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.217677 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:35.219029 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.065162 12812 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 13:17:34.986567 12810 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.198736 12922 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.217303 12860 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.238713 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.257166 10047 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.258136 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:35.259060 12812 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:35.260622 12812 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:35.261044 12812 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:35.262137 12812 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:35.265758 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.259544 12686 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:35.249035 10054 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:35.243314 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.243635 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.249742 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.272421 12686 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:35.276077 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.277107 10056 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.276479 12686 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.278400 12686 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.248664 12966 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.251092 12860 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:35.245185 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46737.6us
I20260709 13:17:35.128986  9950 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.247383 10205 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
W20260709 13:17:35.294925 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 81865.9us
I20260709 13:17:35.240134 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1569) took 2110 ms. Trace:
W20260709 13:17:35.238256 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46852.5us
I20260709 13:17:35.287086 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.324796 10048 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:35.244484 12950 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.601s	user 0.016s	sys 0.000s
I20260709 13:17:35.333056 10047 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:35.334851 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63139.3us
I20260709 13:17:35.268136 12953 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.340493 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.347828 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.351727 10047 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:35.352079 10050 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:35.254057 12934 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:35.351730 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.252640 12851 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.977950 12717 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.349s
I20260709 13:17:35.345705  9950 catalog_manager.cc:5697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.219726 12955 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.251317 12922 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:35.241807 12810 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.370630 10131 raft_consensus.cc:2410] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:17:35.364568 10051 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:35.324474 10053 rpcz_store.cc:276] 0709 13:17:33.130176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.395566 (+1265390us) service_pool.cc:224] Handling call
0709 13:17:34.399736 (+  4170us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:34.520604 (+120868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.520663 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:34.556767 (+ 36104us) log.cc:844] Serialized 52 byte log entry
0709 13:17:34.887846 (+331079us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:34.887918 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.888034 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.239784 (+351750us) raft_consensus.cc:1685] finished
0709 13:17:35.239821 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.240056 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40316,"raft.queue_time_us":50838,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"thread_start_us":295615,"threads_started":1,"wal-append.queue_time_us":382698}
I20260709 13:17:35.369767 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.374184 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:35.396842 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.337107 12963 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:35.402249 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:35.408531 12973 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:17:35.409883 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77183.9us
I20260709 13:17:35.255874 12928 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:17:35.327847 12856 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:17:35.394142 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:35.365111 12934 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:35.420627 10048 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:35.419651 10054 raft_consensus.cc:2410] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:35.395134 12980 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:35.240378 12904 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.413702 12953 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.467444 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:35.393662 12851 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.493476 10047 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:35.479719 12950 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:35.520004 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.484570 12812 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.500628 12848 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:35.536605 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.546397 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.520737 12970 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.524952  9950 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.520291 12950 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:35.548127 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 228905us
W20260709 13:17:35.548911 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 214794us
I20260709 13:17:35.549765 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.554525 10050 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:35.555693 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.556452 12989 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:35.556928 12989 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:35.557928 12989 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.558722 10050 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:17:35.545437 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.563611 10051 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:35.564178 12989 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:35.564476 10047 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:35.564661 12980 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:35.596526 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:35.594398 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 160113us
I20260709 13:17:35.544855 12848 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:35.570753 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.607544 10048 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:35.560381 12950 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:35.619515 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97974.3us
I20260709 13:17:35.626181 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.629824 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1570) took 2356 ms. Trace:
I20260709 13:17:35.620015  9950 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:35.645207 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144738us
I20260709 13:17:35.621259 12848 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:35.650982 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.650943 10048 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:35.651264 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:35.657001 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:35.650890 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.661079 10204 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:35.658643 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.656523 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99040.3us
I20260709 13:17:35.618731 12980 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:35.541807 12934 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.661990 10198 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:35.630458 10055 rpcz_store.cc:276] 0709 13:17:33.130402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.348401 (+1217999us) service_pool.cc:224] Handling call
0709 13:17:34.349213 (+   812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:34.350040 (+   827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:34.350071 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:34.350806 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 13:17:34.789984 (+439178us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800386a40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:34.876766 (+ 86782us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:34.876830 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:34.876938 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.476913 (+599975us) raft_consensus.cc:1685] finished
0709 13:17:35.476941 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.487148 (+ 10207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10613,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":160,"spinlock_wait_cycles":2765056,"thread_start_us":475105,"threads_started":1,"wal-append.queue_time_us":821420}
I20260709 13:17:35.656087 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.651065 12985 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.103s	user 0.000s	sys 0.002s
I20260709 13:17:35.675340 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.643345 12930 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:35.680202 10205 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:35.685338 12953 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:35.685837 12984 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:17:35.687282 12609 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:17:35.689109 12986 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:17:35.689309 12983 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:17:35.691287 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1395) took 1071 ms. Trace:
I20260709 13:17:35.691813 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.692823 10200 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:35.691809 10052 rpcz_store.cc:276] 0709 13:17:34.654725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.914572 (+259847us) service_pool.cc:224] Handling call
0709 13:17:34.948597 (+ 34025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:35.095514 (+146917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:35.095555 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:35.169309 (+ 73754us) log.cc:844] Serialized 26 byte log entry
0709 13:17:35.359615 (+190306us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:35.359650 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:35.359717 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:35.690938 (+331221us) raft_consensus.cc:1685] finished
0709 13:17:35.690963 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:35.691187 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72469,"raft.queue_time_us":144857,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":22775,"thread_start_us":179557,"threads_started":1,"wal-append.queue_time_us":269585}
I20260709 13:17:35.668434 12955 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.699762 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.699544 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.698650 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.708758 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.710321 10204 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:35.715411 10056 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:35.722849 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:35.708212 12922 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:35.726413 12990 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:35.726959 12990 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:35.727237 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.727427 12990 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:35.731609 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56864.1us
I20260709 13:17:35.731907 12741 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.681900 12930 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.735672 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:35.736212 12930 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:35.683739 12940 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:35.773343 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72693.2us
W20260709 13:17:35.772693 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102639us
I20260709 13:17:35.767446 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:35.777391 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.777630 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.783905 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.784823 10897 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:35.785406 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:35.785737 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:35.787364 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 138980us
I20260709 13:17:35.789499 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:35.773928  9950 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.794804 12950 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.797895 12950 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.797650 12860 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.798849 12950 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:35.778241 12940 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:35.808102 12827 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.915s	user 0.015s	sys 0.002s
I20260709 13:17:35.744230 12741 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:35.809801 12959 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 13:17:35.809428  9950 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.841372 12827 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:35.811401 12851 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.852602 12620 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:35.853397 12620 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.856197 12620 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.814433 12934 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:35.824121 10049 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:17:35.857136 10049 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:35.779019 12609 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:35.841341 10204 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:35.841837 10130 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:17:35.847249 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60135.2us
I20260709 13:17:35.844642  9950 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.813174 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:35.864709 10898 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:35.823961 12950 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:35.875875 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:35.867651 10053 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:35.872187 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 139509us
I20260709 13:17:35.870388 10056 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:35.878464 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:35.886137  9950 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 5, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.898243 10054 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:35.900472 10055 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:35.865984 12904 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:35.905324 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.858696 12959 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.915503 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:35.887439 12950 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:35.915328  9950 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:35.924676 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 135311us
W20260709 13:17:35.931344 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.591s (SENT)
I20260709 13:17:35.857757 12998 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:17:35.926874 10894 raft_consensus.cc:2410] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:17:35.936461 10054 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:35.935561 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113203us
I20260709 13:17:35.941509 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:35.944708 12741 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:35.944912 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:35.946821 12609 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.942871 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:35.960013 10199 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:35.971727 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.968045  9950 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.975412 10054 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:35.960013 12620 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.983312 12883 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:17:35.987432 12959 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:35.989027 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57727.5us
W20260709 13:17:35.990113 10019 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.600s (SENT)
W20260709 13:17:35.990258 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74929.6us
I20260709 13:17:35.991652 12741 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:35.991849 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:35.992748 12741 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:35.979653 12966 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:35.995749 12955 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:35.996434 12955 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:35.997704 12955 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:35.994216 12741 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.991894 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:36.002676 12810 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:35.991786 12620 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.004380 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:36.005138 12980 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:35.987447 12940 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.006640 10893 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:36.008849 12987 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:36.009579 12987 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:36.010782 12904 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:35.978055 13003 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:36.011669 12904 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:36.013411 12860 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.016228 13012 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:36.018155 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:35.990669 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.027057 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 212) took 1144 ms. Trace:
I20260709 13:17:36.028378 10055 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:36.038144 10130 rpcz_store.cc:276] 0709 13:17:34.882897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.883297 (+   400us) service_pool.cc:224] Handling call
0709 13:17:36.026560 (+1143263us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:36.026931 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142685440}
I20260709 13:17:36.053264 10127 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:17:36.010805 13008 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:36.055209  9950 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:36.037530 12946 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:17:36.053901 10127 pending_rounds.cc:85] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:17:36.062134 10127 pending_rounds.cc:107] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:36.062597 10127 raft_consensus.cc:2889] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:36.062937 10054 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
W20260709 13:17:36.060418 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115676us
I20260709 13:17:36.069861 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:36.077054 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:35.977350 12848 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:36.085716 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68537.8us
I20260709 13:17:36.085819  9950 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:36.058784 12810 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.059284 12940 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:36.081296 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:36.090394 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100363us
I20260709 13:17:36.090648 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:36.061856 13003 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:36.098222 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 93994.6us
W20260709 13:17:36.099643 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.068500 12827 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.100589  9950 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:36.079794 13008 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:36.108302 10129 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:17:36.108348 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:36.111181 10054 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:36.010943 12987 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.112069 10055 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:17:36.105902 10019 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.606s (SENT)
I20260709 13:17:36.116586 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.115765  9950 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 3, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:36.134255 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:36.135102 10053 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.135525 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:36.136305 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:36.136936 13011 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:36.137475 13011 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.137285 12989 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:36.138008 13011 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.098269 12810 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.139057 12860 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:36.139712 12860 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:36.147331  9950 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:36.150449 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:36.151410 10055 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
I20260709 13:17:36.154748 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:36.158783 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.155936 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.105716 13003 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.167517 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:36.127604 12620 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:36.164083 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69696.9us
I20260709 13:17:36.168275 10129 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:36.192257 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.195247 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.167410 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64755us
I20260709 13:17:36.114038 12990 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.149744 13012 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:36.155524 12940 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:36.198014 10019 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.620s (SENT)
I20260709 13:17:36.203792 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:36.210572 10202 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.157819 12928 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:17:36.125473 12827 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.146780 12989 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:17:36.220355 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:36.221182 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.221333 10199 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:36.218500 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.224157 10053 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:36.163650 12955 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.166749 13003 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:36.234423 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 213) took 1328 ms. Trace:
I20260709 13:17:36.198215 13000 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:36.238997 10127 rpcz_store.cc:276] 0709 13:17:34.905849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:34.906055 (+   206us) service_pool.cc:224] Handling call
0709 13:17:36.035659 (+1129604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:36.072784 (+ 37125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:36.234141 (+161357us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400321c50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:36.234342 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2547990272}
W20260709 13:17:36.252661 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.218402 12928 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.207439 12686 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:17:36.204910 12966 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
W20260709 13:17:36.292732 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.264452 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71210.5us
W20260709 13:17:36.287323 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 68960.8us
I20260709 13:17:36.296671 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.235766 12959 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:36.308916 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.238255 12980 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:36.334429 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 173685us
I20260709 13:17:36.256379 12940 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:36.263419 12810 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.278995 12998 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.341639 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:36.311542 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.647s (SENT)
I20260709 13:17:36.255749 12928 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:36.292568 12987 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:36.347838 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:36.344808 10131 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:36.284515 12922 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:17:36.265066 12955 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.263710 12950 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.324038 13008 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.351303 10202 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
W20260709 13:17:36.535832 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 338125us
I20260709 13:17:36.555642 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:36.531560 10129 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:36.392277 10200 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:36.558353 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:36.471274 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:36.512312 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.560410 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:36.446851 10199 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:17:36.403759 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:36.389922 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:36.567096 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:36.423834 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:36.389394 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:36.568382 10196 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:36.304247 13003 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.324066 12851 raft_consensus.cc:2764] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:36.275781 13012 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:36.382957 12930 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.003s
W20260709 13:17:36.521632 13019 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:17:36.619284 10205 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:17:36.569283 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 260518us
I20260709 13:17:36.625324 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:36.589672 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 242653us
W20260709 13:17:36.426751 12934 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.002s
I20260709 13:17:36.617736 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:36.618660 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.641572 10199 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:36.535163 13013 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:17:36.325507 12963 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.289s
I20260709 13:17:36.558182 12990 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.622713 13034 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:36.656170 13034 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:36.657481 13034 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.658046 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.659812 12810 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:17:36.390180 12829 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.253s
I20260709 13:17:36.661859 12934 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:36.663051 12934 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:36.665709 12934 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.410514 12970 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.410872 12848 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:36.623114 13003 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.641948 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:36.400125 12827 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.614491 12966 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:36.673462 12966 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:36.674042 12966 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:36.563663 13022 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:36.445861 12856 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.679997 12856 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.651822 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.640842 12959 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:36.592772 12998 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.447739 12980 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.219s
I20260709 13:17:36.553596 13011 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.671231 12970 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.568629 12940 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:36.693925 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.690380 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.546829 12928 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:36.726177 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 165829us
I20260709 13:17:36.621502 12930 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:36.717357 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70948.1us
W20260709 13:17:36.721873 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 410331us
I20260709 13:17:36.347792 12812 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.274s
I20260709 13:17:36.545269 12860 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.714063 12959 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.714270 12851 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:36.724789 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 66870.8us
W20260709 13:17:36.739444 10019 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.670s (SENT)
I20260709 13:17:36.756135 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:36.758261 10019 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.787s (SENT)
I20260709 13:17:36.684502 12884 ts_tablet_manager.cc:1929] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 6.5
I20260709 13:17:36.760294 12884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806f35e66c234d569cbafd8b3981518f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:36.744810 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:36.746019 13024 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:17:36.749444 12928 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.749720 12966 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.750370 12959 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.750979 13011 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:36.751890 12970 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.783767 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:36.752226 13018 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:36.758875 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.766427 12860 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.730813 12856 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.773856 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.731869 12940 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:36.772047 10131 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:36.718086 13022 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.766893 13003 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.784754 10127 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:17:36.784708 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 94473.4us
W20260709 13:17:36.780854 12904 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.007s
I20260709 13:17:36.802639 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:36.808761 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:36.784695 12883 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.812355 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:36.813383 10200 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:36.815593 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:36.826180 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 436888us
I20260709 13:17:36.813119 13002 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:36.838585 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:36.847675 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.815074 13003 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.817783 12851 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:36.818568 13018 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:36.835172 12950 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:36.895504 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:36.856842 13040 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:36.857770 12998 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.858767 13022 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:36.825613 12940 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:36.831333 13036 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:36.919015 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:36.919358 13036 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.919978 13036 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:36.903903 13012 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:36.921212 13012 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.871975 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:36.876235 12966 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.829005 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:36.924088 10893 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.923629 13018 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.811199 12904 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.929075 12904 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:36.877179 12930 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.877789 12955 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:36.894217 10196 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:36.896515 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 93988.8us
W20260709 13:17:36.896677 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58174.1us
I20260709 13:17:36.900876 10199 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:36.906286 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:36.853426 12987 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:36.934302 10202 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 4: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:36.918291 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:36.918635 10894 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:36.818439 12883 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:36.943229 10204 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:36.937404 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89867.7us
I20260709 13:17:36.941397 12829 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:36.931741 13033 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:36.935482 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:36.948827 10205 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:36.907603 13040 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.836696 13020 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:17:36.955284 13040 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:36.950241 13033 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:36.960763 13033 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.950935 12829 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:36.964789 12829 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.968120 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:36.969403 10196 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 5: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:36.974136 12829 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.953712 12686 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:36.976090 12686 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:36.977613 12686 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.951761 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:36.983428 12904 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:36.984751 12966 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:36.985496 12966 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:36.985970 12966 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:36.987394 12851 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:36.988255 12851 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:36.987610 13012 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:36.990020 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.001972 10053 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:36.955322 12883 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:36.915279 12998 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:37.005241 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 247009us
I20260709 13:17:36.935281 12987 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:37.016014 10049 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:17:36.935057 12848 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:36.903960 13003 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:37.018594 10019 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.831s (SENT)
I20260709 13:17:37.021934 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:37.024035 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:37.024875 10052 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:37.031056 13042 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:17:37.046018 13022 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:37.045496 12851 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:37.049979 13022 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:37.057742 13003 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.058447 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:37.059499 10205 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:37.063629 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.064530 10048 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:37.018918 13031 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:17:36.952394 13002 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:17:36.999461 12963 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:17:37.068631 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.075414 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:37.082115 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.092028 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.066136 12934 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.055547 13022 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.096453 10050 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:37.068243 12851 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:37.035935 12998 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.108120 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:37.110505 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.091653 12987 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.098084 12848 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:17:37.227411 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.330340 10052 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:37.392328 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 377615us
I20260709 13:17:37.156086 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:37.387917 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:37.387197 10127 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.581s	user 0.019s	sys 0.005s
I20260709 13:17:37.416486 10127 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:17:37.419957 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:17:37.424499 10019 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.832s (SENT)
I20260709 13:17:37.425741 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:37.331954 12959 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.003s
W20260709 13:17:37.200052 13045 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:17:37.226380 13046 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
W20260709 13:17:37.385838 12860 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.003s
I20260709 13:17:37.160046 13031 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:37.116622 13042 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:37.124836 13000 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.107420 12934 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.124519 13050 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.338s
W20260709 13:17:37.426594 10893 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.009s
I20260709 13:17:37.203824 13043 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:37.406515 12848 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.460177 10898 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:37.450137 13031 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.214627 13033 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.480297 10893 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:37.124661 12851 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.228071 13036 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.428905 12922 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:17:37.439577 13040 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.464660 13037 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:37.195538 13022 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.437813 12959 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.475844 13000 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.478319 12860 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:37.483101 13031 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:37.524212 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 422574us
W20260709 13:17:37.517944 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 502885us
W20260709 13:17:37.612915 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 205411us
I20260709 13:17:37.507076 10200 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:37.482849 12940 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:37.604316 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 462683us
W20260709 13:17:37.579830 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:37.554955 12829 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.001s
I20260709 13:17:37.625402 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.627128 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.628237 13059 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:37.628285 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.629070 13059 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:37.629806 13054 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:17:37.630306 13059 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.631263 13043 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:37.631762 13043 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.631901 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.632210 13043 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:37.449714 13042 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:37.638424 13042 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.642261 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.647147 10053 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:37.644086 10052 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:37.647390 13042 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.650034 10055 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:37.653350 13036 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.660315 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:37.661835 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.662156 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.662937 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:37.662230 10205 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:37.663616 10196 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:37.664247 12904 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:37.664712 12904 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.573894 13055 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:37.615573 13052 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:17:37.581151 12998 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:37.626681 12884 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.865s	user 0.012s	sys 0.006s
I20260709 13:17:37.683748 10196 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:37.687801 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.695785 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:37.657100 13037 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.700474 12686 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.677125 12860 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:37.708510 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:37.710489 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 290694us
I20260709 13:17:37.678685 13055 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:37.665679 12904 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.702879 13036 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.718474 10055 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:37.611387 12848 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:37.722030 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 273518us
I20260709 13:17:37.727259 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.709656 13022 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.733565 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.734540 10131 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:37.735739 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:37.713421 13012 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:37.748373 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:37.749336 10127 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:37.745761 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.753494 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:37.755950 10129 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:37.755537 12860 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.757792 13020 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:37.760315 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:37.760946 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.761734 10050 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:17:37.763614 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63813.2us
I20260709 13:17:37.767457 13069 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:37.768317 13069 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.769465 13069 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.743160 12829 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:37.771927 12829 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:37.772590 12829 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:37.775342 10130 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:37.724968 13033 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:37.767506 12884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:60866 (request call id 501) took 6540 ms. Trace:
I20260709 13:17:37.783416 10126 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:37.785089 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.727351 13036 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.788136 10131 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:37.775658 10123 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:37.762768 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.799237 10050 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:37.793228 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:37.803020 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47958.6us
I20260709 13:17:37.807194 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:37.717648 12686 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:37.816879 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.716755 13064 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:17:37.748384 12848 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:37.825032 12851 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:37.826584 13000 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.714936 13043 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.809978 13020 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:37.788235 12959 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.830232 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.818356 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:37.841742 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.831486 12686 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.831678 10049 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:37.841886 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.832604 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:37.844494 10130 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:37.827548 13000 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:37.750939 12904 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.842707 10051 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:37.840353 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.844831 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.850706 10055 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:37.854609 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 421245us
I20260709 13:17:37.851404 10127 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:37.858628 10127 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:17:37.860787 10122 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:37.861169 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:37.861996 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.864022 10131 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.826977 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:37.872412 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.876060 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:37.881294 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.886312 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.889117 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:37.889742 10050 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:37.890174 10130 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:37.849457 13020 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.902837 10055 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:37.903290 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:37.902915 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:37.863057 10127 pending_rounds.cc:85] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 3
I20260709 13:17:37.905450 10127 pending_rounds.cc:107] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:17:37.905740 10127 raft_consensus.cc:2889] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:37.907912 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:37.910900 10122 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:37.882509 10128 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:37.782430 12884 rpcz_store.cc:276] 0709 13:17:31.235057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:31.240133 (+  5076us) service_pool.cc:224] Handling call
0709 13:17:32.170376 (+930243us) ts_tablet_manager.cc:934] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
0709 13:17:34.810659 (+2640283us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:35.805106 (+994447us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:36.645055 (+839949us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:36.759753 (+114698us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 13:17:37.698337 (+938584us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:37.739914 (+ 41577us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:37.767064 (+ 27150us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:37.767297 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":12716,"tablet-copy.queue_time_us":375535,"thread_start_us":590580,"threads_started":2}
I20260709 13:17:37.868611 13018 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:37.917691 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 240372us
I20260709 13:17:37.920533 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:37.943881 10049 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:17:37.904505 12966 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.918s	user 0.016s	sys 0.000s
I20260709 13:17:37.952320 10051 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:37.901171 13000 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:37.909024 13020 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:37.967073 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:37.811887 13055 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.967873 10049 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:37.981410 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 230240us
I20260709 13:17:37.985100 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:37.962384 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.013s	sys 0.026s
I20260709 13:17:37.936060 13040 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:37.965751 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:37.950304 13050 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:17:37.993453 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 228248us
W20260709 13:17:37.987893 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.014s	sys 0.026s
W20260709 13:17:38.004330 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.008940 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:38.011191 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.011961 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.012432 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.012763 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.008586 13018 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:37.985088 12686 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.842188 13037 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:37.955022 12966 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.021844 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:38.041534 10051 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:38.043814 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.045679 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48641.2us
I20260709 13:17:38.006510 13002 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:38.050565 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:37.989455 12884 tablet_copy_client.cc:806] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:38.023069 10893 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:17:38.019237 13050 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:38.068370 10048 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:38.036041 12860 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:38.078300 12860 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.078965 12860 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.059612 13074 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:17:38.070406 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61615.2us
I20260709 13:17:38.067530 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.048651 13055 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.076541 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.076802 13072 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:38.065369 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:38.108263 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.120401 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55123.6us
I20260709 13:17:38.124094 10124 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:38.072194 13076 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:38.094126 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6965672660001019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
W20260709 13:17:38.129521 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6893672660000902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:17:38.130033 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6785672660000728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 13:17:38.130555 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6540791980000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24180000000000001s for a response
W20260709 13:17:38.130956 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6308672659999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 13:17:38.124054 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:38.037070 12686 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.153486 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:38.111764 12884 tablet_copy_client.cc:670] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:38.111974 13087 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:38.159978 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:38.162485 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:38.068589 13002 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.126986 13036 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.128893 13076 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:38.171001 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:38.175246 10049 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:38.178364 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.124179 13012 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:38.179347 10049 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:38.161319 13043 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.111701 13074 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:17:38.160728 13075 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:38.165560 13011 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:38.186980 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55334.4us
I20260709 13:17:38.067996 13000 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.189221 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:38.198002 10019 connection.cc:256] RPC call timeout handler was delayed by -1.6209672659999796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
I20260709 13:17:38.201151 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:38.201539 10019 connection.cc:256] RPC call timeout handler was delayed by -1.601167266000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
I20260709 13:17:38.157920 12998 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:17:38.201956 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5993071500001861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:17:38.202770 10019 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.875s (SENT)
I20260709 13:17:38.170707 13087 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:38.206638 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:38.208201 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:38.163856 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:38.181180 13076 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.156967 13094 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:17:38.215706 10196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:17:38.220866 10019 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.925s (SENT)
I20260709 13:17:38.220782 10196 raft_consensus.cc:606] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:38.199014 13011 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.225322 13011 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.166199 13018 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:38.231245 13077 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:17:38.195384 13038 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:38.189810 13084 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:38.224186 13086 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:38.243959 13081 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:17:38.244566 13093 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:38.246791 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:38.239377 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.260274 10131 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:38.203718 12883 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:38.263505 10051 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:38.296375 10897 raft_consensus.cc:1800] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: voting while copying based on last-logged opid 6.5
I20260709 13:17:38.299347 10897 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.299556 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.300505 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:38.286633 13099 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:17:38.287537 13097 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:38.289104 13069 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:38.222429 12998 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:38.238590 13072 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:38.294981 13086 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.262849 13084 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.311545 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.235538 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.294721 13043 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.313525 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:38.309077 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88248.2us
W20260709 13:17:38.333228 10019 connection.cc:256] RPC call timeout handler was delayed by -1.581307150000157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:17:38.333729 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5795071500001541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:17:38.334623 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5588672660001066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 13:17:38.335139 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5372672660000717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
W20260709 13:17:38.335606 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5327672660000644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260709 13:17:38.336035 10019 connection.cc:256] RPC call timeout handler was delayed by -1.5174672660000397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:17:38.317628 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.337636 10052 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:38.320250 10051 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:38.320780 10049 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:38.339846 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.340736 10049 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:38.331593 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.234979 13020 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.350359 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:38.351212 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.353211 10131 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:38.314191 12998 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:38.327289 13069 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:38.336457 10019 connection.cc:256] RPC call timeout handler was delayed by -1.4688071500002025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 13:17:38.360424 10019 connection.cc:256] RPC call timeout handler was delayed by -1.458067266000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22850000000000004s for a response
W20260709 13:17:38.360904 10019 connection.cc:256] RPC call timeout handler was delayed by -1.4193071500001224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
I20260709 13:17:38.306243 13097 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:38.314519 13072 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.317458 13094 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.215755 13050 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.357409 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.359117 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.321573 13086 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.364441 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.364601 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.365392 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.270731 13038 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.361173 12998 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.332674 13084 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:38.355929 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 231977us
I20260709 13:17:38.380630 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.382381 10125 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:38.369110 13072 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:38.358740 13031 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.008s
I20260709 13:17:38.384467 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.360460 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.388337 10122 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:38.388172 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.377101 10051 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:38.379405 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.351176 13043 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.382907 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:38.331840 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 123700us
I20260709 13:17:38.372979 10054 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:38.403410 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.403393 10049 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:38.404171 10122 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:38.404078 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.381959 13069 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:38.342393 13087 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.374742 13094 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.487468 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 232) took 1495 ms. Trace:
I20260709 13:17:38.479328 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.488583 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.408407 10051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82960478409442549781547254b9d8ea"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:49976
W20260709 13:17:38.436344 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.498090 10051 raft_consensus.cc:606] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:38.498587 10051 raft_consensus.cc:614] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:38.493353 10052 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:17:38.498768 10055 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:38.500559 10049 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:38.395920 13002 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:38.403343 12848 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:17:38.495596 10200 rpcz_store.cc:276] 0709 13:17:36.991999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.992162 (+   163us) service_pool.cc:224] Handling call
0709 13:17:36.992476 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:37.614800 (+622324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.487054 (+872254us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540039dd54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:38.487345 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182716800}
I20260709 13:17:38.419910 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 533) took 1194 ms. Trace:
I20260709 13:17:38.582983 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:38.583775 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.584813 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.504514 10894 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:38.402508 12686 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:38.470400 13058 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.865s	user 0.000s	sys 0.008s
I20260709 13:17:38.456239 10897 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:38.553470 10892 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:38.553906 10895 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:38.607347 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.392992 13031 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.414103 13018 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.144s
I20260709 13:17:38.546744 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.558408 13020 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.561657 10124 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:38.561626 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.564073 13043 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:38.563854 13011 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.565143 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.632787 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:38.418022 12829 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.005s
I20260709 13:17:38.537122 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:38.582729 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.419593 13098 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.506717 13038 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.564498 13106 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
W20260709 13:17:38.599977 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 217264us
I20260709 13:17:38.424222 13108 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:38.608234 10127 raft_consensus.cc:2368] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:17:38.557529 13069 raft_consensus.cc:993] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
W20260709 13:17:38.592307 13059 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.018s	sys 0.026s
I20260709 13:17:38.546658 12963 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:17:38.612550 10896 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:38.616266 10899 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:17:38.624732 10125 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:38.633040 10131 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:38.683624 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1438) took 1618 ms. Trace:
I20260709 13:17:38.582325 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.593977 10891 rpcz_store.cc:276] 0709 13:17:37.225977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.226680 (+   703us) service_pool.cc:224] Handling call
0709 13:17:38.418496 (+1191816us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:17:38.418858 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.419226 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:38.419292 (+    66us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:38.419385 (+    93us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:38.419463 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:38.419563 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.419764 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4917504}
I20260709 13:17:38.652557 13011 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.640393 12829 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:38.655813 13022 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.913s	user 0.015s	sys 0.000s
I20260709 13:17:38.659415 12851 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:38.661746 12934 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.618695 13012 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:38.668205 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.709964 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.710896 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.668566 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1585) took 1053 ms. Trace:
I20260709 13:17:38.669157 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.669425 13094 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:38.671262 12860 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.005s
I20260709 13:17:38.713372 12860 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.714679 10898 raft_consensus.cc:2368] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:38.715919 12860 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.684098 10047 rpcz_store.cc:276] 0709 13:17:37.065202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.067266 (+  2064us) service_pool.cc:224] Handling call
0709 13:17:37.084385 (+ 17119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:37.107761 (+ 23376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:37.107805 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:37.108409 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:17:37.391656 (+283247us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:37.391700 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.391774 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.469309 (+1077535us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:38.673468 (+204159us) raft_consensus.cc:1685] finished
0709 13:17:38.673503 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.683532 (+ 10029us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":526,"raft.queue_time_us":14172,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":45843,"spinlock_wait_cycles":13705472,"thread_start_us":282768,"threads_started":1,"wal-append.queue_time_us":391598}
I20260709 13:17:38.684706 10126 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:38.712261 10048 rpcz_store.cc:276] 0709 13:17:37.614929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.615108 (+   179us) service_pool.cc:224] Handling call
0709 13:17:38.668304 (+1053196us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400369f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:38.668460 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1001657728}
I20260709 13:17:38.690030 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.691570 10122 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:38.696313 13107 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:17:38.697038 10019 connection.cc:256] RPC call timeout handler was delayed by -1.3059672660001524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24540000000000003s for a response
W20260709 13:17:38.721297 10019 connection.cc:256] RPC call timeout handler was delayed by -1.132207150000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 13:17:38.721798 10019 connection.cc:256] RPC call timeout handler was delayed by -1.1288546130001578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:17:38.560323 13097 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.702404 12686 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:17:38.739182 13097 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.739632 12686 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:38.740146 12686 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:38.741873 13079 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.578s	user 0.004s	sys 0.000s
I20260709 13:17:38.699224 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.745900 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.698158 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:38.685242 13055 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.001s
W20260709 13:17:38.692097 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120387us
I20260709 13:17:38.703668 13069 raft_consensus.cc:1081] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:17:38.672418 13076 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:38.704625 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.705530 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.680950 13108 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.716537 13018 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.781538 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.782577 10124 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:38.782073 13018 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.782982 10900 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:38.756516 13012 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:38.784780 13018 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:38.639392 12904 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.006s
I20260709 13:17:38.785259 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.786141 10124 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:38.786514 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.787356 10125 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.787488 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.789151 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.789902 10128 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.790726 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.719684 10123 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:38.791801 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.720633 10894 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:38.794893 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.700714 13099 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:17:38.800282 13031 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.803449 12998 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.805629 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:38.805984 12998 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.807332 13064 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:38.807953 13064 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.808204 13086 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.809369 10897 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:38.810377 13064 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.812077 12829 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:38.820926 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.821645 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.822443 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.822932 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.823772 10123 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6.
I20260709 13:17:38.823721 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:38.824934 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.825094 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:38.826303 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.826408 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:38.826973 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:38.827662 10048 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:38.722347 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.829002 10122 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:38.735641 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.681185 13084 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:38.738147 13057 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.106s	user 0.000s	sys 0.005s
I20260709 13:17:38.701962 12851 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:38.741247 12686 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.747434 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:38.687261 13059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.019s	sys 0.027s
I20260709 13:17:38.674108 13098 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.687144 13111 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:17:38.763289 13106 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:38.773100 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:38.776657 12966 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:38.779204 13108 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:38.832336 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:38.877270 10053 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:38.834296 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:38.837904 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:38.840898 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102418us
I20260709 13:17:38.834378 10129 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:38.778867 12987 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:38.887372 10125 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:38.889151 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.842691 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:38.849888 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161778us
I20260709 13:17:38.891464 10049 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.853845 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:38.784830 13012 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:38.860847 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 87876.9us
I20260709 13:17:38.892772 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.892081 10050 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.894094 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.864203 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:38.827355 12860 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:38.894842 10894 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:38.894908 12860 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:38.895311 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.895581 12860 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:38.896103 10129 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:38.896983 13111 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:38.897274 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.880690 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.883296 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:38.899569 13098 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:38.900449 10054 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:38.901822 10054 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:38.836647 12812 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:38.903193 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1424) took 1915 ms. Trace:
I20260709 13:17:38.903302 12812 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:38.904078 12812 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:38.898883 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.905440 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.906572 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.874622 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:38.907121 12686 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:38.908651 10054 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:38.909062 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:38.903700 10056 rpcz_store.cc:276] 0709 13:17:36.988000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:36.988178 (+   178us) service_pool.cc:224] Handling call
0709 13:17:36.988628 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:37.045737 (+ 57109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:37.045796 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:37.046594 (+   798us) log.cc:844] Serialized 26 byte log entry
0709 13:17:37.159939 (+113345us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:37.160007 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:37.160113 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:38.737129 (+1577016us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:38.902812 (+165683us) raft_consensus.cc:1685] finished
0709 13:17:38.902835 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:38.903079 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49173,"raft.queue_time_us":65271,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":150,"spinlock_wait_cycles":83318528,"thread_start_us":159068,"threads_started":2,"wal-append.queue_time_us":521111}
I20260709 13:17:38.912758 10055 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:17:38.890748 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:38.811019 13064 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:38.897501 13111 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:38.840647 10204 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:38.930552 13000 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:39.184824 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:39.292465 13000 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.293272 13000 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:39.293535 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 108830us
I20260709 13:17:39.293653 10129 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:39.253085 13072 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.004s
W20260709 13:17:39.187732 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.308548 13072 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.947634 13088 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.347s
I20260709 13:17:38.914574 10049 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:38.845891 13034 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:38.896052 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:39.345595 10047 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:38.988484 13076 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.080902 10048 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:17:39.274554 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 314822us
W20260709 13:17:39.180917 13087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.008s	sys 0.019s
I20260709 13:17:39.296443 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:39.243980 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.222833 13018 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.220631 12848 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:38.913930 10123 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:38.990460 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:39.273422 13097 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.002s
I20260709 13:17:39.297343 13111 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:38.863922 13099 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:39.129112 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.398720 10130 raft_consensus.cc:2410] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:17:39.181113 13033 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.012s	sys 0.026s
I20260709 13:17:39.079173 10050 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:38.971515 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54231.8us
I20260709 13:17:39.299031 13064 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.287611 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.406438 10126 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:39.406741 13097 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:39.292711 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:38.959408 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 66263.6us
I20260709 13:17:39.327263 10129 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:39.366577 13000 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:38.947969 12998 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.344s
I20260709 13:17:39.350288 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:39.387022 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:39.390064 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:39.019906 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.381494 12829 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.408315 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:39.353904 13087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.008s	sys 0.021s
W20260709 13:17:39.416379 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58580.6us
I20260709 13:17:39.406520 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.354158 13121 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.460726 13121 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.386998 13116 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:39.421167 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:39.464727 10199 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:39.464599 13116 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.344906 13034 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:39.467088 13119 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:39.426255 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.467382 13034 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.468223 10197 raft_consensus.cc:2368] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 6. Current term is 7.
I20260709 13:17:39.410094 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:39.439970 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89853.1us
I20260709 13:17:39.469937 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:39.356218 13031 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.407346 13098 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:17:39.471575 13031 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.473055 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.473989 13031 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.474202 13000 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.474363 13119 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:39.474826 13119 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:39.475272 13119 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.446744 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:39.475602 13121 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.476429 13121 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:39.477278 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:39.478015 13111 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:39.413841 13033 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.012s	sys 0.029s
I20260709 13:17:39.404554 13018 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.479875 13097 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:39.480036 10125 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.456050 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.459710 12966 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.481482 13018 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.481667 12829 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.482154 13018 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.482825 13018 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.484076 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:39.401361 12848 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.439275 13073 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:17:39.354542 12860 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.417340 13002 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:17:39.427413 13106 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:17:39.456898 10203 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:17:39.150595 13108 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:39.489039 10898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.016s	sys 0.044s
I20260709 13:17:39.490917 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:39.446177 12987 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.005s
I20260709 13:17:39.506645 10125 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:39.528136 10898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.016s	sys 0.045s
I20260709 13:17:39.539062 10196 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:17:39.541636 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48966.2us
I20260709 13:17:39.544253 10202 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.544140 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.477317 13020 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.557863 10894 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:39.562299 10199 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
W20260709 13:17:39.564159 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73339.1us
I20260709 13:17:39.584022 10202 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:17:39.563205 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.586221 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.519614 13098 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.588732 10201 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:17:39.591076 10201 pending_rounds.cc:85] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 6
I20260709 13:17:39.591442 10201 pending_rounds.cc:107] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:17:39.591682 10201 raft_consensus.cc:2889] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:39.583086 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.520537 13097 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:39.593515 10124 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:17:39.529982 13002 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:39.604610 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:39.605928 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:39.584764 13106 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:39.562422 13074 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.620162 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.622318 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.628409 12860 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.629724 13129 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:39.635228 10048 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:39.635627 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:39.636371 10049 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.630883 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:39.642966 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.641731 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:39.594853 13099 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:17:39.602712 12686 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
I20260709 13:17:39.650266 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.651470 10049 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:39.656648 10125 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:39.657214 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 366877us
I20260709 13:17:39.657405 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:39.630846 13129 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:39.561939 12998 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:17:39.652063 10128 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.862s	user 0.017s	sys 0.000s
I20260709 13:17:39.647264 10047 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:39.658584 10048 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.652495 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.589000 13020 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.724382 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:39.709721 13073 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.721395 12998 consensus_queue.cc:1243] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:39.656006 12860 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:39.719736 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.722097 10128 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:39.645247 13072 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.725330 10199 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:39.733697 10125 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:39.611680 13097 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.603820 13098 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.739138 10203 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:39.650686 13002 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:39.739413 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2720) took 1079 ms. Trace:
I20260709 13:17:39.647998 13122 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:39.750030 10128 rpcz_store.cc:276] 0709 13:17:38.660167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.761500 (+101333us) service_pool.cc:224] Handling call
0709 13:17:39.739310 (+977810us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:39.725960 12812 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.821s	user 0.013s	sys 0.001s
I20260709 13:17:39.743790 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:39.735102 13073 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.710285 12829 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.718981 13088 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.746829 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.758857 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:39.753587 10095 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.048s (SENT) [suppressed 28 similar messages]
I20260709 13:17:39.766213 10048 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:39.647281 13074 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.771976 10125 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 7 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:17:39.742383 12998 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:39.767746 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.779918 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.786458 13136 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:39.787072 13136 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:39.787513 13136 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.757407 12860 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:39.760762 13002 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:39.764930 13064 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.765910 13122 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.762146 12812 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.803957 10203 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:39.802417 13122 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.778235 12998 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:39.821343 13131 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:17:39.822614 10203 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:39.823648 13133 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:17:39.825279 13119 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.826792 10201 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:39.815393 12812 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.826608 13093 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:39.831388 13093 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:39.831847 13128 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:17:39.832479 13093 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.833709 13093 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.834527 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.835623 10128 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.836284 13119 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.837556 13116 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.839249 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.839643 13098 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.840533 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.845542 13097 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.817996 13129 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.847975 13084 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.850929 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.851913 13108 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.852540 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.853159 10127 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.853778 13018 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.860492 12860 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.863178 12851 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.864355 12851 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.864907 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.866106 10198 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:39.866508 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.867537 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.868397 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.868420 10197 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:39.869808 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:39.869863 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.871222 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:39.871320 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.871939 10199 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:17:39.872748 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.873286 10199 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:39.868607 13085 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:39.874153 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.874032 13085 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.872766 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.876291 13086 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.880684 12848 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.881009 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.881464 13123 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:39.882275 13116 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:39.882122 13123 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.877070 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:39.882933 13074 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.886548 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2689) took 1292 ms. Trace:
I20260709 13:17:39.887391 13073 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.877125 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 234) took 1561 ms. Trace:
I20260709 13:17:39.888096 13073 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:39.882915 13116 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.870791 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.890733 10196 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:39.773872 13088 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.892122 10198 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:39.888068 10051 rpcz_store.cc:276] 0709 13:17:38.316122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.605230 (+289108us) service_pool.cc:224] Handling call
0709 13:17:38.629217 (+ 23987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:38.902517 (+273300us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:38.902927 (+   410us) log.cc:844] Serialized 14 byte log entry
0709 13:17:39.302207 (+399280us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.308452 (+  6245us) log.cc:844] Serialized 26 byte log entry
0709 13:17:39.318793 (+ 10341us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:39.318934 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:39.876709 (+557775us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:39.876771 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:39.876822 (+    51us) raft_consensus.cc:1685] finished
0709 13:17:39.876911 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:39.877058 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99415,"prepare.queue_time_us":933,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":101514,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":273,"spinlock_wait_cycles":618180992,"thread_start_us":301644,"threads_started":1,"wal-append.queue_time_us":688026}
I20260709 13:17:39.892970 13088 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.893627 13088 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:39.893924 13116 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.876645 13085 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.888932 13073 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.894254 13088 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.896112 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1461) took 1062 ms. Trace:
I20260709 13:17:39.896534 13073 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.896598 10052 rpcz_store.cc:276] 0709 13:17:38.833332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.840487 (+  7155us) service_pool.cc:224] Handling call
0709 13:17:38.865534 (+ 25047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.439918 (+574384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.439951 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:39.439994 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:39.440030 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:39.895852 (+455822us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:39.895905 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:39.896042 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320332544}
I20260709 13:17:39.897476 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.806008 13002 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.899091 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1602) took 2141 ms. Trace:
I20260709 13:17:39.899572 10053 rpcz_store.cc:276] 0709 13:17:37.757801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.880382 (+122581us) service_pool.cc:224] Handling call
0709 13:17:37.910512 (+ 30130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.877917 (+967405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:39.898810 (+1020893us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b5400199954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:39.899023 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222562560}
I20260709 13:17:39.899574 13002 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.902458 12883 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:39.902961 12883 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.851153 13020 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:39.904078 12883 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.905539 12883 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:39.905885 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2705) took 1251 ms. Trace:
I20260709 13:17:39.906354 10130 rpcz_store.cc:276] 0709 13:17:38.654685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.771000 (+116315us) service_pool.cc:224] Handling call
0709 13:17:39.905671 (+1134671us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54001428d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:39.905820 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393140096}
I20260709 13:17:39.898893 13031 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:39.907486 13031 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.907755 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:39.907786 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.908456 13031 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:39.909366 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.909915 13031 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.887069 10124 rpcz_store.cc:276] 0709 13:17:38.594658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.794564 (+199906us) service_pool.cc:224] Handling call
0709 13:17:39.886311 (+1091747us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540036b354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:39.886470 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826839296}
I20260709 13:17:39.910722 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:39.913324 10124 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:39.913759 10130 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:39.915203 13123 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:39.916016 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:17:39.916841 10130 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:39.917251 10130 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:39.918650 13108 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:39.879832 12998 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.920320 12998 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.920780 13108 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:39.922005 13108 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:39.923022 13085 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:39.923664 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.923954 13108 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:39.924208 13085 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:39.925732 13085 consensus_queue.cc:1243] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:39.928978 12904 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.929479 12848 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:39.930083 12848 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.931303 12848 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.925967 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.932525 13034 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.928661 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:39.934132 13086 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:39.934644 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.935392 10896 raft_consensus.cc:1800] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: voting while copying based on last-logged opid 6.5
I20260709 13:17:39.934157 13088 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:39.939888 12848 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:39.942715  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:39.944200 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:39.944866 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.945179 13085 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:39.906705 13020 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:39.944988  9953 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:39.945937 13020 consensus_queue.cc:1243] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:39.946585  9953 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 6, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.948251 12904 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:39.948921 12904 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:39.984187 10127 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:39.936012 10896 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:39.957304 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60591.1us
I20260709 13:17:39.962338 10128 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
W20260709 13:17:39.963433 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 321821us
W20260709 13:17:39.976516 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:39.991180 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.985711 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:39.994565 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:39.989897 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:39.924352 13108 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:39.997020 10899 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:39.997336 13136 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:39.998586 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:39.999560 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.000391 13020 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.000662 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.001022 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.001415 10197 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:40.001489 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.002902 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.003079 10196 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:17:40.004526 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.005491 10199 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:40.005954 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:40.007308 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:40.008149 10202 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:40.008311 13073 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:40.009186 13073 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:40.013819 13134 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:40.014593 13086 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.015232 13086 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:40.015677 10895 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:40.016183 13086 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:40.017035 13034 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:40.001051 13020 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:40.018122 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:40.018930 10128 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:40.018669 13020 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:40.020052 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:39.949280  9953 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:40.022265 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:40.026002 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.009s	sys 0.008s
W20260709 13:17:40.027413 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.010s	sys 0.008s
W20260709 13:17:40.027930 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 615776us
I20260709 13:17:40.030015 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.031857 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.032841 10199 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:40.037881 10123 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.004s	sys 0.012s
W20260709 13:17:40.038324 10123 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.004s	sys 0.012s
W20260709 13:17:40.039930 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.011s	sys 0.023s
W20260709 13:17:40.040623 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.011s	sys 0.023s
I20260709 13:17:40.042271 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:40.044881 12987 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.046887 12987 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:40.047422 13031 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:40.048584 13031 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:40.050137 12987 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:40.049962 13031 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:40.054476 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.055212 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:40.056823 10900 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:40.058197 10126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.012s	sys 0.006s
I20260709 13:17:40.053439 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:40.018122 13121 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.541s	user 0.004s	sys 0.007s
I20260709 13:17:40.058363 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:40.059728 10126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.012s	sys 0.006s
I20260709 13:17:40.068838  9953 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 5, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 5 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:40.073112 10130 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:40.074096  9950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 205) took 17961 ms (client timeout 15000 ms). Trace:
I20260709 13:17:40.075913  9953 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:40.060883 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:40.015707 13134 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:40.083730 13134 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:40.084515 13021 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:40.085444 13021 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:40.085006 13134 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:40.086884 13021 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:40.078652  9953 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 7, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 7 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:40.062636 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:40.096913 10897 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:40.098872 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.036410 13093 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:40.107452 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:40.107695 13093 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.108568 13093 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:40.099038 13087 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.006s	sys 0.032s
I20260709 13:17:40.113534 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.036006 12686 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:40.113636 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.029435 13111 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:17:40.167797 10053 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:40.031312 13084 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
W20260709 13:17:40.117707 13087 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.006s	sys 0.033s
I20260709 13:17:40.117412 12904 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:40.410017 10125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.021s	sys 0.023s
I20260709 13:17:40.812870 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:40.101998 13121 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:40.279948 10048 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:17:40.813516 10125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.059s	sys 0.060s
I20260709 13:17:40.813843 10053 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:40.293423 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.410864 10202 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:40.381313 10127 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:40.350601 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:40.426286 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:40.827404 10051 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:40.284895 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.027s	sys 0.018s
W20260709 13:17:40.828801 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.369s	user 0.027s	sys 0.036s
W20260709 13:17:40.324345 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.826609 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1435) took 1403 ms. Trace:
W20260709 13:17:40.607950 10131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.009s	sys 0.029s
I20260709 13:17:40.829473 10047 raft_consensus.cc:2410] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:17:40.495144 10898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.004s	sys 0.026s
W20260709 13:17:40.336625 10204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.006s	sys 0.038s
W20260709 13:17:40.531850 10096 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.599s (SENT)
W20260709 13:17:40.433496 10129 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.012s	sys 0.013s
I20260709 13:17:40.210950 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.206209  9953 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 2 to 5, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 5 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:40.897414 10128 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:40.887712 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:40.901383 10052 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:40.901219  9953 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 5, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 5 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:40.847781 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.41398e+06us
I20260709 13:17:40.904358  9953 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:17:40.820509 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:40.907291  9953 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 4 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:40.909068 10124 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.010s	sys 0.048s
W20260709 13:17:40.909543 10129 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.022s	sys 0.023s
W20260709 13:17:40.909951 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 378173us
W20260709 13:17:40.910518 10204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.291s	user 0.014s	sys 0.049s
W20260709 13:17:40.910840 10898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.004s	sys 0.045s
W20260709 13:17:40.832453 10131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.011s	sys 0.034s
I20260709 13:17:40.916656 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:40.919175 10200 raft_consensus.cc:606] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:40.919857 10124 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.011s	sys 0.048s
I20260709 13:17:40.919250 10056 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:40.831736 10127 rpcz_store.cc:276] 0709 13:17:39.423747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.380580 (+956833us) service_pool.cc:224] Handling call
0709 13:17:40.381086 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.826156 (+445070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.826507 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.912767 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:40.918036 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.641s (SENT)
I20260709 13:17:40.929888 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:40.930815 10048 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:40.930964 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1436) took 1506 ms. Trace:
I20260709 13:17:40.931634 10127 rpcz_store.cc:276] 0709 13:17:39.424078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.929736 (+1505658us) service_pool.cc:224] Handling call
0709 13:17:40.930195 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.930472 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.930506 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.930579 (+    73us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:40.930621 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.930696 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.930870 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.933920 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.935258 10201 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:40.939628 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1438) took 1509 ms. Trace:
I20260709 13:17:40.940451 10127 rpcz_store.cc:276] 0709 13:17:39.430664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.938063 (+1507399us) service_pool.cc:224] Handling call
0709 13:17:40.938655 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.939042 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.939089 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.939197 (+   108us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:40.939258 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.939346 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.939523 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.942394 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.336129 13099 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:40.568800 13066 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.610s	user 0.017s	sys 0.037s
W20260709 13:17:40.425786 13122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.010s	sys 0.017s
W20260709 13:17:40.944201 13122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.013s	sys 0.017s
I20260709 13:17:40.943672 13099 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:40.381412 13073 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:40.380820 13136 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:40.946805 13099 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:40.950577 12848 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.010s	user 0.009s	sys 0.000s
I20260709 13:17:40.951978 10201 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:40.953004 12963 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.953397 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.953749 12963 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:40.943706 13066 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.985s	user 0.019s	sys 0.046s
I20260709 13:17:40.957430 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:40.961548 10052 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:40.964221 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1422) took 2336 ms. Trace:
I20260709 13:17:40.964792 10126 rpcz_store.cc:276] 0709 13:17:38.627447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.407455 (+780008us) service_pool.cc:224] Handling call
0709 13:17:39.443012 (+ 35557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.443406 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.443451 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.468106 (+ 24655us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.067855 (+599749us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:40.067899 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.067973 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:40.963920 (+895947us) raft_consensus.cc:1685] finished
0709 13:17:40.963948 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.964137 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":599317,"threads_started":1,"wal-append.queue_time_us":1304877}
I20260709 13:17:40.969897 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1441) took 1467 ms. Trace:
I20260709 13:17:40.970283 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.973454 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:40.918406 10205 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:40.957616 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:40.813103 13147 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.960322 10056 raft_consensus.cc:1217] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-6.4]   Dedup: 4.3->[6.4-6.4]
I20260709 13:17:40.984283 10056 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:17:40.968721 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1440) took 1478 ms. Trace:
I20260709 13:17:40.976853 10201 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:40.970499 10128 rpcz_store.cc:276] 0709 13:17:39.502383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.968940 (+1466557us) service_pool.cc:224] Handling call
0709 13:17:40.969322 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.969530 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.969557 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.969604 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:40.969644 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.969704 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.969825 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.985117 10126 rpcz_store.cc:276] 0709 13:17:39.490275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.966483 (+1476208us) service_pool.cc:224] Handling call
0709 13:17:40.966896 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.968347 (+  1451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.968382 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.968434 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:40.968471 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.968539 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.968637 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.986689 10128 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:40.738132 13002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.014s	sys 0.014s
I20260709 13:17:40.987622 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1443) took 1458 ms. Trace:
I20260709 13:17:40.988045 10128 rpcz_store.cc:276] 0709 13:17:39.529676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.986016 (+1456340us) service_pool.cc:224] Handling call
0709 13:17:40.986427 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.987290 (+   863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.987541 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.977265 10198 raft_consensus.cc:2376] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9. Re-sending same reply.
I20260709 13:17:40.992535 10198 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:40.992249 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:40.996558 10050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.037s	sys 0.032s
W20260709 13:17:40.997299 10050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.038s	sys 0.032s
I20260709 13:17:40.998239 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1442) took 1478 ms. Trace:
I20260709 13:17:40.999571 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:41.001240 10198 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:40.167428 13098 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.017326 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 231) took 1276 ms. Trace:
I20260709 13:17:40.982276 10048 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:40.153558 13116 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.998869 10127 rpcz_store.cc:276] 0709 13:17:39.520150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.971289 (+1451139us) service_pool.cc:224] Handling call
0709 13:17:40.971636 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.997798 (+ 26162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.997827 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:40.997876 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:40.997903 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.997964 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:40.998132 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:41.033428 10127 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:41.034293 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 233) took 1267 ms. Trace:
W20260709 13:17:40.987864 13002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.018s	sys 0.014s
I20260709 13:17:41.041270 13073 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:40.977496 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.049393 10205 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:40.208256 13111 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:41.017997 10128 rpcz_store.cc:276] 0709 13:17:39.741188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.016119 (+1274931us) service_pool.cc:224] Handling call
0709 13:17:41.016596 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.016860 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.016899 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.016957 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:41.017008 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.017081 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.017238 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:41.020138 10894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.026s	sys 0.047s
W20260709 13:17:41.061066 10894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.026s	sys 0.047s
I20260709 13:17:40.207479 13034 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:40.285943 13064 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:40.380283 13106 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.155612 12686 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:41.002413 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:41.030035 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:40.074738  9950 rpcz_store.cc:269] 0709 13:17:22.131402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.184284 (+ 52882us) service_pool.cc:224] Handling call
0709 13:17:40.073955 (+17889671us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:36.860869 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:36.861689 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:36.861749 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:17:36.861788 (+    39us) tablet.cc:662] Decoding operations
0709 13:17:36.881219 (+ 19431us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:36.881320 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:36.881356 (+    36us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:17:36.883097 (+  1741us) tablet.cc:701] Row locks acquired
0709 13:17:36.883222 (+   125us) write_op.cc:260] PREPARE: finished
0709 13:17:36.883524 (+   302us) write_op.cc:270] Start()
0709 13:17:36.883732 (+   208us) write_op.cc:276] Timestamp: P: 1783603056883428 usec, L: 0
0709 13:17:36.883757 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:36.919347 (+ 35590us) log.cc:844] Serialized 5915 byte log entry
0709 13:17:38.055608 (+1136261us) op_driver.cc:464] REPLICATION: finished
0709 13:17:38.261637 (+206029us) write_op.cc:301] APPLY: starting
0709 13:17:38.301711 (+ 40074us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:38.373686 (+ 71975us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:38.393600 (+ 19914us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:38.744940 (+351340us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:38.745330 (+   390us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:17:38.745376 (+    46us) write_op.cc:312] APPLY: finished
0709 13:17:38.745890 (+   514us) log.cc:844] Serialized 107 byte log entry
0709 13:17:38.746126 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:38.746329 (+   203us) write_op.cc:454] Released schema lock
0709 13:17:38.746407 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:38.878994 (+132587us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:38.917901 (+ 38907us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10024,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":143810,"apply.run_cpu_time_us":10453,"apply.run_wall_time_us":538637,"num_ops":15,"prepare.queue_time_us":602273,"prepare.run_cpu_time_us":43989,"prepare.run_wall_time_us":188472,"raft.queue_time_us":426857,"raft.run_cpu_time_us":21654,"raft.run_wall_time_us":97413,"replication_time_us":1171668,"thread_start_us":968524,"threads_started":4,"wal-append.queue_time_us":139229,"wal-append.run_cpu_time_us":36626,"wal-append.run_wall_time_us":2710355}]]}
I20260709 13:17:41.034756 10127 rpcz_store.cc:276] 0709 13:17:39.767102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.032512 (+1265410us) service_pool.cc:224] Handling call
0709 13:17:41.032950 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.033922 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.034213 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:40.983647 13136 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:41.011704 12812 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.033s	sys 0.039s
W20260709 13:17:41.044544 10123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.005s	user 0.009s	sys 0.018s
W20260709 13:17:41.020828 13142 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:17:41.050520 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.050684 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 146133us
W20260709 13:17:41.028744 12860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.076s	user 0.026s	sys 0.049s
W20260709 13:17:41.036785 13129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.034s	sys 0.063s
W20260709 13:17:41.123447 10123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.009s	sys 0.018s
W20260709 13:17:41.074375 13086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.001s	sys 0.022s
I20260709 13:17:41.140767 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:41.037578 13084 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
W20260709 13:17:41.072021 13020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.011s	sys 0.015s
I20260709 13:17:41.167795 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.145953 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1424) took 1833 ms. Trace:
W20260709 13:17:41.150588 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.007s	sys 0.009s
I20260709 13:17:41.141801 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.139533 12812 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.033s	sys 0.039s
I20260709 13:17:41.060875 13111 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:40.996598 13147 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:41.144465 12860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.026s	sys 0.049s
W20260709 13:17:41.161850 12998 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.241s	user 0.015s	sys 0.000s
I20260709 13:17:41.044651 13116 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.150578 13139 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:41.087013 13034 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.230685 10126 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:41.096567 13064 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.031808 13098 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.241782 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1425) took 1913 ms. Trace:
W20260709 13:17:41.144912 13129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.035s	sys 0.064s
W20260709 13:17:41.176996 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.027s
W20260709 13:17:41.179339 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.009s	sys 0.009s
I20260709 13:17:41.145124 13136 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:41.242832 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.009s	sys 0.027s
W20260709 13:17:41.243700 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.245626 13136 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:41.169003 13020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.012s	sys 0.016s
I20260709 13:17:41.230855 13084 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.247318 12686 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.247992 13084 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.248874 13085 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.249518 13085 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.250566 13084 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:41.204124 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.006s	sys 0.013s
W20260709 13:17:41.251972 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.006s	sys 0.014s
I20260709 13:17:41.252712 10201 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:41.169338 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.008s	sys 0.009s
I20260709 13:17:41.172701 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.138989 13106 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.255653 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:41.170621 10205 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:17:41.246899 10169 consensus_peers.cc:597] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cebbf225b866442ab895577b0496de1b. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:41.155381 13086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.002s	sys 0.023s
I20260709 13:17:41.256487 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.243065 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.009s	sys 0.009s
I20260709 13:17:41.180747 13073 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:41.262610 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:41.264850 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 249535us
W20260709 13:17:41.264670 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.268209 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.266218 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.270855 10047 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:41.271442 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1417) took 2645 ms. Trace:
I20260709 13:17:41.272518 10196 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:41.272356 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.272073 10122 rpcz_store.cc:276] 0709 13:17:38.626075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.405720 (+779645us) service_pool.cc:224] Handling call
0709 13:17:39.431021 (+ 25301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.442141 (+ 11120us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:39.442218 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.556965 (+114747us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.060300 (+503335us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:40.060361 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.060466 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.271019 (+1210553us) raft_consensus.cc:1685] finished
0709 13:17:41.271055 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.271318 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12677,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":424417,"threads_started":1,"wal-append.queue_time_us":1132348}
I20260709 13:17:41.276834 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.277634 10205 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:41.281185 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 108571us
W20260709 13:17:41.281895 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 467121us
W20260709 13:17:41.242934 13021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.009s	sys 0.019s
I20260709 13:17:41.258863 10129 rpcz_store.cc:276] 0709 13:17:39.328166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.421243 (+ 93077us) service_pool.cc:224] Handling call
0709 13:17:39.476200 (+ 54957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.476413 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:40.919529 (+1443116us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.241222 (+321693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.241301 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:41.241387 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.241481 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.241683 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40273,"prepare.queue_time_us":1481076,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":26125,"raft.queue_time_us":21691,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":5986,"thread_start_us":1673339,"threads_started":2}
W20260709 13:17:41.333236 10020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.298s	user 0.006s	sys 0.028s
I20260709 13:17:41.297385 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:41.334517 10020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.007s	sys 0.028s
I20260709 13:17:41.287344 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.335177 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.30029e+06us
I20260709 13:17:41.338215 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:41.339600 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:41.232569 13139 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.351464 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:41.360357 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63066.8us
I20260709 13:17:41.361706 10130 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:41.361637 10055 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:41.266427 13073 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:41.369037 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 239) took 1494 ms. Trace:
W20260709 13:17:41.283875 10049 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.006s	sys 0.016s
W20260709 13:17:41.371941 10049 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.007s	sys 0.016s
I20260709 13:17:41.355752 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.285920 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:41.387998 10196 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:41.389575 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.393468 10055 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:17:41.331466 13021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.010s	sys 0.019s
W20260709 13:17:41.349531 13031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.014s	sys 0.030s
I20260709 13:17:41.280311 13084 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:41.275183 13136 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.279477 13153 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:41.286602 13066 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:17:41.279718 13099 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:17:41.412269 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 126769us
I20260709 13:17:41.417124 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:41.417510 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:41.411273 12966 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.442101 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:41.287566 12963 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.424515 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45986 (request call id 1433) took 1952 ms. Trace:
W20260709 13:17:41.420655 13031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.330s	user 0.014s	sys 0.031s
I20260709 13:17:41.392241 13073 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:41.425706 10129 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:17:41.465337 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1451) took 1538 ms. Trace:
I20260709 13:17:41.270558 13111 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.419425 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.443725 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:41.451692 10130 rpcz_store.cc:276] 0709 13:17:39.422049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.915873 (+493824us) service_pool.cc:224] Handling call
0709 13:17:41.361553 (+1445680us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b4c004c1ac0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2762d0a20 00007fa2762e1a29 00007fa2762e213f 00007fa2762a62ab 00007fa2762d3f3c 00007fa2762d43fe 00007fa27656188b 00007fa277f60495 00007fa277fb130f 00007fa277ffba55 00007fa27cfcbcd0
0709 13:17:41.362690 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":290,"spinlock_wait_cycles":70029696}
I20260709 13:17:41.465844 10129 rpcz_store.cc:276] 0709 13:17:39.927396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.392908 (+1465512us) service_pool.cc:224] Handling call
0709 13:17:41.425371 (+ 32463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.464918 (+ 39547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.465234 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:41.466969 10898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.011s	sys 0.011s
I20260709 13:17:41.435897 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:41.475203 10898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.012s	sys 0.011s
I20260709 13:17:41.421311 13084 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.482635 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.488986 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 203162us
I20260709 13:17:41.409157 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 237) took 1534 ms. Trace:
I20260709 13:17:41.494309 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.480295 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.493954 10127 rpcz_store.cc:276] 0709 13:17:39.874377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.111836 (+1237459us) service_pool.cc:224] Handling call
0709 13:17:41.130029 (+ 18193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.130330 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:17:41.178571 (+ 48241us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.408383 (+229812us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.408601 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.408653 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:41.408731 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.408832 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.409022 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43629,"prepare.run_cpu_time_us":1597,"prepare.run_wall_time_us":13328,"raft.queue_time_us":1798,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":182,"thread_start_us":228704,"threads_started":1}
I20260709 13:17:41.495093 10196 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:41.442540 13153 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.502101 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1452) took 1574 ms. Trace:
I20260709 13:17:41.439617 13021 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:41.510383 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.452287 12860 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:41.519464 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.521970 10196 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 4.
I20260709 13:17:41.522330 10127 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:41.522992 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1455) took 1594 ms. Trace:
I20260709 13:17:41.523098 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.523401 10127 rpcz_store.cc:276] 0709 13:17:39.928722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.520660 (+1591938us) service_pool.cc:224] Handling call
0709 13:17:41.521953 (+  1293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.522718 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.522937 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:41.524327 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.510700 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.464030 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:41.455978 13085 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.527920 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.485944 12884 tablet_copy_client.cc:538] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:41.528079 10196 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:41.500432 13111 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:41.527491 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:41.534668 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 381) took 2699 ms. Trace:
W20260709 13:17:41.607247 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 79368.8us
I20260709 13:17:41.610051 13166 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:41.493358 12966 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.529680 10130 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:41.622066 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1456) took 1693 ms. Trace:
W20260709 13:17:41.534569 13154 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:17:41.493106 13036 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:41.489152 13084 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:41.615003 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87605.6us
I20260709 13:17:41.616451 10898 rpcz_store.cc:276] 0709 13:17:38.826793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.827360 (+   567us) service_pool.cc:224] Handling call
0709 13:17:38.855595 (+ 28235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:38.855875 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:39.543658 (+687783us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.487859 (+1944201us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.525662 (+ 37803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.525727 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:41.525785 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.525874 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.526036 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37502,"prepare.queue_time_us":652125,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":136,"raft.queue_time_us":74350,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":241,"spinlock_wait_cycles":10621312,"thread_start_us":2591219,"threads_started":3,"wal-append.queue_time_us":1600690}
I20260709 13:17:41.673151 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.751246 10196 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:41.486785 12963 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.125s
I20260709 13:17:41.534252 13087 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.193s
I20260709 13:17:41.511458 13153 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:41.679138 13164 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.536589 13116 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:41.683637 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 321867us
I20260709 13:17:41.671530 13166 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:41.725540 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.526085 13074 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.619232 13085 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:41.726913 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1479) took 1445 ms. Trace:
I20260709 13:17:41.520865 13099 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:41.622637 10130 rpcz_store.cc:276] 0709 13:17:39.929058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.528886 (+1599828us) service_pool.cc:224] Handling call
0709 13:17:41.529409 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.621714 (+ 92305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.621994 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:41.789309 10049 rpcz_store.cc:276] 0709 13:17:40.281420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.281558 (+   138us) service_pool.cc:224] Handling call
0709 13:17:40.281840 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:40.282069 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:41.372622 (+1090553us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.693867 (+321245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.726495 (+ 32628us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:41.726580 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.726684 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.726839 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72341,"prepare.queue_time_us":350907,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":44791,"spinlock_wait_cycles":36195712,"thread_start_us":1267729,"threads_started":3,"wal-append.queue_time_us":138356}
I20260709 13:17:41.791078 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:41.684190 12848 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:41.669770 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 189521us
I20260709 13:17:41.777256 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:41.740130 13151 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.493s	user 0.003s	sys 0.008s
I20260709 13:17:41.503151 10129 rpcz_store.cc:276] 0709 13:17:39.927738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.474755 (+1547017us) service_pool.cc:224] Handling call
0709 13:17:41.501264 (+ 26509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.501558 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.501599 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.501664 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:41.501713 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.501800 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.501997 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:41.726233 13147 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.740186 13136 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:41.742550 13129 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:41.772374 13161 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.442s	user 0.001s	sys 0.001s
I20260709 13:17:41.678704 12966 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.795653 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:41.801959 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 155135us
I20260709 13:17:41.748353 13036 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:41.786047 13156 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 13:17:41.788342 13034 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.817330 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:41.793473 13085 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.794976 12848 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.833665 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:41.834219 12966 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:41.834944 12966 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:41.829263 13141 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
I20260709 13:17:41.801780 13064 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:41.836251 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.836874 13166 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.836824 10019 leader_election.cc:400] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6.
I20260709 13:17:41.837857 10019 leader_election.cc:403] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:41.818360 13174 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
I20260709 13:17:41.839781 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:41.795382 13106 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:41.870208 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:41.842895 10205 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.000s
I20260709 13:17:41.845268 13064 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6.
W20260709 13:17:41.845701 13160 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.411s	user 0.001s	sys 0.001s
I20260709 13:17:41.846429 13021 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:41.849500 13175 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
W20260709 13:17:41.849807 13093 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.741s	user 0.015s	sys 0.000s
W20260709 13:17:41.850462 13163 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
W20260709 13:17:41.851343 13186 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:17:41.852245 13178 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:17:41.904405 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.910781 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 215) took 1925 ms. Trace:
I20260709 13:17:41.852228 10898 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:41.853888 10900 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:41.855240 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1471) took 2414 ms. Trace:
I20260709 13:17:41.805462 13098 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:41.876374 10205 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:41.911550 10894 rpcz_store.cc:276] 0709 13:17:39.985420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.985561 (+   141us) service_pool.cc:224] Handling call
0709 13:17:39.999595 (+ 14034us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:40.012186 (+ 12591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.012275 (+    89us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:40.012986 (+   711us) log.cc:844] Serialized 26 byte log entry
0709 13:17:41.061788 (+1048802us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:41.061858 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.061980 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.910523 (+848543us) raft_consensus.cc:1685] finished
0709 13:17:41.910547 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.910696 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12011,"raft.queue_time_us":15822,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":1048230,"threads_started":1,"wal-append.queue_time_us":1803180}
I20260709 13:17:41.892729 10196 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:17:41.825613 13036 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:41.840680 13162 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.415s	user 0.000s	sys 0.002s
I20260709 13:17:41.827610 13195 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:41.831024 13085 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:41.932216 13012 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:41.824431 13129 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.843557 12848 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.936556 13129 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:41.937779 12812 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:41.899660 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72214us
I20260709 13:17:41.903546 12966 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:41.912699 10050 rpcz_store.cc:276] 0709 13:17:39.441369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.441578 (+   209us) service_pool.cc:224] Handling call
0709 13:17:39.442009 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.780269 (+338260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.780302 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.780879 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.997798 (+1216919us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:40.997840 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.997913 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.854912 (+856999us) raft_consensus.cc:1685] finished
0709 13:17:41.854936 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.855162 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17077,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":111,"thread_start_us":1521281,"threads_started":2,"wal-append.queue_time_us":1495789}
I20260709 13:17:41.922676 13194 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:41.925889 13064 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.851007 13073 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.944532 13106 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:41.926828 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:41.946553 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.946798 13073 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:41.947682 13064 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:41.947675 13099 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:41.948390 10205 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:41.948520 13099 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:41.948866 12966 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:41.948549 13064 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:41.950851 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.949908 13099 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.951845 13194 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.951105 13064 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.953445 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:41.929678 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:41.955708 13194 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:41.957175 12812 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:41.957513 13194 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:41.958230 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:41.957476 13031 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.958951 13031 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.959473 13188 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.960039 13188 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.960718 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 9) took 3156 ms. Trace:
I20260709 13:17:41.961189 10200 rpcz_store.cc:276] 0709 13:17:38.804616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:38.805477 (+   861us) service_pool.cc:224] Handling call
0709 13:17:41.960519 (+3155042us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:41.960623 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4858976000}
I20260709 13:17:41.900429 13021 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.964030 10200 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:17:41.964494 13021 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:41.965813 13173 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:17:41.966089 10200 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:17:41.910912 13185 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:17:41.967278 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1428) took 2638 ms. Trace:
I20260709 13:17:41.947299 12963 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:17:41.967732 10125 rpcz_store.cc:276] 0709 13:17:39.329202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.781097 (+451895us) service_pool.cc:224] Handling call
0709 13:17:39.795840 (+ 14743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:40.822820 (+1026980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:40.822853 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:40.897124 (+ 74271us) log.cc:844] Serialized 26 byte log entry
0709 13:17:41.330259 (+433135us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:41.330304 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.330408 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.966948 (+636540us) raft_consensus.cc:1685] finished
0709 13:17:41.966972 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.967172 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29126,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"thread_start_us":1413807,"threads_started":2,"wal-append.queue_time_us":958859}
I20260709 13:17:41.968981 12963 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:41.969471 12963 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:41.971355 13099 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.972385 10200 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:17:41.973043 10200 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 5
I20260709 13:17:41.973318 10200 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:17:41.973601 10200 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:41.912145 13093 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.974212 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1457) took 2006 ms. Trace:
I20260709 13:17:41.974622 10127 rpcz_store.cc:276] 0709 13:17:39.968210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.524193 (+1555983us) service_pool.cc:224] Handling call
0709 13:17:41.524542 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:41.527203 (+  2661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.527326 (+   123us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:41.608098 (+ 80772us) log.cc:844] Serialized 26 byte log entry
0709 13:17:41.823044 (+214946us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:41.823109 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.823238 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:41.973945 (+150707us) raft_consensus.cc:1685] finished
0709 13:17:41.973965 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.974127 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13975,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":79540,"thread_start_us":96458,"threads_started":1,"wal-append.queue_time_us":297807}
I20260709 13:17:41.976398 13093 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:41.977237 10198 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:41.961609 13031 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:41.978418 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48840.5us
I20260709 13:17:41.950202 10205 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:41.985982 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:41.921689 13134 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.832s	user 0.011s	sys 0.022s
W20260709 13:17:41.992630 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49888.4us
I20260709 13:17:41.989671 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:41.970480 12963 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.996959 10196 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:41.997300 13194 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:41.998709 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:41.998823 13108 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:41.999953 13108 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:42.000535 13153 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.004010 10895 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:42.005380 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.006225 10203 raft_consensus.cc:2368] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
W20260709 13:17:42.006677 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1615) took 1666 ms (client timeout 2172 ms). Trace:
I20260709 13:17:42.007231 13201 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:42.007951 13201 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:42.008350 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.009441 10197 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:42.009979 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1616) took 1582 ms. Trace:
I20260709 13:17:42.010306 10197 rpcz_store.cc:276] 0709 13:17:40.428655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.428875 (+   220us) service_pool.cc:224] Handling call
0709 13:17:42.008164 (+1579289us) spinlock_profiling.cc:243] Waited 968 us on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:42.009904 (+  1740us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2226432}
I20260709 13:17:42.016192 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1498) took 1039 ms. Trace:
I20260709 13:17:42.016659 10047 rpcz_store.cc:276] 0709 13:17:40.976382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.023279 (+ 46897us) service_pool.cc:224] Handling call
0709 13:17:41.047938 (+ 24659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.271437 (+223499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.015870 (+744433us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001b8d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.016111 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512034048}
W20260709 13:17:41.991722 13134 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.902s	user 0.011s	sys 0.022s
I20260709 13:17:41.962975 13188 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:42.032388 13036 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.041428 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 235) took 2248 ms. Trace:
I20260709 13:17:42.043236 13136 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:42.007035 10203 rpcz_store.cc:269] 0709 13:17:40.340258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.340486 (+   228us) service_pool.cc:224] Handling call
0709 13:17:42.005233 (+1664747us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:42.006611 (+  1378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5102592}
I20260709 13:17:42.008363 13201 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:42.049034 13149 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.832s	user 0.000s	sys 0.004s
I20260709 13:17:42.015818 13085 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.071956 10894 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:42.163738 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:42.067238 10127 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:17:42.235846 10127 pending_rounds.cc:85] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:17:42.236361 10127 pending_rounds.cc:107] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:17:42.236773 10127 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:42.185889 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 200328us
I20260709 13:17:42.087512 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:41.988011 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.239388 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:42.239622 10095 connection.cc:256] RPC call timeout handler was delayed by -1.4543801129998428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:17:42.240540 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3450651209998341s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4194s for a response
I20260709 13:17:42.189780 10128 rpcz_store.cc:276] 0709 13:17:39.796036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.061254 (+1265218us) service_pool.cc:224] Handling call
0709 13:17:41.061739 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.240554 (+178815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.240660 (+   106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.240791 (+   131us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:41.240873 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.958441 (+717568us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:17:42.040980 (+ 82539us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54001428d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.041069 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.041323 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5774,"raft.queue_time_us":20538,"raft.run_cpu_time_us":2906,"raft.run_wall_time_us":698539,"spinlock_wait_cycles":403820288}
I20260709 13:17:42.152096 13194 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:42.009445 12848 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.077270 13084 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:42.196981 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.209542 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.191661 13136 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.197156 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1495) took 1222 ms. Trace:
I20260709 13:17:42.170753 10196 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:42.254220 10053 rpcz_store.cc:276] 0709 13:17:40.975098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.975331 (+   233us) service_pool.cc:224] Handling call
0709 13:17:41.020248 (+ 44917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.267501 (+247253us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:41.288230 (+ 20729us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.919559 (+631329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.919617 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:41.919672 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.196805 (+277133us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.196868 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.197064 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20908,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":91,"raft.queue_time_us":50613,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":20774,"spinlock_wait_cycles":567820160,"thread_start_us":630600,"threads_started":2,"wal-append.queue_time_us":480378}
I20260709 13:17:42.210901 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.186533 12904 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.221745 12883 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:42.224237 13179 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:42.205152 13018 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:42.225008 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2681) took 1122 ms. Trace:
I20260709 13:17:42.222710 13147 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:17:42.101112 12884 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
W20260709 13:17:42.227417 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1607) took 2174 ms (client timeout 1872 ms). Trace:
W20260709 13:17:42.228529 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.208256 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.245636 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:42.243145 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.287036 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.193s (SENT)
I20260709 13:17:42.242658 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:42.252655 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.260792 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:42.254528 13097 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.291322 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:41.369797 10122 rpcz_store.cc:276] 0709 13:17:39.875061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.287556 (+1412495us) service_pool.cc:224] Handling call
0709 13:17:41.288026 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.348506 (+ 60480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.348562 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.348643 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:41.368612 (+ 19969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.368728 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.368939 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":327,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":29608}
I20260709 13:17:42.306524 10128 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:42.258610 10203 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:17:42.285563 10202 rpcz_store.cc:269] 0709 13:17:40.053322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.054160 (+   838us) service_pool.cc:224] Handling call
0709 13:17:42.227087 (+2172927us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540012a054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:42.227295 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815819008}
I20260709 13:17:42.264000 10898 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:42.329021 10130 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:42.329813 10130 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:42.264709 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.332245 10050 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:42.266938 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.193928 13153 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:42.205637 13085 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:42.271642 10198 rpcz_store.cc:276] 0709 13:17:41.102301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.157807 (+ 55506us) service_pool.cc:224] Handling call
0709 13:17:41.158289 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.977664 (+819375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.224411 (+246747us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b540039d854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.224875 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882859264}
I20260709 13:17:42.272838 10047 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:42.198104 13036 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:42.203575 13139 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:42.242763 13111 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.278667 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:42.263561 13039 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.200s	user 0.005s	sys 0.025s
I20260709 13:17:42.289199 10048 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:42.262756 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:42.254513 13148 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:42.269771 12686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.018s	sys 0.030s
W20260709 13:17:42.310460 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 67913.2us
I20260709 13:17:42.268270 12883 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:42.312237 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.376926 10197 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.243356 13086 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:17:42.279531 12966 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:42.339671 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.343494 13116 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.343248 13073 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:42.345202 10049 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:17:42.349843 10055 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:42.391383 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.254992 13074 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:17:42.340269 10130 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.278810 13018 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:42.254537 13134 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:17:42.374891 13099 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:17:42.370036 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.278143 13123 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:17:42.372357 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.386438 13111 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.388827 13148 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:41.168764 10123 rpcz_store.cc:276] 0709 13:17:39.312877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.409151 (+ 96274us) service_pool.cc:224] Handling call
0709 13:17:39.442886 (+ 33735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.443247 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:40.038892 (+595645us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.145434 (+1106542us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.145534 (+   100us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:41.145616 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:41.145727 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:41.145878 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1378067,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":1421,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":12436,"thread_start_us":1627938,"threads_started":2}
I20260709 13:17:42.312501 13097 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:42.386473 12686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.023s	sys 0.030s
I20260709 13:17:42.299135 13196 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.375840 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.384873 10128 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:42.267874 13129 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.093s
W20260709 13:17:42.374066 13039 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.351s	user 0.005s	sys 0.029s
I20260709 13:17:42.283987 13170 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 13:17:42.341003 10051 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:42.394906 13203 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:42.363361 13087 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.396936 10052 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:42.474222 13087 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.475330 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:42.476615 13087 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:42.392292 10055 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:42.390004 10049 pending_rounds.cc:85] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 3
I20260709 13:17:42.399300 10053 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:42.400995 10198 raft_consensus.cc:2376] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8. Re-sending same reply.
I20260709 13:17:42.402385 13123 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:42.569169 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.484071 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1431) took 3154 ms. Trace:
W20260709 13:17:42.387990 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 125394us
I20260709 13:17:42.417397 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.425344 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2677) took 1403 ms. Trace:
I20260709 13:17:42.427261 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 548) took 1013 ms. Trace:
I20260709 13:17:42.427878 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 230) took 2693 ms. Trace:
I20260709 13:17:42.597019 10126 rpcz_store.cc:276] 0709 13:17:39.734769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.011830 (+1277061us) service_pool.cc:224] Handling call
0709 13:17:41.046398 (+ 34568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.231356 (+184958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.427564 (+1196208us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.427790 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394196864}
I20260709 13:17:42.434399 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.435891 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.598404 10196 raft_consensus.cc:2376] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
I20260709 13:17:42.598598 10127 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:42.588855 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.439621 10130 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.440385 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.601940 10050 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:42.444276 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.603004 10128 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:42.604022 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.604661 10128 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:42.605894 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:42.452497 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:42.456331 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 177786us
I20260709 13:17:42.457674 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.607008 10051 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.607514 10056 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:42.463258 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.465687 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.393496 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:42.610307 10048 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:42.438743 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:42.451901 10203 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:42.453979 10204 raft_consensus.cc:2410] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:17:42.455750 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.611696 10054 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:42.461495 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:42.500581 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.403671 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:42.613060 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 151722us
I20260709 13:17:42.587486 10049 pending_rounds.cc:107] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:17:42.463363 12848 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:42.467003 10899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 218) took 2442 ms (client timeout 2172 ms). Trace:
W20260709 13:17:42.592006 10205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.005s	sys 0.095s
I20260709 13:17:42.600304 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.437333 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:42.620055 10205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.006s	sys 0.095s
I20260709 13:17:42.620517 10047 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:42.612233 10202 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:42.613114 10125 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:42.623853 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.628484 10127 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 5: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:42.614024 10049 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:42.442394 13204 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:17:42.596107 10201 rpcz_store.cc:276] 0709 13:17:41.022274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.026968 (+  4694us) service_pool.cc:224] Handling call
0709 13:17:41.027501 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:41.253218 (+225717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.424901 (+1171683us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.425237 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517302144}
I20260709 13:17:42.596164 10895 rpcz_store.cc:276] 0709 13:17:41.414232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.428263 (+ 14031us) service_pool.cc:224] Handling call
0709 13:17:42.426942 (+998679us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b540053b454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:42.427169 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":914872576}
W20260709 13:17:42.617275 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2672) took 2014 ms (client timeout 1541 ms). Trace:
I20260709 13:17:42.592715 10124 rpcz_store.cc:276] 0709 13:17:39.330099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.914274 (+584175us) service_pool.cc:224] Handling call
0709 13:17:39.914591 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:39.956997 (+ 42406us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:40.920625 (+963628us) log.cc:844] Serialized 14 byte log entry
0709 13:17:41.219628 (+299003us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:41.220054 (+   426us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:41.220109 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.483640 (+1263531us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b54002d0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:42.483721 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.483966 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56291,"prepare.queue_time_us":1064011,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":2774,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":6393,"spinlock_wait_cycles":96839552,"thread_start_us":1216148,"threads_started":2,"wal-append.queue_time_us":1091152}
I20260709 13:17:42.586726 10053 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:42.641589 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:17:42.448956 13152 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.043s	user 0.012s	sys 0.000s
I20260709 13:17:42.632351 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:42.633188 10199 rpcz_store.cc:269] 0709 13:17:40.602494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.901662 (+299168us) service_pool.cc:224] Handling call
0709 13:17:42.465444 (+1563782us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:17:42.617185 (+151741us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3890432}
I20260709 13:17:42.413867 12963 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:42.655846 10201 raft_consensus.cc:2376] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6. Re-sending same reply.
W20260709 13:17:42.663218 10204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 236) took 1435 ms (client timeout 1526 ms). Trace:
I20260709 13:17:42.664103 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.742336 10196 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:42.471478 13129 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:42.425925 13097 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:42.415297 13138 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:42.440876 12998 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.465606 13194 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:42.734844 10202 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:42.804992 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.442543 13002 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:42.411285 13116 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.620946 13066 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.172s
I20260709 13:17:42.656091 10198 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:42.406606 13111 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:42.746726 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.744190 10204 rpcz_store.cc:269] 0709 13:17:41.227747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.267461 (+ 39714us) service_pool.cc:224] Handling call
0709 13:17:42.645563 (+1378102us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:42.663084 (+ 17521us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2571343232}
I20260709 13:17:42.665356 10049 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:17:42.748646 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.407502 13084 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:42.586071 13179 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:42.754284 10051 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:42.662709 13123 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 13:17:42.757884 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.435371 13148 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:42.619869 13188 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:42.663529 13064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.019s	sys 0.072s
W20260709 13:17:42.841348 13064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.020s	sys 0.072s
I20260709 13:17:42.812422 12998 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.807278 13002 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:42.843422 13002 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:42.845006 13002 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:42.631017 10899 rpcz_store.cc:269] 0709 13:17:40.024637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:40.025797 (+  1160us) service_pool.cc:224] Handling call
0709 13:17:42.466708 (+2440911us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b54004650d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:42.466886 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5365791616}
I20260709 13:17:42.639230 13170 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:42.796540 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:42.735491 10047 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:42.814929 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:42.429056 13018 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.630895 13139 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.179s
W20260709 13:17:42.819461 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 211798us
I20260709 13:17:42.855525 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.855865 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:42.857123 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.857239 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:42.857900 10204 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:42.858601 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:42.821868 10129 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.017s	sys 0.015s
I20260709 13:17:42.456936 12883 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.593699 13085 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:42.702217 12966 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 13:17:42.442337 13196 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.755638 13195 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:17:42.734848 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.849236 10126 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:42.853364 10128 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:42.764235 13108 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:17:42.858907 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:42.859364 13073 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:42.869375 10199 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:42.871341 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.003s	sys 0.032s
I20260709 13:17:42.871948 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1429) took 3542 ms. Trace:
I20260709 13:17:42.775296 13129 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.872687 10131 rpcz_store.cc:276] 0709 13:17:39.329495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:39.843461 (+513966us) service_pool.cc:224] Handling call
0709 13:17:39.844030 (+   569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:39.844441 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:39.844477 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:39.845029 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:17:40.921813 (+1076784us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:40.921876 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:40.921986 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:42.448164 (+1526178us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:42.863359 (+415195us) raft_consensus.cc:1685] finished
0709 13:17:42.863384 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:42.871834 (+  8450us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1846,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":21062400,"thread_start_us":1076065,"threads_started":1,"wal-append.queue_time_us":1411978}
I20260709 13:17:42.871001 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:42.872515 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.004s	sys 0.032s
I20260709 13:17:42.885586 10197 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:42.887017 10196 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:42.806710 13138 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:42.917508 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.852041 13188 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:42.929382 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:42.929688 10200 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:17:42.930217 10197 raft_consensus.cc:2376] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8. Re-sending same reply.
I20260709 13:17:42.851711 13203 raft_consensus.cc:2764] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:17:42.868690 13064 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:42.932284 13064 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:42.896196 10199 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:17:42.932916 13064 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:42.868122 13170 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:42.879236 13085 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:42.895941 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 532183us
I20260709 13:17:42.994999 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:42.866534 13176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.027s	sys 0.069s
I20260709 13:17:42.941880 10202 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:42.942288 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:42.847996 13179 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:42.997031 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:42.896004 10128 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:42.965544 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:42.949633 13202 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:17:42.974308 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.978695 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:42.956831 13201 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.006s
I20260709 13:17:43.012470 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1627) took 1027 ms. Trace:
W20260709 13:17:43.014874 10170 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.018s	sys 0.042s
I20260709 13:17:43.015575 10197 rpcz_store.cc:276] 0709 13:17:41.984608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.943001 (+958393us) service_pool.cc:224] Handling call
0709 13:17:43.012346 (+ 69345us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:43.016209 10170 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.019s	sys 0.042s
W20260709 13:17:43.012506 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:42.952116 13073 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:43.025311 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:43.027412 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 392812us
I20260709 13:17:43.034224 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:43.013545 13176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.028s	sys 0.071s
W20260709 13:17:43.019744 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77724.9us
W20260709 13:17:43.017086 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 672050us
W20260709 13:17:43.070290 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.092240 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54725.5us
I20260709 13:17:42.949781 13036 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:17:43.101003 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:43.102720 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:43.098886 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1625) took 1114 ms. Trace:
I20260709 13:17:43.103977 10199 rpcz_store.cc:276] 0709 13:17:41.984235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.870506 (+886271us) service_pool.cc:224] Handling call
0709 13:17:43.098801 (+228295us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:43.109414 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.110400 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:43.113643 10125 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:43.121011 10048 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:17:43.123646 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:43.126525 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:42.861131 13194 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:43.109992 13039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.022s	sys 0.046s
I20260709 13:17:43.094022 12686 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:43.033205 13087 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:43.099951 13139 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:43.023382 13201 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.145004 12884 tablet_bootstrap.cc:700] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent opening tablet: real 0.361s	user 0.003s	sys 0.000s
W20260709 13:17:43.177704 13039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.022s	sys 0.046s
I20260709 13:17:43.100093 13097 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:17:43.129305 13138 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.191576 12686 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.192004 13087 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:17:43.253371 13208 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 13:17:43.255633 13200 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.494s	user 0.006s	sys 0.005s
I20260709 13:17:43.256153 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 252) took 1777 ms. Trace:
I20260709 13:17:43.205358 13182 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:43.274340 10054 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 13:17:43.293522 10200 rpcz_store.cc:276] 0709 13:17:41.478269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.977651 (+499382us) service_pool.cc:224] Handling call
0709 13:17:42.646304 (+668653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.930296 (+283992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.254423 (+324127us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:43.255776 (+  1353us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54002de6d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:43.256033 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2186743040}
I20260709 13:17:43.296653 10054 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:43.200191 13139 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:43.331808 10122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.029s	sys 0.033s
I20260709 13:17:43.277550 13087 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.250174 12686 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:43.292550 13214 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:43.352203 13206 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.476s	user 0.001s	sys 0.001s
I20260709 13:17:43.359146 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 258) took 1095 ms. Trace:
I20260709 13:17:43.363505 10202 rpcz_store.cc:276] 0709 13:17:42.263403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.804587 (+541184us) service_pool.cc:224] Handling call
0709 13:17:43.358840 (+554253us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:43.359027 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312613760}
W20260709 13:17:43.336373 10122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.030s	sys 0.033s
I20260709 13:17:43.300904 13116 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.198570 13170 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.268950 13210 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
W20260709 13:17:43.392534 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.015s	sys 0.028s
I20260709 13:17:43.342041 13139 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:43.395434 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.015s	sys 0.028s
I20260709 13:17:43.373348 13129 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:43.425041 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.381405 13214 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:43.395259 13097 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:43.392215 13196 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:43.451447 10127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.006s	sys 0.021s
I20260709 13:17:43.395044 13116 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:43.458318 10127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.006s	sys 0.022s
I20260709 13:17:43.395533 13216 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:43.437491 13207 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:17:43.464305 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 242) took 1633 ms. Trace:
I20260709 13:17:43.466864 13194 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.472604 13194 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.473181 13194 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.473578 12966 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:43.453079 13218 ts_tablet_manager.cc:933] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.469144 10129 rpcz_store.cc:276] 0709 13:17:41.830843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.831062 (+   219us) service_pool.cc:224] Handling call
0709 13:17:41.831733 (+   671us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:41.832325 (+   592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.832372 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:41.833181 (+   809us) log.cc:844] Serialized 78 byte log entry
0709 13:17:42.280543 (+447362us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:42.861024 (+580481us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.911051 (+ 50027us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.948791 (+ 37740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.948972 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.254244 (+305272us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:43.464001 (+209757us) raft_consensus.cc:1685] finished
0709 13:17:43.464036 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.464220 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75998,"prepare.queue_time_us":1413965,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":139,"raft.queue_time_us":76250,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":27614,"spinlock_wait_cycles":40601472,"thread_start_us":1009865,"threads_started":2,"wal-append.queue_time_us":758170}
I20260709 13:17:43.476078 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:43.464318 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 605736us
I20260709 13:17:43.451642 13179 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:43.447737 13085 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:43.474337 12966 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.500145 13201 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:43.501104 13201 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:43.501791 13201 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.504835 13148 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:43.505435 13148 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:43.507706 13221 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 13:17:43.509543 13215 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:17:43.511502 13136 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.235s	user 0.007s	sys 0.009s
I20260709 13:17:43.512032 13136 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.512928 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1473) took 1155 ms. Trace:
I20260709 13:17:43.513589 10128 rpcz_store.cc:276] 0709 13:17:42.363553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.605531 (+241978us) service_pool.cc:224] Handling call
0709 13:17:43.512604 (+907073us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b540023afd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:43.512820 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567234432}
I20260709 13:17:43.515597 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 253) took 1074 ms. Trace:
I20260709 13:17:43.516444 13188 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:43.516269 10056 rpcz_store.cc:276] 0709 13:17:42.440720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.815790 (+375070us) service_pool.cc:224] Handling call
0709 13:17:42.816336 (+   546us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:42.816882 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.816927 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:42.833033 (+ 16106us) log.cc:844] Serialized 52 byte log entry
0709 13:17:43.143803 (+310770us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:43.143874 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.143972 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.515210 (+371238us) raft_consensus.cc:1685] finished
0709 13:17:43.515243 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.515506 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2042,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":1997,"thread_start_us":296761,"threads_started":1,"wal-append.queue_time_us":364482}
W20260709 13:17:43.522246 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.515625 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 253) took 1574 ms. Trace:
I20260709 13:17:43.527980 13214 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.528558 13123 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:43.529280 13123 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:43.482378 13129 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.534756 12883 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:43.535698 13129 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.525264 10205 rpcz_store.cc:276] 0709 13:17:41.940926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:41.980449 (+ 39523us) service_pool.cc:224] Handling call
0709 13:17:41.980776 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:41.981207 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:41.981227 (+    20us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:42.056230 (+ 75003us) log.cc:844] Serialized 52 byte log entry
0709 13:17:42.620750 (+564520us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:42.702950 (+ 82200us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.734861 (+ 31911us) log.cc:844] Serialized 14 byte log entry
0709 13:17:42.735183 (+   322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:42.735314 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.508309 (+772995us) spinlock_profiling.cc:243] Waited 8.56 ms on lock 0x7b18002ac400. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:43.515326 (+  7017us) raft_consensus.cc:1685] finished
0709 13:17:43.515356 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.515537 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7723,"prepare.queue_time_us":199812,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":11219,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":39848,"spinlock_wait_cycles":19676928,"thread_start_us":563917,"threads_started":1,"wal-append.queue_time_us":1135793}
W20260709 13:17:43.544224 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.498875 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:43.485643 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:43.500231 13002 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.655s	user 0.014s	sys 0.003s
W20260709 13:17:43.556409 10124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.010s	sys 0.024s
W20260709 13:17:43.559338 10124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.010s	sys 0.024s
W20260709 13:17:43.559518 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60808.2us
I20260709 13:17:43.514261 13136 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.561503 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:43.558627 13066 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:43.530616 13123 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.571877 10204 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:43.506605 13148 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.573963 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:43.538723 13129 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:43.575248 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:43.567291 13122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.010s	sys 0.046s
I20260709 13:17:43.554577 13225 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:43.557025 13195 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.019s	sys 0.017s
I20260709 13:17:43.598518 13036 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:43.603274 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.570569 13203 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:43.626806 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1624) took 1642 ms. Trace:
W20260709 13:17:43.631547 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56400.3us
I20260709 13:17:43.653192 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:43.608371 13122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.011s	sys 0.047s
I20260709 13:17:43.642398 10128 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:17:43.641193 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.017s
I20260709 13:17:43.655867 10196 rpcz_store.cc:276] 0709 13:17:41.984050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.961302 (+977252us) service_pool.cc:224] Handling call
0709 13:17:43.098696 (+137394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.626237 (+527541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.626283 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:43.626344 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:43.626393 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.626491 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.626710 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33230,"raft.queue_time_us":375733,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"thread_start_us":486755,"threads_started":1}
I20260709 13:17:43.661152 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:43.657845 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.021s
W20260709 13:17:43.661749 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87914.9us
W20260709 13:17:43.662251 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.001s	sys 0.021s
W20260709 13:17:43.662688 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 553425us
I20260709 13:17:43.662819 10019 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:17:43.663280 10019 leader_election.cc:403] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:17:43.604302 13194 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:43.708523 13201 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:43.709642 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:43.673763 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:43.640858 13195 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.020s	sys 0.017s
I20260709 13:17:43.652184 13036 consensus_queue.cc:1243] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:43.711036 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 248) took 1250 ms. Trace:
I20260709 13:17:43.715691 10122 rpcz_store.cc:276] 0709 13:17:42.460484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.813606 (+353122us) service_pool.cc:224] Handling call
0709 13:17:42.814276 (+   670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:42.814663 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.814702 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:42.821134 (+  6432us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.367389 (+546255us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:43.367454 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.367557 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:43.710670 (+343113us) raft_consensus.cc:1685] finished
0709 13:17:43.710704 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:43.710931 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":544616,"threads_started":1,"wal-append.queue_time_us":780724}
I20260709 13:17:43.677244 12966 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:43.683025 13064 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.000s
I20260709 13:17:43.683641 13223 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:43.683893 12963 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.934s	user 0.003s	sys 0.011s
I20260709 13:17:43.640429 13148 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.653854 13123 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.689248 13002 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.692646 10128 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:17:43.652623 13136 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:43.709841 13111 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:43.675032 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:43.661000 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.011s	sys 0.017s
W20260709 13:17:43.740756 10199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.014s	sys 0.014s
W20260709 13:17:43.767354 10199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.014s	sys 0.015s
I20260709 13:17:43.651998 13222 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:43.572146 13066 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.704434 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:43.764228 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:43.768668 10201 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:17:43.714995 10020 leader_election.cc:343] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Tablet error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:43.711872 13195 consensus_queue.cc:1243] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:43.771651 13002 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.762235 13064 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.779994 13020 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:43.767088 13201 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:17:43.786470 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:43.783845 13020 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.788756 13020 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.788945 12848 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:43.789469 12848 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.789860 12848 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:43.790273 12963 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:43.772933 13223 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:43.740439 13203 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:43.771365 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 644903us
W20260709 13:17:43.798786 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136091us
W20260709 13:17:43.781016 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 66091.9us
I20260709 13:17:43.768256 13111 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:43.779556 13222 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.817989 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:43.823513 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:43.815390 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:43.784288 13195 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:43.831914 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.835517 13139 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.838279 13139 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:43.844290 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:43.817346 13036 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:43.892971 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:43.893996 10122 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
W20260709 13:17:43.895426 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.844666 13129 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:43.864436 13172 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:43.864513 13196 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:17:43.842152 13020 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.864589 13085 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:17:43.863688 13139 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:43.903466 13139 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:43.903981 13139 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.879624 13066 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:43.883598 13203 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.799391 13176 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:17:43.884668 10196 pending_rounds.cc:85] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 7
I20260709 13:17:43.910005 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:43.885011 13018 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.887547 13225 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:43.888307 12963 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:43.889380 13223 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:43.890506 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:43.867813 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.820993 13222 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:43.844126 12883 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:43.869320 13201 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:43.802794 12686 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.905443 13172 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.846045 13195 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.910167 10196 pending_rounds.cc:107] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:17:43.939644 10196 raft_consensus.cc:2889] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:43.940378 10125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.030s	sys 0.011s
I20260709 13:17:43.799091 13122 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:17:43.940809 10125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.030s	sys 0.011s
I20260709 13:17:43.868175 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:43.899750 13036 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:43.874962 10053 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.922024 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:43.919322 13225 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.926446 13020 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:43.927242 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2693) took 1672 ms. Trace:
I20260709 13:17:43.871742 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:43.961432 10204 rpcz_store.cc:276] 0709 13:17:42.255189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.858623 (+603434us) service_pool.cc:224] Handling call
0709 13:17:43.920002 (+1061379us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:43.927166 (+  7164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1615043840}
W20260709 13:17:43.963361 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 91747.4us
I20260709 13:17:43.964766 10095 leader_election.cc:400] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
I20260709 13:17:43.965216 10095 leader_election.cc:403] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:17:43.860694 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:43.928757 12883 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:43.967015 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:43.973763 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:43.971414 10198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.015s	sys 0.044s
W20260709 13:17:43.974711 10198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.015s	sys 0.044s
I20260709 13:17:43.975054 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:43.977865 10131 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.022s	sys 0.013s
W20260709 13:17:43.982600 10123 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.020s	sys 0.020s
W20260709 13:17:43.946475 13153 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.007s	sys 0.029s
W20260709 13:17:43.944772 13039 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.013s	sys 0.022s
I20260709 13:17:43.985903 10053 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:17:43.979043 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 110976us
W20260709 13:17:43.987957 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.015s	sys 0.035s
I20260709 13:17:43.940666 13172 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:43.993192 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:43.993527 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:43.993225 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:43.995254 10048 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:43.996356 10055 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:17:43.996932 10047 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:43.983234 10131 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.022s	sys 0.013s
I20260709 13:17:43.998172 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:44.033661 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.015s	sys 0.035s
W20260709 13:17:44.060906 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.03565e+06us
I20260709 13:17:43.988091 13216 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:44.003648 10126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.013s	sys 0.031s
W20260709 13:17:44.062855 10126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.014s	sys 0.031s
W20260709 13:17:44.007858 10127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.038s
W20260709 13:17:44.063657 10127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.002s	sys 0.038s
W20260709 13:17:44.009402 10203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.020s	sys 0.024s
W20260709 13:17:44.064483 10203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.020s	sys 0.024s
W20260709 13:17:44.011001 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89099.7us
I20260709 13:17:44.012445 13020 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:44.012786 13179 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:44.013181 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 250) took 1377 ms. Trace:
I20260709 13:17:44.069947 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:44.077497 10124 rpcz_store.cc:276] 0709 13:17:42.635565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.850241 (+214676us) service_pool.cc:224] Handling call
0709 13:17:42.874683 (+ 24442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:42.875075 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:42.875102 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:42.912120 (+ 37018us) log.cc:844] Serialized 26 byte log entry
0709 13:17:43.579015 (+666895us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:43.579078 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:43.579252 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.012915 (+433663us) raft_consensus.cc:1685] finished
0709 13:17:44.012938 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.013116 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":607827,"threads_started":1,"wal-append.queue_time_us":961147}
I20260709 13:17:43.978237 12883 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:43.980803 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:44.014569 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:44.014797 13229 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:43.988531 13087 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.625s	user 0.017s	sys 0.005s
I20260709 13:17:43.982029 13116 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:44.016372 10202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.009s	sys 0.036s
W20260709 13:17:44.025506 10200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.026s	sys 0.039s
W20260709 13:17:44.089394 10200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.026s	sys 0.039s
W20260709 13:17:44.089470 10202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.009s	sys 0.036s
I20260709 13:17:43.940845 13201 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:43.987799 13224 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:44.044427 10050 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:44.064002 13064 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.047612 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.117292 10048 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:44.117615 10096 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.008s	sys 0.038s
I20260709 13:17:44.048834 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:44.118191 10096 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.008s	sys 0.038s
I20260709 13:17:44.118747 10055 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:44.049978 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.119786 10047 raft_consensus.cc:2410] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:44.120955 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.076948 13179 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.079262 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:44.031989 13206 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.011s	sys 0.034s
W20260709 13:17:44.061559 13153 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.007s	sys 0.030s
I20260709 13:17:43.981045 13210 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:44.083278 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 102658us
I20260709 13:17:43.950207 13111 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:44.027149 10123 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.020s	sys 0.024s
W20260709 13:17:44.059337 13039 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.013s	sys 0.022s
W20260709 13:17:44.080622 13170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.006s	sys 0.037s
I20260709 13:17:44.045287 10054 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:44.104228 13201 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.089177 13116 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.104225 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.126360 13224 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:44.094475 13020 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:44.115005 10053 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:44.122409 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.186765 10047 raft_consensus.cc:2376] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6. Re-sending same reply.
I20260709 13:17:44.051689 10051 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:44.195652 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1507) took 1855 ms. Trace:
I20260709 13:17:44.196687 10053 rpcz_store.cc:276] 0709 13:17:42.340630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.343758 (+  3128us) service_pool.cc:224] Handling call
0709 13:17:42.374382 (+ 30624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.011765 (+637383us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b4c00429340. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2762d0a20 00007fa2762e1a29 00007fa2762a7644 00007fa2762cef2d 00007fa2762ceb93 00007fa277fff18b 00007fa277fffcb5 00007fa277ffaf30 00007fa277ffc7de 00007fa278006b49 00007fa278007416
0709 13:17:43.874484 (+862719us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540009fc20. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f8206f 00007fa277ffaf60 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62
0709 13:17:44.191467 (+316983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.195514 (+  4047us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1829764608}
W20260709 13:17:44.129618 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.020s	sys 0.033s
W20260709 13:17:44.198323 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.020s	sys 0.033s
I20260709 13:17:44.132531 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:44.135766 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56666.6us
I20260709 13:17:44.131917 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.199736 10055 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:44.200822 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.201508 10055 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:44.146948 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:44.153028 10020 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 2.155s (SENT)
I20260709 13:17:44.202699 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:44.203706 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50755.1us
I20260709 13:17:44.029587 13235 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:44.120752 12963 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:44.137554 13206 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.013s	sys 0.034s
I20260709 13:17:44.164032 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.217975 10054 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:44.134790 13179 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.169123 10048 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:44.125941 13194 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:44.127329 13226 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:44.179639 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.105268 13230 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.095671 13087 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:44.125377 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.03458e+06us
I20260709 13:17:44.161263 13231 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 13:17:44.162336 13170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.006s	sys 0.038s
I20260709 13:17:44.146229 13234 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:44.149552 13064 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:44.211783 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.212293 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.206897 13235 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:44.210600 13218 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.009s	sys 0.013s
I20260709 13:17:44.223718 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.274406 10047 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:44.287075 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.301590 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:44.224352 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 77487.2us
I20260709 13:17:44.302165 10049 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:44.302856 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:44.303259 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.223721 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1459) took 1896 ms. Trace:
I20260709 13:17:44.101238 13229 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:44.180588 13073 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:44.229557 10050 raft_consensus.cc:2376] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7. Re-sending same reply.
I20260709 13:17:44.229715 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:44.304544 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:44.183341 13116 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:44.230530 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.320169 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:44.243084 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.246941 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:44.251955 13084 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:17:44.263932 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.325562 10900 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:44.326148 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.326850 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.228466 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.210919 13136 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:44.278817 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:44.280974 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2691) took 2035 ms (client timeout 2155 ms). Trace:
I20260709 13:17:44.205302 10051 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:44.228003 12998 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:44.304556 10130 rpcz_store.cc:276] 0709 13:17:42.327361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.328524 (+  1163us) service_pool.cc:224] Handling call
0709 13:17:42.328876 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.142322 (+813446us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:43.566031 (+423709us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.223319 (+657288us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.223389 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:44.223440 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.223514 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.223651 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95794,"prepare.queue_time_us":239378,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":98042,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":49384,"thread_start_us":1191421,"threads_started":3}
I20260709 13:17:44.302170 10048 raft_consensus.cc:2376] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6. Re-sending same reply.
I20260709 13:17:44.356078 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.260102 13216 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:44.305871 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.361758 10050 raft_consensus.cc:2376] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7. Re-sending same reply.
I20260709 13:17:44.309973 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.310918 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1474) took 1920 ms. Trace:
W20260709 13:17:44.316504 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49584us
I20260709 13:17:44.304597 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.321496 10055 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:44.367343 10049 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:44.368961 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:44.369830 10049 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:44.366957 10047 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:44.330667 10054 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:44.339004 10197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.020s	sys 0.031s
W20260709 13:17:44.374019 10197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.021s	sys 0.036s
I20260709 13:17:44.374686 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.375772 10054 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:44.377133 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.380407 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:44.351073 10201 rpcz_store.cc:269] 0709 13:17:42.245039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.757387 (+512348us) service_pool.cc:224] Handling call
0709 13:17:44.280670 (+1523283us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b5400228b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:44.280865 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148958464}
I20260709 13:17:44.351919 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.382457 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.392057 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.389750 10051 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 5.
I20260709 13:17:44.394157 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.386473 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.396742 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.397550 10050 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:44.399273 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.279342 13002 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:44.266490 13252 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.276917 12686 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:44.300459 13218 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.009s	sys 0.013s
I20260709 13:17:44.366586 10127 rpcz_store.cc:276] 0709 13:17:42.390743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.659416 (+268673us) service_pool.cc:224] Handling call
0709 13:17:42.659919 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.660216 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:43.458985 (+798769us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.234504 (+775519us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b1800279d00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:17:44.278250 (+ 43746us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.287958 (+  9708us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.288272 (+   314us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.288319 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:44.288373 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.288450 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.310819 (+ 22369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54383,"prepare.queue_time_us":721299,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":156,"raft.queue_time_us":10850,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":52151,"spinlock_wait_cycles":77332992,"thread_start_us":1303177,"threads_started":2}
I20260709 13:17:44.298503 13138 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.420495 10051 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 4.
I20260709 13:17:44.305940 13073 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:44.421944 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.429616 10051 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:44.419936 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.430377 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:44.324316 13036 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.418229 10053 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:44.264603 13226 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.226305 12963 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.265045 13194 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.353888 13201 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:44.448791 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.458565 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 249) took 1997 ms. Trace:
I20260709 13:17:44.435590 13002 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:44.407752 13195 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.463596 10052 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:44.413722 12998 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:44.465327 10049 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:44.521529 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.522526 10049 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:44.284677 13064 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:44.456738 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 364274us
I20260709 13:17:44.525192 10048 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:17:44.467975 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:44.267490 13230 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:44.469746 10123 rpcz_store.cc:276] 0709 13:17:42.460882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.850227 (+389345us) service_pool.cc:224] Handling call
0709 13:17:42.873854 (+ 23627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:42.940389 (+ 66535us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:44.150228 (+1209839us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.458045 (+307817us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:44.458124 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:44.458192 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.458283 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.458479 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61003,"prepare.queue_time_us":366677,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":2686,"raft.queue_time_us":75115,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":148,"thread_start_us":1427425,"threads_started":2}
W20260709 13:17:44.509107 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.532811 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 230034us
I20260709 13:17:44.462718 13129 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6.
W20260709 13:17:44.469332 13232 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.389s	user 0.000s	sys 0.001s
I20260709 13:17:44.404426 13020 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.515472 13225 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.305789 13172 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.530369 10204 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:44.472498 13218 tablet_copy_client.cc:323] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
W20260709 13:17:44.554136 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.552744 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 223423us
I20260709 13:17:44.421048 13252 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.432286 13138 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:44.578418 13214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.006s	sys 0.002s
I20260709 13:17:44.579344 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:44.580169 10130 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:44.458973 13224 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.460819 13216 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:17:44.586941 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:44.546857 13084 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:44.595057 10204 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:44.595212 13084 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:44.510515 13201 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:44.579035 13214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.007s	sys 0.002s
I20260709 13:17:44.596897 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:44.606763 10204 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:44.610723 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:44.614619 13258 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:17:44.616161 13253 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:17:44.617036 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 261) took 2307 ms. Trace:
W20260709 13:17:44.617924 13259 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:17:44.617595 10199 rpcz_store.cc:276] 0709 13:17:42.309578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.104720 (+795142us) service_pool.cc:224] Handling call
0709 13:17:43.105088 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.768838 (+663750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.768891 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.769745 (+   854us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.163170 (+393425us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:44.425185 (+262015us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.425781 (+   596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.425930 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.616739 (+190809us) raft_consensus.cc:1685] finished
0709 13:17:44.616768 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.616953 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38894,"raft.queue_time_us":346995,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":25688,"thread_start_us":1235093,"threads_started":3,"wal-append.queue_time_us":585325}
I20260709 13:17:44.628577 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:44.638769 13229 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:44.634961 10199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e14489a089594151b58748865ea36e81"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:17:44.619284 13249 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:17:44.619143 12963 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:44.472365 13235 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.668040 13235 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:44.669579 13252 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:44.620095 13188 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:44.672149 13222 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.005s
W20260709 13:17:44.672829 13260 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:17:44.666841 13172 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:44.540577 13073 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.633155 13216 consensus_queue.cc:1243] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:17:44.645159 12848 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.013s
I20260709 13:17:44.678071 13087 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:44.713059 13226 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.714013 13201 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:44.716256 13084 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:44.687062 13138 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:44.717875 13256 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.118s	user 0.000s	sys 0.002s
I20260709 13:17:44.749161 13194 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:44.752736 13182 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.455s	user 0.015s	sys 0.003s
I20260709 13:17:44.753589 13182 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:44.755915 13262 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:17:44.758510 13261 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 13:17:44.771397 13210 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.607s	user 0.015s	sys 0.000s
I20260709 13:17:44.772162 13210 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:44.776465 13230 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:44.672832 13222 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.529160 10049 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:17:44.836174 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:44.838829 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.839967 10049 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:17:44.517757 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.473198 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.842299 10056 raft_consensus.cc:2368] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9.
W20260709 13:17:44.840574 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:44.847410 10051 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:44.860455 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 322716us
W20260709 13:17:44.733515 13123 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.965s	user 0.018s	sys 0.000s
I20260709 13:17:44.924578 10129 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:44.930456 13018 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.006s	user 0.009s	sys 0.008s
I20260709 13:17:44.931798 10201 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:44.937176 13064 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:44.943050 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.965271 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 263) took 1856 ms. Trace:
I20260709 13:17:44.949131 10205 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:17:44.947742 12848 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:44.951828 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:44.952072 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:44.953804 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 264) took 1844 ms. Trace:
W20260709 13:17:44.954232 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.000s	sys 0.006s
I20260709 13:17:44.702636 13073 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:44.968464 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2695) took 2712 ms. Trace:
I20260709 13:17:44.948639 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:44.986418 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.001s	sys 0.006s
I20260709 13:17:44.994729 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2701) took 1103 ms. Trace:
I20260709 13:17:44.994185 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2707) took 1076 ms. Trace:
I20260709 13:17:45.000913 10205 rpcz_store.cc:276] 0709 13:17:43.881531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.928094 (+1046563us) service_pool.cc:224] Handling call
0709 13:17:44.948270 (+ 20176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.978359 (+ 30089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.984932 (+  6573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.009130 10201 rpcz_store.cc:276] 0709 13:17:43.907576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.929943 (+1022367us) service_pool.cc:224] Handling call
0709 13:17:44.930963 (+  1020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.974813 (+ 43850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.984274 (+  9461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.023726 10055 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:17:45.027271 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.000s	sys 0.014s
W20260709 13:17:45.027841 10022 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.010s	sys 0.002s
I20260709 13:17:45.028085 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 266) took 1003 ms. Trace:
W20260709 13:17:45.028553 10022 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.010s	sys 0.003s
I20260709 13:17:45.026980 13226 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.035311 10122 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:45.028764 10196 rpcz_store.cc:276] 0709 13:17:43.880629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.883923 (+  3294us) service_pool.cc:224] Handling call
0709 13:17:43.884402 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:43.946837 (+ 62435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.946892 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:43.947835 (+   943us) log.cc:844] Serialized 52 byte log entry
0709 13:17:44.427394 (+479559us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:44.427469 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.427599 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.875497 (+447898us) raft_consensus.cc:1685] finished
0709 13:17:44.875580 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.884065 (+  8485us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3505,"raft.queue_time_us":42556,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":17933,"thread_start_us":478990,"threads_started":1,"wal-append.queue_time_us":561945}
W20260709 13:17:45.027938 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.015s
I20260709 13:17:45.055864 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:45.056236 10201 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:45.057116 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:45.059573 13226 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:45.073487 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1479) took 1825 ms. Trace:
I20260709 13:17:44.978689 10197 rpcz_store.cc:276] 0709 13:17:42.257460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.016960 (+759500us) service_pool.cc:224] Handling call
0709 13:17:43.133793 (+116833us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:43.625239 (+491446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.625293 (+    54us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:43.637968 (+ 12675us) log.cc:844] Serialized 78 byte log entry
0709 13:17:44.374590 (+736622us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:44.611935 (+237345us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.612802 (+   867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.612927 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.716817 (+103890us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:44.742138 (+ 25321us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b1800105040. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:44.967795 (+225657us) raft_consensus.cc:1685] finished
0709 13:17:44.967841 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.968254 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":290455,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":331088,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":8488,"spinlock_wait_cycles":75611776,"thread_start_us":1401579,"threads_started":3,"wal-append.queue_time_us":898462}
I20260709 13:17:45.075270 13226 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:45.074226 10126 rpcz_store.cc:276] 0709 13:17:43.227582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.227733 (+   151us) service_pool.cc:224] Handling call
0709 13:17:43.254186 (+ 26453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:44.063484 (+809298us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:17:44.090013 (+ 26529us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.412392 (+322379us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:44.426562 (+ 14170us) log.cc:844] Serialized 52 byte log entry
0709 13:17:44.426893 (+   331us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:44.427377 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.000431 (+573054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.022550 (+ 22119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.045007 (+ 22457us) raft_consensus.cc:1685] finished
0709 13:17:45.045035 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.052365 (+  7330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20995,"prepare.queue_time_us":183484,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":123,"raft.queue_time_us":540144,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":4417,"spinlock_wait_cycles":1761664,"thread_start_us":1592662,"threads_started":3,"wal-append.queue_time_us":443599}
W20260709 13:17:45.077064 13139 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.172s	user 0.010s	sys 0.013s
I20260709 13:17:45.081919 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:44.975198 10198 rpcz_store.cc:276] 0709 13:17:43.108901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.136063 (+ 27162us) service_pool.cc:224] Handling call
0709 13:17:43.142244 (+  6181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.979528 (+837284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:43.979565 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.997812 (+ 18247us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.324588 (+326776us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:44.324641 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.324728 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.964514 (+639786us) raft_consensus.cc:1685] finished
0709 13:17:44.964577 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.964949 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":341042,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":10648,"spinlock_wait_cycles":848128,"thread_start_us":1047897,"threads_started":2,"wal-append.queue_time_us":652773}
I20260709 13:17:45.083818 13020 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:45.082965 13236 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:45.035856 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 826240us
I20260709 13:17:45.087931 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet cebbf225b866442ab895577b0496de1b from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:17:45.088428 13222 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:45.089555 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.026815 13196 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.090553 10130 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:45.035323 13203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.028s	sys 0.066s
I20260709 13:17:45.095266 10047 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:45.098835 12848 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:45.095244 13020 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:44.977274 10200 rpcz_store.cc:276] 0709 13:17:43.109171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.315848 (+206677us) service_pool.cc:224] Handling call
0709 13:17:43.316391 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:43.316863 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:43.316903 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:43.341062 (+ 24159us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.089898 (+748836us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:44.089964 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.090056 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.952990 (+862934us) raft_consensus.cc:1685] finished
0709 13:17:44.953050 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:44.953516 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187561,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":691695,"threads_started":1,"wal-append.queue_time_us":1287665}
I20260709 13:17:45.098836 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:45.121433 13203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.028s	sys 0.067s
I20260709 13:17:45.111346 13139 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.144980 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.157591 10205 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
W20260709 13:17:45.158223 13148 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.393s	user 0.017s	sys 0.006s
I20260709 13:17:45.159263 10199 raft_consensus.cc:606] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:45.160142 10131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.027s	sys 0.024s
W20260709 13:17:45.160609 10131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.027s	sys 0.024s
I20260709 13:17:45.161765 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1478) took 1968 ms. Trace:
I20260709 13:17:45.113430 13236 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.162166 10131 rpcz_store.cc:276] 0709 13:17:43.193866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.194451 (+   585us) service_pool.cc:224] Handling call
0709 13:17:43.225712 (+ 31261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:43.226026 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:44.050399 (+824373us) log.cc:844] Serialized 14 byte log entry
0709 13:17:44.757493 (+707094us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:45.161143 (+403650us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.161253 (+   110us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.161338 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.161460 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.161700 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27000,"prepare.queue_time_us":301125,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":189568,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":58,"spinlock_wait_cycles":6563200,"thread_start_us":1854074,"threads_started":3,"wal-append.queue_time_us":499109}
I20260709 13:17:45.131584 13020 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:45.178086 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56396.3us
W20260709 13:17:45.188637 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 107080us
I20260709 13:17:45.191152 10129 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:45.188731 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.194387 10128 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:45.196091 10047 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:45.197175 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1543) took 1082 ms. Trace:
I20260709 13:17:45.197638 10047 rpcz_store.cc:276] 0709 13:17:44.114521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.195093 (+1080572us) service_pool.cc:224] Handling call
0709 13:17:45.195710 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.196754 (+  1044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.197081 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.200019 13214 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:45.200731 13214 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:45.200809 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49602.7us
I20260709 13:17:45.201341 13214 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:45.210402 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.018s	sys 0.007s
W20260709 13:17:45.211287 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.020s	sys 0.007s
W20260709 13:17:45.215406 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.215184 12884 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:45.229826 10130 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:45.230767 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:45.213517 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.500s (SENT)
I20260709 13:17:45.185017 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet cebbf225b866442ab895577b0496de1b from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-cebbf225b866442ab895577b0496de1b
W20260709 13:17:45.245134 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.249428 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 259) took 1151 ms. Trace:
I20260709 13:17:45.136837 13196 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:45.256898 13179 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.991s	user 0.008s	sys 0.014s
I20260709 13:17:45.257577 13179 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:45.262801 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 164145us
I20260709 13:17:45.246803 13236 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.269156 13245 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:45.271857 10204 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:45.272861 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59733.3us
W20260709 13:17:45.274032 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.505s (SENT)
I20260709 13:17:45.274650 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:45.276393 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.235877 13234 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.265791 13236 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:45.277004 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2700) took 1393 ms. Trace:
I20260709 13:17:45.296113 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.297531 10125 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:45.254920 13139 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:45.305459 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.287652 10204 rpcz_store.cc:276] 0709 13:17:43.881273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.650192 (+768919us) service_pool.cc:224] Handling call
0709 13:17:44.650976 (+   784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.272749 (+621773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.274736 (+  1987us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001fed54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:45.275049 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426642304}
W20260709 13:17:45.309674 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106641us
I20260709 13:17:45.302217 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.311612 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1544) took 1189 ms. Trace:
I20260709 13:17:45.312176 10047 rpcz_store.cc:276] 0709 13:17:44.114833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.217032 (+1102199us) service_pool.cc:224] Handling call
0709 13:17:45.242654 (+ 25622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.292596 (+ 49942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.292643 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.303938 (+ 11295us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:45.304000 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.304089 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.304250 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.309353 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:45.337487 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 51102.9us
I20260709 13:17:45.285791 13234 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.326391 13236 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:45.357757 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.302132 13188 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:17:45.363355 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 256) took 1291 ms. Trace:
I20260709 13:17:45.363744 13188 consensus_queue.cc:1243] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:17:45.363922 10054 rpcz_store.cc:276] 0709 13:17:44.071859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.845692 (+773833us) service_pool.cc:224] Handling call
0709 13:17:44.864688 (+ 18996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.875708 (+ 11020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.875755 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.963643 (+ 87888us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.050492 (+ 86849us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:45.050548 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.050646 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.265829 (+215183us) raft_consensus.cc:1685] finished
0709 13:17:45.265856 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.362843 (+ 96987us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b5400235ad0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:45.363163 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37879168,"thread_start_us":86141,"threads_started":1,"wal-append.queue_time_us":271742}
I20260709 13:17:45.366106 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.368040 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 262) took 2259 ms. Trace:
I20260709 13:17:45.368610 10203 rpcz_store.cc:276] 0709 13:17:43.108628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.144417 (+ 35789us) service_pool.cc:224] Handling call
0709 13:17:43.360924 (+216507us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:44.064958 (+704034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.065011 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:44.076872 (+ 11861us) log.cc:844] Serialized 52 byte log entry
0709 13:17:44.316646 (+239774us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:44.316695 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.323652 (+  6957us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:44.883802 (+560150us) raft_consensus.cc:1685] finished
0709 13:17:44.883854 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.367519 (+483665us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54001ff750. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:45.367807 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143257,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"spinlock_wait_cycles":420529024,"thread_start_us":942347,"threads_started":2,"wal-append.queue_time_us":556938}
I20260709 13:17:45.369897 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.370347 12884 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:17:45.372831 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98936.9us
W20260709 13:17:45.373814 10096 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.647s (SENT)
I20260709 13:17:45.249997 10050 rpcz_store.cc:276] 0709 13:17:44.093823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.047763 (+953940us) service_pool.cc:224] Handling call
0709 13:17:45.097951 (+ 50188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.098419 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:45.171343 (+ 72924us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.244098 (+ 72755us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.244317 (+   219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.244385 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:45.244439 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.244524 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.244683 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54244,"prepare.queue_time_us":71915,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":151,"raft.queue_time_us":56705,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":39706,"thread_start_us":77418,"threads_started":2}
I20260709 13:17:45.374485 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:45.375947 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1536) took 1292 ms. Trace:
I20260709 13:17:45.376508 10049 rpcz_store.cc:276] 0709 13:17:44.083422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.855939 (+772517us) service_pool.cc:224] Handling call
0709 13:17:44.856724 (+   785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.857642 (+   918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.857699 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.858887 (+  1188us) log.cc:844] Serialized 26 byte log entry
0709 13:17:44.994597 (+135710us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:44.994662 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:44.994757 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.375648 (+380891us) raft_consensus.cc:1685] finished
0709 13:17:45.375669 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.375870 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14639,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":193,"thread_start_us":128486,"threads_started":1,"wal-append.queue_time_us":386325}
I20260709 13:17:45.379817 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.380152 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:45.373854 13179 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:45.381062 10049 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:45.382462 13111 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.383845 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.380406 10200 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 13:17:45.382839 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.390872 10050 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:45.388682 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 10) took 1246 ms. Trace:
I20260709 13:17:45.366946 10054 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:45.385016 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.392388 10199 rpcz_store.cc:276] 0709 13:17:44.110786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.634319 (+523533us) service_pool.cc:224] Handling call
0709 13:17:45.340778 (+706459us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54002fbcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:45.356846 (+ 16068us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984819968}
W20260709 13:17:45.410409 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.390560 12686 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:45.374804 13020 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.416388 12860 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:45.436448 13233 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:45.437160 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:45.438269 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64563.1us
I20260709 13:17:45.423646 13123 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:45.440299 13289 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.362s	user 0.002s	sys 0.005s
I20260709 13:17:45.443998 13234 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.414597 12686 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:17:45.445026 10096 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.656s (SENT)
I20260709 13:17:45.445334 12686 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.445771 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:45.437311 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.430486 12860 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.451017 13196 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:45.451313 13123 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:45.430821 13235 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.638s	user 0.015s	sys 0.008s
I20260709 13:17:45.464855 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.451999 12860 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:45.452229 13236 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.468139 10196 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:17:45.469842 13292 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:45.471254 13292 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:45.466078 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.464881 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:45.471647 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.451354 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 265) took 2325 ms. Trace:
I20260709 13:17:45.456115 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:45.462736 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.458789 10047 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:45.480947 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.457765 13153 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:17:45.491036 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.492312 10897 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:45.492769 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1545) took 1372 ms. Trace:
I20260709 13:17:45.494520 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:45.495545 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140793us
I20260709 13:17:45.496490 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:45.497292 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1546) took 1316 ms. Trace:
I20260709 13:17:45.497960 13196 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:45.498322 10054 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.498859 13196 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:45.499884 13179 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:45.500388 13179 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.500809 13179 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:45.500181 13196 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:45.502987 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.504782 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1489) took 1613 ms. Trace:
I20260709 13:17:45.505329 10128 rpcz_store.cc:276] 0709 13:17:43.891748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.892518 (+   770us) service_pool.cc:224] Handling call
0709 13:17:45.504476 (+1611958us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540014bc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:45.504665 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934973696}
I20260709 13:17:45.498018 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:45.497773 10047 rpcz_store.cc:276] 0709 13:17:44.181184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.496161 (+1314977us) service_pool.cc:224] Handling call
0709 13:17:45.496644 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.496915 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.496950 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.496994 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:45.497029 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.497095 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.497220 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.493458 10055 rpcz_store.cc:276] 0709 13:17:44.120277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.491224 (+1370947us) service_pool.cc:224] Handling call
0709 13:17:45.491863 (+   639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.492241 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.492282 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.492347 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.492405 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.492487 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.492642 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:45.511073 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.526132 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1534) took 1444 ms. Trace:
W20260709 13:17:45.533588 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.476778 10202 rpcz_store.cc:276] 0709 13:17:43.109447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.377632 (+268185us) service_pool.cc:224] Handling call
0709 13:17:43.378208 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.090406 (+712198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.090452 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.963077 (+872625us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:17:45.089649 (+126572us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.226041 (+136392us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:45.233636 (+  7595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.238367 (+  4731us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.412581 (+174214us) raft_consensus.cc:1685] finished
0709 13:17:45.412617 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.435051 (+ 22434us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126190,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":4365184,"thread_start_us":847298,"threads_started":2,"wal-append.queue_time_us":317838}
W20260709 13:17:45.539492 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94577.9us
I20260709 13:17:45.531525 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.543222 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.541020 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:45.542938 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.551994 10055 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:45.526985 10051 rpcz_store.cc:276] 0709 13:17:44.081581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.851715 (+770134us) service_pool.cc:224] Handling call
0709 13:17:44.852595 (+   880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:44.853266 (+   671us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:44.985913 (+132647us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.134043 (+148130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:45.134123 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.144716 (+ 10593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.144835 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.406780 (+261945us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b1800254560. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:45.525729 (+118949us) spinlock_profiling.cc:243] Waited 9.75 ms on lock 0x7b1800254560. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4
0709 13:17:45.525957 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16351,"prepare.queue_time_us":290053,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":62,"raft.queue_time_us":37651,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":76376,"spinlock_wait_cycles":79049216,"thread_start_us":187315,"threads_started":2}
I20260709 13:17:45.551044 10050 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:45.576449 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:45.568593 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.685s (SENT) [suppressed 43 similar messages]
I20260709 13:17:45.581568 13223 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.582509 13223 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.585657 13223 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.591603 10202 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [5.2-7.3]
I20260709 13:17:45.593171 10205 raft_consensus.cc:2368] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:45.593667 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1484) took 2218 ms. Trace:
I20260709 13:17:45.596073 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:45.594312 10129 rpcz_store.cc:276] 0709 13:17:43.362545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:43.485196 (+122651us) service_pool.cc:224] Handling call
0709 13:17:45.565775 (+2080579us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:45.580419 (+ 14644us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3050691840}
I20260709 13:17:45.587046 12884 tablet_bootstrap.cc:492] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:17:45.563828 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:45.604751 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:45.623236 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:45.588536 13235 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:45.619091 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:45.610782 10054 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:45.575222 13123 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:45.587208 13020 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.643545 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.645502 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:45.652524 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:45.654639 10201 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:45.642345 12883 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.385s	user 0.027s	sys 0.008s
I20260709 13:17:45.654755 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:45.665968 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:45.668939 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 282178us
I20260709 13:17:45.639484 13235 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:45.677395 10204 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:45.683754 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 185809us
I20260709 13:17:45.688695 10897 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:45.682093 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1535) took 1455 ms. Trace:
I20260709 13:17:45.695816 10047 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:45.697667 13315 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:45.698403 13315 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:45.698771 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:45.701144 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44095.5us
I20260709 13:17:45.703883 10129 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:45.690529 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.706966 13282 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.717083 13282 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.710637 13203 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:17:45.711447 13116 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.268s	user 0.010s	sys 0.016s
I20260709 13:17:45.721197 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:45.722431 10125 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:45.721939 13116 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.723335 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:45.723986 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.724342 10126 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:45.724581 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.725147 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.725731 10123 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:17:45.726370 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.727373 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 252) took 1156 ms. Trace:
I20260709 13:17:45.727869 10123 rpcz_store.cc:276] 0709 13:17:44.571093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.571359 (+   266us) service_pool.cc:224] Handling call
0709 13:17:44.950912 (+379553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.726233 (+775321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.726987 (+   754us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:45.727277 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780631936}
W20260709 13:17:45.736644 13291 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.579s	user 0.000s	sys 0.006s
I20260709 13:17:45.737283 13116 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.734525 10203 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:17:45.738806 13073 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:45.740141 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:45.739866 13073 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.741384 10203 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:45.698410 10047 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:45.744184 13301 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:17:45.746021 13302 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:17:45.726683 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.748673 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1539) took 1664 ms. Trace:
I20260709 13:17:45.749272 10053 rpcz_store.cc:276] 0709 13:17:44.084359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.860397 (+776038us) service_pool.cc:224] Handling call
0709 13:17:44.861242 (+   845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.862318 (+  1076us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:44.984727 (+122409us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.137598 (+152871us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.159517 (+ 21919us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.159756 (+   239us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.159798 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.159887 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.748173 (+588286us) raft_consensus.cc:1685] finished
0709 13:17:45.748211 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.748388 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42882,"prepare.queue_time_us":295744,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":36790,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":8526,"thread_start_us":219006,"threads_started":2,"wal-append.queue_time_us":476902}
I20260709 13:17:45.757932 13282 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:45.760195 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:45.761523 13303 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:17:45.761265 13315 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:45.765213 13315 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:45.702704 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:45.788198 10197 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:17:45.792140 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1542) took 1677 ms. Trace:
W20260709 13:17:45.802420 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 99948.7us
I20260709 13:17:45.805116 13223 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:45.805899 13300 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.362s	user 0.000s	sys 0.005s
I20260709 13:17:45.806423 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:45.792902 10056 rpcz_store.cc:276] 0709 13:17:44.114253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.079500 (+965247us) service_pool.cc:224] Handling call
0709 13:17:45.080084 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.080848 (+   764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.080903 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.081703 (+   800us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.226591 (+144888us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:45.226706 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.226831 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.791685 (+564854us) raft_consensus.cc:1685] finished
0709 13:17:45.791727 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.792012 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5572,"raft.queue_time_us":9578,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":36963,"thread_start_us":144139,"threads_started":1,"wal-append.queue_time_us":517772}
I20260709 13:17:45.676657 13235 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:45.820490 10124 raft_consensus.cc:2368] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:45.835779 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:45.851359 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:45.692678 10048 rpcz_store.cc:276] 0709 13:17:44.082173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.853455 (+771282us) service_pool.cc:224] Handling call
0709 13:17:44.854608 (+  1153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.981694 (+127086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.981750 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.048838 (+ 67088us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.223472 (+174634us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:45.251560 (+ 28088us) log.cc:844] Serialized 14 byte log entry
0709 13:17:45.290711 (+ 39151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.290998 (+   287us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.533154 (+242156us) raft_consensus.cc:1685] finished
0709 13:17:45.533196 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.537834 (+  4638us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27129,"prepare.queue_time_us":19451,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":40296,"raft.run_cpu_time_us":720,"raft.run_wall_time_us":48666,"thread_start_us":101226,"threads_started":1,"wal-append.queue_time_us":285489}
I20260709 13:17:45.780817 12884 ts_tablet_manager.cc:1403] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 3.680s	user 0.078s	sys 0.051s
I20260709 13:17:45.869740 10122 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:17:45.870504 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:45.819782 13073 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.833081 13236 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:45.885674 10129 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:45.885282 10196 raft_consensus.cc:1217] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[5.3-7.4]   Dedup: 7.4->[]
I20260709 13:17:45.937431 10196 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:17:45.690399 12883 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:45.919440 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1537) took 1737 ms. Trace:
I20260709 13:17:45.923017 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:45.772794 13123 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:45.944559 10125 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:45.951589 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83490.4us
I20260709 13:17:45.938655 10052 rpcz_store.cc:276] 0709 13:17:44.083711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:44.856388 (+772677us) service_pool.cc:224] Handling call
0709 13:17:44.857193 (+   805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:44.858146 (+   953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:44.858189 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:44.863103 (+  4914us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.124095 (+260992us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:45.124276 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.124427 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:45.691263 (+566836us) spinlock_profiling.cc:243] Waited 7.74 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:45.808498 (+117235us) raft_consensus.cc:1685] finished
0709 13:17:45.808530 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:45.821528 (+ 12998us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14443,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":152,"spinlock_wait_cycles":17792896,"thread_start_us":156879,"threads_started":1,"wal-append.queue_time_us":514346}
W20260709 13:17:45.944006 13111 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.005s
I20260709 13:17:45.956422 10214 tablet_copy_source_session.cc:215] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.958187 13123 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:45.960263 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58234.8us
I20260709 13:17:45.956229 10131 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:17:45.973038 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.985087 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:45.986438 10130 raft_consensus.cc:2368] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:45.987264 12883 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:45.995760 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 186523us
W20260709 13:17:46.004141 13214 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.802s	user 0.002s	sys 0.018s
I20260709 13:17:46.004989 13214 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.006814 10896 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:46.013191 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:45.930325 13292 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:46.027760 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.039374 10201 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:46.045516 10125 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:46.051447 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73376.5us
I20260709 13:17:46.049625 13172 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:17:45.977072 13196 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:46.072300 10050 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:17:46.083256 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 430822us
I20260709 13:17:45.965057 12686 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:46.084995 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53019.4us
I20260709 13:17:46.088564 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.094709 10125 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:17:46.112383 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.113312 10196 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:46.120290 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.076514 13020 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:17:46.062690 12884 raft_consensus.cc:359] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.139449 12884 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: FOLLOWER
I20260709 13:17:46.140843 12884 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:46.119762 10019 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.511s (SENT)
I20260709 13:17:46.133083 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:46.149582 10894 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:46.147491 13214 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:46.166702 13218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cebbf225b866442ab895577b0496de1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:46.166435 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:46.166296 10131 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:46.142881 12860 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.007s
W20260709 13:17:46.172856 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49133.3us
I20260709 13:17:46.173714 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:46.173484 13179 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.010s
W20260709 13:17:46.178364 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.176488 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60927.8us
W20260709 13:17:46.187232 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.189534 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:46.194969 10196 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:46.175884 12883 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:46.169478 13235 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:46.227268 13309 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.584s	user 0.003s	sys 0.003s
W20260709 13:17:46.234270 10096 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.032s	sys 0.045s
W20260709 13:17:46.235324 10096 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.033s	sys 0.045s
W20260709 13:17:46.235867 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 675449us
I20260709 13:17:46.181295 12860 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.204806 12883 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:46.247210 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.247467 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:46.248152 10124 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:46.248446 10127 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.239838 10197 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:46.242267 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.241663 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.246232 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.190203 13179 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:46.249779 10125 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:46.248241 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.250202 10130 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:46.261214 10129 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
W20260709 13:17:46.286870 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.728s (SENT)
I20260709 13:17:46.287833 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:46.289454 10128 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.247239 12860 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:46.299949 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:46.241276 13320 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
I20260709 13:17:46.316823 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:46.314761 12884 ts_tablet_manager.cc:1434] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.432s	user 0.012s	sys 0.003s
I20260709 13:17:46.329702 10128 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:46.337709 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.318545 13236 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:46.379055 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 268267us
I20260709 13:17:46.324577 13111 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:46.338851 12860 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:46.382899 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.385993 10130 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.334481 12686 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:46.388639 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1551) took 1125 ms. Trace:
W20260709 13:17:46.389650 13327 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:17:46.390197 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 117397us
W20260709 13:17:46.391242 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.770s (SENT)
I20260709 13:17:46.391659 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:46.393256 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.822s (SENT)
I20260709 13:17:46.393757 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:46.394914 13282 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:46.395864 13282 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:46.396088 13328 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:17:46.400638 13179 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:46.344569 13236 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:46.333523 13196 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:46.389233 10049 rpcz_store.cc:276] 0709 13:17:45.277275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.621361 (+344086us) service_pool.cc:224] Handling call
0709 13:17:45.657497 (+ 36136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.709925 (+ 52428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.709966 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.710685 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 13:17:46.064325 (+353640us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:46.064395 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.064520 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.388345 (+323825us) raft_consensus.cc:1685] finished
0709 13:17:46.388376 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.388542 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10381,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":352595,"threads_started":1,"wal-append.queue_time_us":605173}
I20260709 13:17:46.431983 10127 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:46.386122 13235 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:46.385704 12860 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:46.519963 12860 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.521581 12860 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.403101 13236 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.484825 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.524044 10130 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:46.524555 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.410527 13196 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:46.487056 13223 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:46.528923 13223 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.529772 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.530433 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.499193 13315 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:46.503728 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:46.531217 10127 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 4.
I20260709 13:17:46.531709 10196 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:46.490370 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 374092us
I20260709 13:17:46.532581 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.533901 10125 raft_consensus.cc:2368] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:46.521159 12883 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:46.535490 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:46.536010 13312 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.664s	user 0.004s	sys 0.000s
I20260709 13:17:46.536217 12883 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:46.536571 10124 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.530534 10122 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:46.508437 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f received from {username='slave'} at 127.0.0.1:50726
I20260709 13:17:46.538254 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.538870 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-806f35e66c234d569cbafd8b3981518f on tablet 806f35e66c234d569cbafd8b3981518f with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:46.539069 10126 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
W20260709 13:17:46.517154 13314 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.399s	user 0.002s	sys 0.000s
I20260709 13:17:46.543622 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:46.544409 10122 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:46.545424 10019 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.730s (SENT)
I20260709 13:17:46.546017 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:46.539996 13223 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:46.549190 13292 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:46.463653 13203 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:46.470271 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 237924us
I20260709 13:17:46.561826 13292 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:17:46.565274 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.604532 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.605854 10126 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.612145 10128 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:46.582746 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:46.612344 10196 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:17:46.620664 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 336842us
I20260709 13:17:46.626130 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:46.626686 10127 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:46.627436 10127 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:46.623453 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.623292 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.631038 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.619549 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:46.628911 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 274) took 1006 ms. Trace:
W20260709 13:17:46.633891 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164481us
I20260709 13:17:46.653605 10125 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:46.657357 10128 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:46.659479 10130 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:46.653060 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:46.665643 13223 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:46.665789 13316 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.667551 13316 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.667874 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:46.668833 10125 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:46.671321 13084 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:46.671900 13084 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.671610 13316 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:46.672333 13084 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:46.680419 10127 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:46.682216 10127 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.633314 10205 rpcz_store.cc:276] 0709 13:17:45.622882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.623058 (+   176us) service_pool.cc:224] Handling call
0709 13:17:45.623696 (+   638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:45.624265 (+   569us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:45.854496 (+230231us) log.cc:844] Serialized 14 byte log entry
0709 13:17:46.310420 (+455924us) log.cc:844] Serialized 14 byte log entry
0709 13:17:46.627771 (+317351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:46.628519 (+   748us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:46.628595 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:46.628676 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.628836 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11399,"prepare.queue_time_us":456044,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":125,"raft.queue_time_us":102492,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":6488448,"thread_start_us":898917,"threads_started":3,"wal-append.queue_time_us":533853}
I20260709 13:17:46.690413 10124 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:17:46.685251 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:46.699492 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64654.1us
I20260709 13:17:46.700796 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:46.705221 10122 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:46.709794 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:46.711329 13235 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:46.711943 13235 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:46.712396 13235 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:46.716837 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:46.721009 10123 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.757067 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:46.760597 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 426546us
I20260709 13:17:46.751349 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:46.713811 13172 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:46.785916 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.782555 13336 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:46.798790 13324 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.502s	user 0.003s	sys 0.007s
I20260709 13:17:46.798696 13282 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:46.850843 13111 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:46.873204 13111 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.874660 10127 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 3. (term mismatch)
I20260709 13:17:46.875103 13123 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:46.875921 13123 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:46.876402 13313 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:46.877172 13313 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:46.878463 13123 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:46.879375 13332 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:17:46.875329 10127 pending_rounds.cc:85] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:46.889734 10130 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:46.867676 13218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:60866 (request call id 556) took 4274 ms. Trace:
W20260709 13:17:46.904670 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.882828 10127 pending_rounds.cc:107] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.3
I20260709 13:17:46.909394 10127 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:46.912026 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:46.910241 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 516948us
W20260709 13:17:46.917127 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 355274us
I20260709 13:17:46.929157 10894 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:46.940418 10203 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:46.928102 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:46.857769 13282 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:46.955675 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:46.958910 10096 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.872s (SENT)
I20260709 13:17:46.959626 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:46.878666 13313 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:46.908196 13123 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:17:46.980970 10020 leader_election.cc:400] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:46.981469 10020 leader_election.cc:403] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:46.987423 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 275) took 1361 ms. Trace:
I20260709 13:17:46.992118 10202 rpcz_store.cc:276] 0709 13:17:45.626012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:45.626209 (+   197us) service_pool.cc:224] Handling call
0709 13:17:45.626825 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:45.627626 (+   801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:45.627673 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:45.628635 (+   962us) log.cc:844] Serialized 26 byte log entry
0709 13:17:45.884456 (+255821us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:45.884518 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:45.898830 (+ 14312us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:46.766787 (+867957us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b24003c2828. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274d9d9c4 00007fa274d9bacb 00007fa274f56dc9 00007fa274f56a55 00007fa274f5609d 00007fa277b8810e 00007fa277b9f88c 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221
0709 13:17:46.977525 (+210738us) raft_consensus.cc:1685] finished
0709 13:17:46.977552 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:46.987297 (+  9745us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2046,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":175,"spinlock_wait_cycles":11647232,"thread_start_us":255139,"threads_started":1,"wal-append.queue_time_us":589510}
I20260709 13:17:46.976228 13315 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:46.986927 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:46.993453 13292 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:17:47.008594 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 151725us
W20260709 13:17:47.001143 13341 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.255s	user 0.001s	sys 0.001s
I20260709 13:17:46.995960 13313 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:47.017617 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:47.020268 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.023077 13351 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:47.023798 13351 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:47.024896 13351 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.026572 13351 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:47.043440 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.037634 13292 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.051296 13111 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:47.052124 13111 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:17:47.052404 12860 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.006s
I20260709 13:17:47.051349 13292 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.055397 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102674us
I20260709 13:17:47.071795 13282 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:47.072757 13282 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.080506 13223 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:47.081306 13223 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.082954 13292 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.085325 13292 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:47.086196 13292 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.091692 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.889s (SENT)
I20260709 13:17:47.092494 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:47.094408 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.028s (SENT)
I20260709 13:17:47.028957 13345 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.108569 13345 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.123291 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:46.900594 13218 rpcz_store.cc:276] 0709 13:17:42.593114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.593290 (+   176us) service_pool.cc:224] Handling call
0709 13:17:43.481609 (+888319us) ts_tablet_manager.cc:934] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
0709 13:17:46.127852 (+2646243us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:46.548023 (+420171us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:46.548203 (+   180us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:46.833612 (+285409us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:46.856424 (+ 22812us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:46.867254 (+ 10830us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:46.867488 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":50851,"tablet-copy.queue_time_us":744898,"thread_start_us":1151032,"threads_started":2}
I20260709 13:17:47.144001 13218 tablet_copy_client.cc:806] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:47.144852 13218 tablet_copy_client.cc:670] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:47.039435 13315 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:47.133950 13282 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.160048 13282 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:47.160806 13282 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.165597  9953 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 116) took 11948 ms (client timeout 15000 ms). Trace:
I20260709 13:17:47.189424 13292 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.190243 13292 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.094947 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:47.196364 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102050us
W20260709 13:17:47.197182 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.079s (SENT)
I20260709 13:17:47.197587 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:47.201246 13236 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.002s
I20260709 13:17:47.201901 13236 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.198961 10096 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.084s (SENT)
I20260709 13:17:47.210383 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:47.211717 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 431750us
W20260709 13:17:47.203287 13203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.026s	sys 0.022s
W20260709 13:17:47.214913 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 443081us
I20260709 13:17:47.216419 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2737) took 1029 ms. Trace:
W20260709 13:17:47.230330 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.226727 10131 rpcz_store.cc:276] 0709 13:17:46.186749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:46.186945 (+   196us) service_pool.cc:224] Handling call
0709 13:17:46.204422 (+ 17477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:46.221917 (+ 17495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:46.221958 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:46.254191 (+ 32233us) log.cc:844] Serialized 26 byte log entry
0709 13:17:46.435575 (+181384us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:47.133095 (+697520us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b1800375040. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf
0709 13:17:47.133287 (+   192us) log.cc:844] Serialized 14 byte log entry
0709 13:17:47.133560 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.133665 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:47.203565 (+ 69900us) raft_consensus.cc:1685] finished
0709 13:17:47.216108 (+ 12543us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:47.216317 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":409712,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":6834,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":141,"spinlock_wait_cycles":36927360,"thread_start_us":510961,"threads_started":2,"wal-append.queue_time_us":334359}
W20260709 13:17:47.228967 13084 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.003s
W20260709 13:17:47.235371 13203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.026s	sys 0.022s
I20260709 13:17:47.217377 13336 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:47.157850 13315 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.229024 13234 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.271658 13357 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:47.272545 13357 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:47.275300 13354 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:47.166467  9953 rpcz_store.cc:269] 0709 13:17:35.228758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:35.242012 (+ 13254us) service_pool.cc:224] Handling call
0709 13:17:47.165360 (+11923348us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:41.537000 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:41.537722 (+   722us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:41.537761 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:17:41.537781 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:41.782834 (+245053us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:41.786472 (+  3638us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:41.789052 (+  2580us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:17:41.959774 (+170722us) tablet.cc:701] Row locks acquired
0709 13:17:41.959794 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:17:41.959977 (+   183us) write_op.cc:270] Start()
0709 13:17:41.960118 (+   141us) write_op.cc:276] Timestamp: P: 1783603061959921 usec, L: 0
0709 13:17:41.960139 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:17:41.960954 (+   815us) log.cc:844] Serialized 4334 byte log entry
0709 13:17:44.537345 (+2576391us) op_driver.cc:464] REPLICATION: finished
0709 13:17:44.732393 (+195048us) write_op.cc:301] APPLY: starting
0709 13:17:44.732592 (+   199us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:44.733021 (+   429us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:44.733198 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:44.741061 (+  7863us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:44.741481 (+   420us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:17:44.741558 (+    77us) write_op.cc:312] APPLY: finished
0709 13:17:44.742100 (+   542us) log.cc:844] Serialized 83 byte log entry
0709 13:17:44.742358 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:44.742601 (+   243us) write_op.cc:454] Released schema lock
0709 13:17:44.742659 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:44.780661 (+ 38002us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:44.780719 (+    58us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":71945,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":177089,"apply.run_cpu_time_us":5607,"apply.run_wall_time_us":15663,"num_ops":11,"prepare.queue_time_us":386918,"prepare.run_cpu_time_us":48969,"prepare.run_wall_time_us":848802,"raft.queue_time_us":951228,"raft.run_cpu_time_us":11815,"raft.run_wall_time_us":263004,"replication_time_us":2510458,"thread_start_us":1583157,"threads_started":4,"wal-append.queue_time_us":843044,"wal-append.run_cpu_time_us":55056,"wal-append.run_wall_time_us":2931682}]]}
W20260709 13:17:47.280110 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.283277 13282 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.284034 13282 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.247418 13352 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.249804 13343 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:47.257879 13353 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.289870 13123 consensus_queue.cc:1243] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:17:47.299819 13353 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.303161 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.303082 13353 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.308678 13345 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.309127 13353 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:47.266650 13084 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:17:47.266850 13235 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.549s	user 0.015s	sys 0.002s
I20260709 13:17:47.309482 13111 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:47.310286 13111 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:47.285748  9951 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 5, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 5 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.311614 13111 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.256899 13020 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:17:47.324935 13111 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.325738 13084 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:47.327894 13172 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.002s
I20260709 13:17:47.331881 13223 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:47.333582 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 316078us
I20260709 13:17:47.270750 13234 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.340248 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 215114us
I20260709 13:17:47.336346 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:47.276018 13354 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.349901 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 418249us
I20260709 13:17:47.335103 13172 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.310256 13235 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.309804 13353 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:47.377844 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75168.5us
I20260709 13:17:47.375792  9951 catalog_manager.cc:5697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 8, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.386191 13354 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.388154 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:47.389642 13123 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:47.390678 13357 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:47.391142 13123 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:47.391571 13357 consensus_queue.cc:1243] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:17:47.390066  9951 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.391765 13365 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:47.392218 13357 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:47.392390 13365 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:47.392665 13357 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:47.393615 13357 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.393540 13365 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.319777 13343 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.396104 13365 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:47.397326 13343 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:47.313017 13352 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.398027 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61809.2us
I20260709 13:17:47.395787 13084 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:47.420845 10198 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:47.408252 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 209331us
I20260709 13:17:47.324935 13336 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.424043 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:47.425673 10095 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:47.423267 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.474s (SENT)
I20260709 13:17:47.432520 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:47.397367 13234 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:47.465332 10205 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:47.432139 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:47.436259 13367 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.472506 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.472335 13367 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.472096  9951 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 4, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 4 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.395843 13356 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:47.421712 13084 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:47.475606 13356 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:47.476455 13084 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.476965 13356 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.478349 13084 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:47.391832 13123 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:47.465816 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.416862 13357 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.480486 10127 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:17:47.470919 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:47.475291 13218 tablet_copy_client.cc:538] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:47.483732 10131 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:47.466375 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:47.484567 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 52591.2us
I20260709 13:17:47.474511  9951 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:47.536234 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48794.3us
W20260709 13:17:47.541687 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.484480 13236 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:47.548179  9951 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 4, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 4 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.604241 13123 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.486711 13367 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:47.555863 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70264.2us
I20260709 13:17:47.546769 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.619787 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:47.616528 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 119105us
I20260709 13:17:47.628408 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.628890  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:47.631466 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.625788 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:47.625815 13351 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:47.639393 13352 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.641741 13292 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:47.645274 10196 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:47.646977 10049 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:17:47.623448 13234 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:47.647727 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:47.642877 13235 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:47.654045 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:47.615310 13369 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.659554 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:47.691344 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 268181us
I20260709 13:17:47.665585  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:47.669113 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.644100 13111 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.670719 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:47.671576 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:47.699698 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:47.699528 10047 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:47.671437  9951 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.648945 13203 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:47.704802 10895 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:47.680092 10019 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.162s (SENT)
I20260709 13:17:47.680081 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:47.681306 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.681914 13234 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:47.683318 13084 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.684070 13282 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.684530 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:47.682825 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:47.686774 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.710727 10123 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:47.687574 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.687358 13345 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:17:47.651535 13351 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:47.708348 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.714962 13344 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:47.714836 13084 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.717969 13234 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:47.719556 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:47.730966 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.731578 10124 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.731828 10048 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:47.721171 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.732510 10052 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:47.732823 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.734177 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:47.735169 10050 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:47.743392 10130 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:47.743993 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 89145.2us
I20260709 13:17:47.745178 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:47.746755  9951 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 5, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 5 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.769330 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:47.769337 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.681525 13362 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:17:47.755008 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.798733 10047 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.795915  9951 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 6, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:47.728165 13292 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.808851 10198 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:47.811239 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:47.813700 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44817.3us
I20260709 13:17:47.813995 10056 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:47.816658 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:47.817622 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.856745 10054 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:47.819489 13353 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:47.820240 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.825145 13351 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:47.721835 13203 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:47.863054 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:47.864565 13231 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:47.865161 13203 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:47.832962 10125 raft_consensus.cc:1217] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-6.4]   Dedup: 2.2->[5.3-6.4]
I20260709 13:17:47.865891 10125 pending_rounds.cc:85] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 2
I20260709 13:17:47.834174 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.866143 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:47.866519 10126 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:47.866179 10125 pending_rounds.cc:107] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:17:47.867483 10125 raft_consensus.cc:2889] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.867642 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:47.868314 10126 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:47.835556 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.835901 10127 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:47.836800 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:47.861595 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:47.874475 10128 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:47.770889 13364 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:47.876350 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:47.877008 10127 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.841792 13084 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:47.845438  9951 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 2 to 3, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 3 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:47.890183 10051 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:47.901136 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.902073 10056 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:17:47.904999 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.744547 13344 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:47.913715 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 229278us
I20260709 13:17:47.914146 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.915472 10197 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:47.917717 10129 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:17:47.739822 13369 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:47.859719 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 179728us
I20260709 13:17:47.859954 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.933672 10054 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:17:47.934130 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.860486 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.935170 10050 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:47.862890 13374 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:17:47.862619 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:47.944054 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.945163 10053 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:47.907658 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.850884 10052 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:47.930725 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:47.939621  9951 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.918510 10129 pending_rounds.cc:85] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:17:47.951198 10129 pending_rounds.cc:107] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:17:47.951716 10129 raft_consensus.cc:2889] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:47.953487 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.902957 13231 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:47.959128 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:47.960773 10052 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:47.977442 10128 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:47.983621 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:47.963165 13231 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:47.995225 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:47.984681 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.004419 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.010609 10127 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:17:48.012825 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.013271 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.014556 10127 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 6: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:48.005479 10129 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:48.023100 10052 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:48.023350 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.033639 10050 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:48.040622 10126 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:47.971506 13364 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.042066 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:48.043438 10054 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:48.045552 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:48.047901 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:48.049491 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:48.053169 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.054299 10126 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:48.041571 13315 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.016160 13231 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.025265 13292 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:48.040939 13351 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:17:47.971276 13357 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:17:48.056255 13352 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:17:48.059070 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.123903 10127 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:17:48.125219 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:48.061710 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.044976 13353 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:48.128032 10019 leader_election.cc:400] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:48.128674 10019 leader_election.cc:403] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:48.129469 13353 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:48.129715 13344 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:48.134614 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:48.113008 13364 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.076022 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:48.085068 13123 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:48.093212 13357 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:48.096899 13352 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:17:48.098003 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48668.6us
I20260709 13:17:48.172453 10124 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:48.173746 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:48.059706 13315 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.139477 10127 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.143786 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.145157  9951 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 1 to 5, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:48.162434 10048 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:48.132717 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:48.165673 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:48.178872 10050 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:48.166782 10056 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:17:48.170357 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94438.6us
I20260709 13:17:48.180828 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.167356 10047 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:48.184471 10052 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
W20260709 13:17:48.196642 13365 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.777s	user 0.017s	sys 0.000s
I20260709 13:17:48.178025 13344 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:48.206877 13344 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.207342 13344 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.209882 13218 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:17:48.212133  9951 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 7, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 7 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:48.214042 10056 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:17:48.214454 10056 pending_rounds.cc:85] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:17:48.213828  9951 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:48.214756 10056 pending_rounds.cc:107] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:17:48.216619  9951 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 5, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 5 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.219847 10131 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:17:48.222277 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.225083 13344 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.223022 10131 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:48.215016 10056 raft_consensus.cc:2889] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:48.218595  9951 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 6, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 6 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:48.169535 13357 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:48.229018  9951 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 5, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 5 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:17:48.236615 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55923.5us
I20260709 13:17:48.237633 10170 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:48.238734 13234 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.239216 13234 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:48.239745 13234 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.247565 13020 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.248629 13020 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:17:48.248775 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:48.249459 13020 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.257571 13315 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:48.173103 13352 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:48.259076 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.260450 13383 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:48.261646 13383 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:48.263509 13383 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.272543 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:48.230406 13357 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.274951 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:48.278883 10128 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:48.298800 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:48.274082 13337 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:48.305850 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:48.302539 10053 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:48.190289 13214 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.190601 13336 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:48.313974 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 337891us
I20260709 13:17:48.326890 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.329780 10126 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:48.331965 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 158308us
I20260709 13:17:48.332813 10127 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:48.332921 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.334388 10126 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 9 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:17:48.311766 13379 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.244s	user 0.005s	sys 0.000s
I20260709 13:17:48.331681 13337 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:48.334015 13236 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.311295 13360 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:17:48.340673 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:48.346395 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 213788us
I20260709 13:17:48.350234 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.351097 10129 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:48.353794 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:48.354115 13345 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:48.314080 13380 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:48.361460 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.357357 10047 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:48.365247 13172 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:48.397380 10129 pending_rounds.cc:85] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:17:48.398013 10129 pending_rounds.cc:107] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:17:48.398358 10129 raft_consensus.cc:2889] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:48.386636 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:48.419307 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:48.421723 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:48.427325 13357 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:48.335357 13214 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.359367 13344 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:48.358357 13345 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:48.440241 13369 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:48.450385 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91142.6us
I20260709 13:17:48.452385 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:48.360702 13360 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:48.453167 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:48.454077 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:48.454203 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:48.417079 13123 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:48.355535 13236 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.444881 13362 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:17:48.464839 13315 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:48.472677 13382 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:17:48.473934 10899 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:48.474282 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:48.452078 13234 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.472783 13315 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.460865 13367 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:48.475478 13315 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.465101 13282 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:48.468358 13365 raft_consensus.cc:2764] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:17:48.514885 10127 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:17:48.518213 10127 pending_rounds.cc:85] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
I20260709 13:17:48.525525 13234 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:48.476598 13362 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:48.533919 10127 pending_rounds.cc:107] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:17:48.535241 10127 raft_consensus.cc:2889] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:48.514812 13367 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:48.511809 13282 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:48.540835 13362 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:17:48.575798 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 209455us
I20260709 13:17:48.471065 13337 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:48.683291 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 216864us
W20260709 13:17:48.723639 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 145469us
I20260709 13:17:48.559260 13365 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:48.583942 13360 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:17:48.734879 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 457946us
I20260709 13:17:48.736656 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:48.738003 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:48.759248 10049 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:17:48.757184 10863 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.896s (SENT)
I20260709 13:17:48.724406 13236 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:48.772239 10048 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:17:48.776049 13391 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.181s	user 0.010s	sys 0.002s
I20260709 13:17:48.776338 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.776643 13364 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:48.776397 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:48.744946 13386 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.390s	user 0.009s	sys 0.000s
I20260709 13:17:48.734355 13292 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:48.790421 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:48.799590 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:48.760233 13020 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
W20260709 13:17:48.825528 10128 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.000s
I20260709 13:17:48.827414 10899 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:48.830862 10128 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:48.843277 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:48.768141 13383 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.852485  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:48.853740 13214 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:48.853986 10200 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:48.855264 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:48.856153 10198 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:48.857905 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:48.734509 13367 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.855470 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:48.853657 10205 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:48.859352 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:48.863938 10203 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:48.868880 13352 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.870193 13352 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:48.873183 13084 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.874033 10123 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:48.873859 13084 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.875344 13353 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:48.875854 13353 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:48.876264 13353 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.762892 13315 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.881412 13315 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:48.884861 13231 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.799s	user 0.012s	sys 0.004s
I20260709 13:17:48.885668 13231 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.818236 13320 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.887574 13320 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.893379 13398 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.894135 13398 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.896731 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:48.899271 10863 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.902s (SENT)
I20260709 13:17:48.909722 10899 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 9. Ops: []
I20260709 13:17:48.819144 13292 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:48.914306 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:48.918951 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1581) took 1096 ms. Trace:
I20260709 13:17:48.919610 10055 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:48.919538 10056 rpcz_store.cc:276] 0709 13:17:47.822051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.228491 (+406440us) service_pool.cc:224] Handling call
0709 13:17:48.777476 (+548985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:48.777769 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:48.778202 (+   433us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.918541 (+140339us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:48.918604 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:48.918659 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.918738 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.918875 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1586,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"thread_start_us":139921,"threads_started":1}
W20260709 13:17:48.916330 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58612.1us
W20260709 13:17:48.850739 13387 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
I20260709 13:17:48.906605 13218 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20260709 13:17:48.927376 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 103145us
W20260709 13:17:48.929852 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 455687us
I20260709 13:17:48.933301 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:48.848945 13383 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.934201 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:48.935029 10051 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.936571 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:48.913425 13356 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.037s	sys 0.059s
I20260709 13:17:48.904932 13364 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.942935 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:48.947207 10198 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:17:48.947026 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:48.947885 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:48.950392 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:48.951498 10049 raft_consensus.cc:2376] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10. Re-sending same reply.
W20260709 13:17:48.943019 13356 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.037s	sys 0.060s
I20260709 13:17:48.959528 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:48.960525 13236 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:48.961172 13236 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.963265 13356 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:48.963780 13356 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:48.964207 13356 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:48.964241 13084 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:48.824153 13282 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.965745 13282 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:48.913652 13380 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.967667 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:48.925371 13292 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.974282 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1505) took 1142 ms. Trace:
I20260709 13:17:48.975476 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:48.976873 10051 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:48.988711 13236 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:48.989509 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:48.990283 10128 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:48.959570 13234 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:48.992584 10124 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:17:48.993060 10124 pending_rounds.cc:85] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 3
I20260709 13:17:48.993315 10124 pending_rounds.cc:107] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:17:48.993562 10124 raft_consensus.cc:2889] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:48.995555 13414 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:48.996198 13414 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:48.997233 10127 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:48.845239 13020 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:48.998672 13414 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:48.996348 13231 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:49.000867 13020 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:49.001523 13380 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.006918 13380 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:48.941149 13402 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:48.971197 13364 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.008153 13402 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.008931 13402 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:48.975592 10125 rpcz_store.cc:276] 0709 13:17:47.831707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.831923 (+   216us) service_pool.cc:224] Handling call
0709 13:17:47.832643 (+   720us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:47.868295 (+ 35652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.868330 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:47.868963 (+   633us) log.cc:844] Serialized 52 byte log entry
0709 13:17:48.184054 (+315091us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:48.493146 (+309092us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.817065 (+323919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.817198 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:48.974013 (+156815us) raft_consensus.cc:1685] finished
0709 13:17:48.974038 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:48.974205 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4598,"prepare.queue_time_us":401104,"prepare.run_cpu_time_us":10914,"prepare.run_wall_time_us":141270,"raft.queue_time_us":274863,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":46175,"thread_start_us":915083,"threads_started":3,"wal-append.queue_time_us":671293}
I20260709 13:17:48.941306 13383 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:49.011648 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 143030us
W20260709 13:17:49.013068 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45578.2us
I20260709 13:17:49.015744 10128 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:49.018999 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:49.087599 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:49.089807 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 278) took 1229 ms. Trace:
I20260709 13:17:49.091385 10050 rpcz_store.cc:276] 0709 13:17:47.860130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.272132 (+412002us) service_pool.cc:224] Handling call
0709 13:17:48.272730 (+   598us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.273060 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.273102 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.273883 (+   781us) log.cc:844] Serialized 26 byte log entry
0709 13:17:48.557766 (+283883us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:48.653724 (+ 95958us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.087848 (+434124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.087986 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.088035 (+    49us) raft_consensus.cc:1685] finished
0709 13:17:49.088049 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.089704 (+  1655us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53685,"prepare.queue_time_us":95655,"prepare.run_cpu_time_us":2701,"prepare.run_wall_time_us":48346,"raft.queue_time_us":141807,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":579553,"threads_started":2,"wal-append.queue_time_us":681282}
I20260709 13:17:49.094478 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:49.096428 10894 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:49.100235 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.105535 10899 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:48.972710 13411 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:49.115588 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 96728.5us
I20260709 13:17:49.121780 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.124702 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.127197 10020 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:49.127665 10020 leader_election.cc:403] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:49.129560 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 141298us
I20260709 13:17:49.088229 13084 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:49.130939 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1518) took 1068 ms. Trace:
W20260709 13:17:49.132411 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 153130us
I20260709 13:17:49.136462 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:49.135200 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 398647us
I20260709 13:17:49.145401 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.141860 10126 rpcz_store.cc:276] 0709 13:17:48.062595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.403467 (+340872us) service_pool.cc:224] Handling call
0709 13:17:48.404017 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:48.404311 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:48.655573 (+251262us) log.cc:844] Serialized 14 byte log entry
0709 13:17:49.130369 (+474796us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:49.130451 (+    82us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:49.130516 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.130671 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.130852 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32471,"prepare.queue_time_us":348106,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":131,"raft.queue_time_us":92741,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":27983,"thread_start_us":579696,"threads_started":3,"wal-append.queue_time_us":215755}
I20260709 13:17:49.147419 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.101198 13234 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:49.115368 13218 tablet_bootstrap.cc:492] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:17:49.115092 13380 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.151494 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.151721 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.170548 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.171463 10201 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:49.172722 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1512) took 1337 ms. Trace:
I20260709 13:17:49.120911 13414 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:49.191107 13414 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.193787 13420 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.193434 13353 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.194308 13420 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.194391 13414 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.194779 13420 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.148581 13235 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.196331 13235 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:49.197229 13357 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.004s
I20260709 13:17:49.197906 13365 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.198463 13365 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.199609 13416 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.199823 10131 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:49.200248 13416 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.202781 13416 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:49.177425 13362 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.592s	user 0.016s	sys 0.000s
I20260709 13:17:49.116286 13320 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:49.179387 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.144732 13355 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:49.180168 13403 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:17:49.180423 13356 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.181967 13398 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.207809 13235 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:49.106447 13236 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:17:49.183549 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.129252 13394 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:17:49.148849 13344 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:49.146482 13020 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:49.188192 10197 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:49.189265 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:49.251452 10198 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.134712 13369 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:17:49.151759 13409 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:49.173214 10130 rpcz_store.cc:276] 0709 13:17:47.834869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.903808 (+ 68939us) service_pool.cc:224] Handling call
0709 13:17:47.929952 (+ 26144us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:47.961718 (+ 31766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.961774 (+    56us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:17:47.971565 (+  9791us) log.cc:844] Serialized 78 byte log entry
0709 13:17:48.404839 (+433274us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:48.722565 (+317726us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.723049 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 13:17:48.735008 (+ 11959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.735097 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.172471 (+437374us) raft_consensus.cc:1685] finished
0709 13:17:49.172494 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.172659 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24406,"prepare.queue_time_us":395151,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":222,"raft.queue_time_us":289194,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":22050,"thread_start_us":668677,"threads_started":3,"wal-append.queue_time_us":514745}
I20260709 13:17:49.204260 13424 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:49.193533 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.216741 10196 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 5. Current term is 6. Ops: []
W20260709 13:17:49.217574 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47044.1us
I20260709 13:17:49.265235 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:49.265407 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71994.3us
I20260709 13:17:49.219858 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.225996 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:49.240425 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96358.1us
I20260709 13:17:49.109948 13419 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:17:49.182542 13413 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.270512 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.271785 10196 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:49.271414 13421 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:49.273058 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.276224 13356 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:49.279572 13416 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:17:49.281016 13369 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:49.281668 13369 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:49.248036 13218 ts_tablet_manager.cc:1403] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 1.036s	user 0.044s	sys 0.020s
I20260709 13:17:49.216710 13414 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:17:49.122889 13411 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.276400 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.291971 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.281085 13422 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.253872 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.263504 13424 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:49.258486 13380 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.296021 10197 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:17:49.248644 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99776us
I20260709 13:17:49.300038 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.301199 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1516) took 1337 ms. Trace:
I20260709 13:17:49.301832 10129 rpcz_store.cc:276] 0709 13:17:47.963870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.353223 (+389353us) service_pool.cc:224] Handling call
0709 13:17:48.397037 (+ 43814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:48.419398 (+ 22361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:48.419434 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:48.420008 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:17:48.670659 (+250651us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:48.670725 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:48.670835 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:49.300843 (+630008us) raft_consensus.cc:1685] finished
0709 13:17:49.300881 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.301102 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":250106,"threads_started":1,"wal-append.queue_time_us":479883}
I20260709 13:17:49.284593 13364 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.269224 13020 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:49.285765 13398 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.224964 13362 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.307595 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.282760 13369 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.298398 13413 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.267642 13292 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.297935 13084 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:49.339762 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46681.2us
I20260709 13:17:49.313939 13394 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:49.312250 13398 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.327646 13353 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.302634 13380 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.309306 13336 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:17:49.339742 13409 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:17:49.362989 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 86700.6us
I20260709 13:17:49.362000 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.368209 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.344933 13394 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.303002 13234 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.351933 13398 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.327085 13364 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:17:49.376942 13433 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:49.377933 10048 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:49.381801 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:49.299674 13422 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.389788 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:49.397620 10054 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:49.398161 10050 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:49.391122 13218 raft_consensus.cc:359] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 6 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.374293 13344 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:49.392720 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:49.403956 10049 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.404294 10048 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:49.393117 13410 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:49.413499 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104970us
I20260709 13:17:49.393666 13084 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.394851 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:49.360457 13413 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.400996 10198 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:49.366403 13409 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.406126 10052 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:17:49.377643 13433 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.408238 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:49.411386 13427 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:17:49.381620 13424 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.403537 13020 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.412921 13422 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.413173 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:49.393940 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86466.3us
I20260709 13:17:49.374876 13394 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:49.418545 10051 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:49.420315 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:49.421788 10050 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:49.419699 13409 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:49.423650 13235 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:49.424600 13345 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.416496 13355 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:49.432390 10196 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:49.422555 13433 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:49.419955 13413 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.432729 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.452448 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:49.453953 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.452644 13427 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:49.468087 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.420148 13426 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:17:49.471171 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.471990 13364 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.472963 13365 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:49.474077 13352 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.603s	user 0.016s	sys 0.001s
I20260709 13:17:49.475993 13179 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.476814 13427 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:49.478319 13369 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.483356 13411 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:49.484822 13377 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.423s	user 0.019s	sys 0.004s
I20260709 13:17:49.487861 13413 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.488330 13434 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.521839 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.467999 13336 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:17:49.517172 13203 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:17:49.504652 13362 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.523892 13420 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.490198 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.527568 10198 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:49.491084 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.488392 13402 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.496564 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:49.503424 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.532996 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.496537 13364 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.471223 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:49.453243 13292 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.515300 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.516105 10050 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:49.504864 13123 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.540493 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:49.540629 13123 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.492660 13345 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:49.524032 13434 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:49.528411 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.523660 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1493) took 2042 ms. Trace:
W20260709 13:17:49.496511 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50187.3us
I20260709 13:17:49.504262 13422 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.534664 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:49.506134  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:49.503676 10049 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:49.543334 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:49.547967 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.547560 10122 rpcz_store.cc:276] 0709 13:17:47.480990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:47.481143 (+   153us) service_pool.cc:224] Handling call
0709 13:17:47.481597 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:47.481873 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:47.481902 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:47.490905 (+  9003us) log.cc:844] Serialized 26 byte log entry
0709 13:17:47.872739 (+381834us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:47.872808 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:47.872947 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:48.541331 (+668384us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:49.484368 (+943037us) spinlock_profiling.cc:243] Waited 7.2 ms on lock 0x7b240028f748. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274d9d9c4 00007fa274d9bacb 00007fa274f56dc9 00007fa274f56a55 00007fa274f5609d 00007fa277b8810e 00007fa277b9f88c 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221
0709 13:17:49.523346 (+ 38978us) raft_consensus.cc:1685] finished
0709 13:17:49.523376 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:49.523575 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179729408,"thread_start_us":226105,"threads_started":1,"wal-append.queue_time_us":436358}
I20260709 13:17:49.550040 13231 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:49.554344 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.533914 13420 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.556125 13420 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:49.556340 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.557003 13420 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:49.557390 13420 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.557751 13420 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:49.557859 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.558374 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:49.559077 10048 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:49.523785 13435 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:49.560434 13435 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:49.561393 13438 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:17:49.512010 13320 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:17:49.561558 13435 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.550261 13422 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.559770 13353 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:49.564635 13422 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.565398 13353 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:49.565851 13353 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.566262 13353 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:49.554461 13231 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:49.567389 13231 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.559824 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.514114 13218 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:17:49.571481 13362 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.572073 13362 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.571877 13218 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:49.573731 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:49.574342 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82358.5us
I20260709 13:17:49.574661 13362 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.572461 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:49.552183 10050 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:49.581552 10199 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:49.582751 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.552939 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1522) took 1488 ms. Trace:
I20260709 13:17:49.583815 10131 rpcz_store.cc:276] 0709 13:17:48.064302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.227291 (+162989us) service_pool.cc:224] Handling call
0709 13:17:49.552662 (+1325371us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540027d3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:49.552856 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967043712}
W20260709 13:17:49.591197 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58266.9us
W20260709 13:17:49.595928 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 125264us
I20260709 13:17:49.596710 10050 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:49.546669 13364 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:49.610435 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 113983us
I20260709 13:17:49.585837 13362 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:49.586006 13434 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:49.594832 13367 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:49.626973 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.635226 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:49.604267 13196 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:49.637691 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.638738 10128 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:49.586506 13235 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:17:49.608467 13413 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.663069 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:49.661131 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47694.3us
I20260709 13:17:49.663808 10125 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:49.661904 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.665829 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.666579 10130 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:49.639909 13411 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.600932 13236 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:17:49.639456 13365 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.523761 13427 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.658419 13367 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:49.657533 13203 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:49.673564 13383 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.003s
W20260709 13:17:49.689917 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63092.5us
I20260709 13:17:49.694876 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:49.637032 13320 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:17:49.699596 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:49.703303 13439 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:17:49.703646 13352 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:49.704310 13352 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.704789 13352 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:49.712242 13123 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.712761 13433 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:49.712834 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:49.714483 13367 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.715992 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:49.717111 13292 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:49.724575 13320 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:49.729404 13402 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:49.673913 13411 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.750600 10899 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:49.762535 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 103041us
I20260709 13:17:49.673286 13413 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.764987 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:49.766497 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 101263us
I20260709 13:17:49.750599 13123 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:49.691870 13365 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.752296 13414 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:49.848344 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 272039us
I20260709 13:17:49.699331 13383 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.859213 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:49.772475 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:49.776533 13196 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:49.830279 13443 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:17:49.862394 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 260774us
I20260709 13:17:49.868359 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:49.870255 10130 raft_consensus.cc:2376] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6. Re-sending same reply.
I20260709 13:17:49.831264 13218 ts_tablet_manager.cc:1434] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.548s	user 0.013s	sys 0.000s
I20260709 13:17:49.871558 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.776518 13434 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:49.874768 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:49.766806 13292 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:17:49.912690 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:49.873171 10129 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:49.872563 13453 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:49.862938 13351 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.920758 10203 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:17:49.865837 13362 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:49.934659 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75550.9us
W20260709 13:17:49.875582 13218 ts_tablet_manager.cc:1467] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Tablet startup took 590ms
W20260709 13:17:49.924409 13398 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.002s
I20260709 13:17:49.858914 13123 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.922708 13292 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.931456 13351 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.922354 13453 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:49.943713 13449 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:17:49.944028 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.932034 13362 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:49.932317 13410 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.003s
W20260709 13:17:49.948798 13448 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:17:49.949779 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:49.951853 13411 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.955473 13084 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:49.956410 13418 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:49.964188 13446 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.257s	user 0.003s	sys 0.000s
I20260709 13:17:49.967170 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.123915 10124 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:17:50.125007 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.125628 10124 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:49.966928 13231 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:49.970360 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.980064 13451 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:49.981379 13196 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.984579 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:49.993523 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2735) took 1086 ms. Trace:
I20260709 13:17:50.098595 13410 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:49.997833 13365 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.999279 13413 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:49.918535 13434 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.008942 13421 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:49.988296 13320 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:17:50.012029 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.020195 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:49.871681 13414 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:50.052630 13345 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:17:49.986394 13353 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.940989 13409 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:49.988412 13458 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.110s
W20260709 13:17:50.121798 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 265120us
I20260709 13:17:49.950745 13422 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.138051 10203 rpcz_store.cc:276] 0709 13:17:48.907744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.908091 (+   347us) service_pool.cc:224] Handling call
0709 13:17:48.928363 (+ 20272us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:49.922676 (+994313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:49.993197 (+ 70521us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54002de1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:49.993447 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201842048}
I20260709 13:17:49.996357 13369 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.138927 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:49.992616 13352 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.029922 13398 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:50.121521 13394 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.674s	user 0.014s	sys 0.001s
W20260709 13:17:50.169353 10125 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.001s
I20260709 13:17:50.199478 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.122082 13084 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:50.181842 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 238074us
I20260709 13:17:50.123214 13418 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:50.138221 13451 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.185354 10122 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:50.188484 10124 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:50.194890 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.205577 10129 raft_consensus.cc:2368] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:50.197443 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:50.199616 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:50.206883 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.076521 13292 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.202698 10130 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:50.204059 10056 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:50.207360 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.214522 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:50.207767 10128 raft_consensus.cc:2376] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7. Re-sending same reply.
W20260709 13:17:50.203629 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 431228us
I20260709 13:17:50.216223 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.217062 10128 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:50.218006 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.219724 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.220515 10125 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 5: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:50.221385 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.222513 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.223999 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.224829 10125 raft_consensus.cc:2376] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9. Re-sending same reply.
I20260709 13:17:50.226087 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.194964 13369 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.196493 13320 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:50.204484 13084 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.212098 13451 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:50.182629 13421 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.227147 10125 raft_consensus.cc:2368] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:50.281955 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.285111 13398 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.287336 13418 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.289052 13418 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.204895 13394 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.202461 13419 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 13:17:50.240437 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 365748us
I20260709 13:17:50.312440 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.243394 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.313462 10125 raft_consensus.cc:2368] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:50.228078 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.315012 10052 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:17:50.229609 10131 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.235399 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.316234 10050 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:17:50.232060 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 289336us
I20260709 13:17:50.252976 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.317369 10049 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:50.229316 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.319332 10054 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:17:50.231848 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.228150 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:50.321451 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.219105 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:50.242540 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45237.1us
I20260709 13:17:50.194293 13453 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.324084 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.324633 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:50.326112 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.327128 10128 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:50.331367 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.331997 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.333223 10127 raft_consensus.cc:2368] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:50.335717 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.336862 10125 raft_consensus.cc:2376] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8. Re-sending same reply.
I20260709 13:17:50.252445 13383 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:50.265985 13337 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:50.338197 13475 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:50.338569 13337 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.339155 13337 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:50.339558 13123 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:50.340108 13461 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:50.340559 13461 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:50.341576 13421 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.341739 13450 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.142s
I20260709 13:17:50.343487 13353 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.343748 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.344738 10047 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:50.345769 13231 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.346251 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.347365 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:50.347716 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:50.243247 13320 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:50.348369 10048 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:50.349136 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.350605 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.265475 13413 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.351292 10053 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:50.353044 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:50.338954 13475 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.263299 13351 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:50.351611 13413 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.520812 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:50.521966 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 169024us
I20260709 13:17:50.466984 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.524353 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:50.245466 13369 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.481832 10899 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.520802 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:50.353443 10047 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:50.381865 10050 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.524883 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:50.537748 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:50.404155 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:50.380122 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.541622 10055 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:50.532594 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.253465 13345 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:50.537343 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:50.382750 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.365533 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:50.453755 10129 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:17:50.542670 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.402277 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.543488 10050 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:50.360721 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:50.545892 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:50.399135 13467 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:17:50.345793 13364 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.722s	user 0.018s	sys 0.013s
W20260709 13:17:50.491367 13420 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.933s	user 0.015s	sys 0.001s
I20260709 13:17:50.548791 10128 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:50.547447 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.543488 10125 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:50.352478 13356 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:50.557132 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.333966 13435 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.546356 10049 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:50.563911 10048 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 5.
I20260709 13:17:50.564524 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:50.349133 13472 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:50.479439 13352 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.412158 13362 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:50.570729 10053 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:50.411795 13409 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.379386 13419 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:50.410845 13434 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:50.576391 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.016s	sys 0.067s
W20260709 13:17:50.592985 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.016s	sys 0.067s
W20260709 13:17:50.593362 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 719233us
I20260709 13:17:50.574448 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:50.408389 13394 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.581743 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.595225 10048 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.595626 10019 leader_election.cc:400] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:50.567302 10050 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:50.582469 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:50.377195 13464 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:50.565455 13351 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:50.601290 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:17:50.601904 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.589402 10128 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:17:50.589529 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51863.5us
I20260709 13:17:50.598662 13410 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.599473 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:50.544730 13468 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:50.546710 13414 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:50.364764 13422 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.608697 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:50.608835 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.612146 10053 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.349210 13320 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.555984 13345 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.628386 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:50.685328 10051 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:50.661651 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:50.686002 10198 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.660871 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:50.665645 13292 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.573834 13364 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.600126 13411 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.701684 10051 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:50.600782 13419 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:49.942168 13218 ts_tablet_manager.cc:1469] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Trace:
0709 13:17:42.593114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:42.593290 (+   176us) service_pool.cc:224] Handling call
0709 13:17:43.481609 (+888319us) ts_tablet_manager.cc:934] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
0709 13:17:46.127852 (+2646243us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:46.548023 (+420171us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:46.548203 (+   180us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:46.833612 (+285409us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:46.856424 (+ 22812us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:46.867254 (+ 10830us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:46.867488 (+   234us) inbound_call.cc:177] Queueing success response
0709 13:17:48.209435 (+1341947us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:49.282843 (+1073408us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:49.283541 (+   698us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:49.348445 (+ 64904us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":50851,"mutex_wait_us":75419,"raft.queue_time_us":27958,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7843,"tablet-copy.queue_time_us":744898,"tablet-download-pool.queue_time_us":194,"tablet-download-pool.run_cpu_time_us":1542,"tablet-download-pool.run_wall_time_us":326428,"thread_start_us":1151032,"threads_started":2}
I20260709 13:17:50.703030 10895 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:17:50.687296 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.568727 13424 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:17:50.578964 13420 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.583729 13435 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:50.341611 13461 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.611198 13464 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.795030 10054 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:50.707803 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:50.771807 10052 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:50.903404 10896 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:50.901396 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 292592us
I20260709 13:17:50.855223 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 264) took 1184 ms. Trace:
I20260709 13:17:50.907230 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1652) took 1783 ms. Trace:
I20260709 13:17:50.907827 10205 rpcz_store.cc:276] 0709 13:17:49.123989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.124232 (+   243us) service_pool.cc:224] Handling call
0709 13:17:50.906801 (+1782569us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54001fed54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:50.907059 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3762813312}
W20260709 13:17:50.900825 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 240058us
W20260709 13:17:50.771852 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:50.909865 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:50.899981 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 291340us
I20260709 13:17:50.912057 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1537) took 1282 ms. Trace:
I20260709 13:17:50.904242 10128 rpcz_store.cc:276] 0709 13:17:49.671061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.603580 (+932519us) service_pool.cc:224] Handling call
0709 13:17:50.604070 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.604379 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:50.617450 (+ 13071us) log.cc:844] Serialized 14 byte log entry
0709 13:17:50.852098 (+234648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:50.852187 (+    89us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:50.852259 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.852359 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:50.855098 (+  2739us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12147,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":148,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":144,"thread_start_us":234003,"threads_started":1}
I20260709 13:17:50.913560 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 286) took 1381 ms. Trace:
I20260709 13:17:50.914096 10056 rpcz_store.cc:276] 0709 13:17:49.531695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.532462 (+   767us) service_pool.cc:224] Handling call
0709 13:17:50.913262 (+1380800us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:50.913430 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450342784}
I20260709 13:17:50.918195 13410 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:50.920370 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:50.921447 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:50.923035 13480 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:50.923964 13480 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:50.912729 10131 rpcz_store.cc:276] 0709 13:17:49.629268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.629532 (+   264us) service_pool.cc:224] Handling call
0709 13:17:49.629883 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:50.319512 (+689629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.911664 (+592152us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:50.911961 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378652928}
I20260709 13:17:50.925344 13480 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:50.934644 13410 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.935555 13410 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:50.936211 13410 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:50.943953 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:50.911990 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:50.948266 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.955215 13292 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:17:50.958474 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.964596 13292 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.971024 10019 leader_election.cc:400] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:17:50.973658 10019 leader_election.cc:403] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:17:50.977128 13292 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
W20260709 13:17:50.978724 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:50.989265 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 302113us
W20260709 13:17:50.989449 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77607.4us
I20260709 13:17:50.707753 13468 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:50.708012 13353 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:50.988936 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:50.826167 13421 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:50.995672 10894 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:50.736132 13320 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:50.999900 10054 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:50.687305 13351 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:50.767518 13380 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:50.756918 13172 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:50.804979 13481 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:50.988605 13411 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.984546 13356 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:50.994385 13362 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.001272 13477 raft_consensus.cc:2764] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:17:51.001372 13422 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.000721 13458 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:51.006026 13394 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.060652 13469 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:51.011256 13414 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.002625 13353 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:50.736209 13231 consensus_queue.cc:1175] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d51ad94710c348889d417f7f00032948" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 8 index: 9 } }
I20260709 13:17:51.042657 13409 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.050751 13172 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.001471 13473 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.083448 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:51.087381 13472 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.088439 13464 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.089658 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1541) took 1455 ms. Trace:
I20260709 13:17:51.091495 13123 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.061880 13351 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.098639 10050 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:51.099768 13364 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:51.100049 13084 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.000s
I20260709 13:17:51.138643 10126 rpcz_store.cc:276] 0709 13:17:49.634711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.634852 (+   141us) service_pool.cc:224] Handling call
0709 13:17:51.089213 (+1454361us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54002ef254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:51.089543 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837878144}
I20260709 13:17:51.101660 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1547) took 1447 ms. Trace:
I20260709 13:17:51.038880 13450 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:51.194713 10124 rpcz_store.cc:276] 0709 13:17:49.654645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.406739 (+752094us) service_pool.cc:224] Handling call
0709 13:17:50.412944 (+  6205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.413485 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.413527 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.543091 (+129564us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.809964 (+266873us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:50.810011 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.810095 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.101404 (+291309us) raft_consensus.cc:1685] finished
0709 13:17:51.101425 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.101595 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10298,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":31806,"thread_start_us":215028,"threads_started":1,"wal-append.queue_time_us":435561}
I20260709 13:17:51.110038 10019 leader_election.cc:400] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:51.199375 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.202943 10019 leader_election.cc:403] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:17:51.108359 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:51.116226 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 266) took 1444 ms (client timeout 1721 ms). Trace:
I20260709 13:17:51.057278 13292 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:51.071151 13431 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:51.122422 13414 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.219982 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.220386 13410 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.221427 13414 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:51.134047 10055 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.000s
W20260709 13:17:51.137163 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53778.3us
I20260709 13:17:51.223385 10055 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:17:51.223954 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:51.224495 13435 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.224634 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:51.225268 13420 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.225648 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:51.226002 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:51.226584 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.124527 13353 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:51.227329 13422 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:51.227639 10052 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:17:51.227792 13422 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.228312 13422 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:51.230672 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.223409 10894 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:51.220082 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.226764 10055 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:51.226996 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:51.233774 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:51.102561 13413 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.080698 13468 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.106845 13398 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.106602 13352 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.063006 13411 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:51.192680 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2783) took 1430 ms (client timeout 1509 ms). Trace:
I20260709 13:17:51.062649 13419 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.194896 10205 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:17:51.195732 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.198112 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:51.280277 10130 rpcz_store.cc:269] 0709 13:17:49.762248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.871161 (+108913us) service_pool.cc:224] Handling call
0709 13:17:51.192494 (+1321333us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:51.192616 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756863744}
I20260709 13:17:51.281497 10048 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:51.281665 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.198616 13462 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:51.199321 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.288514 10202 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:51.199625 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.200029 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.073879 13362 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.116634 13380 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:17:51.138140 13475 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.295428 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.295629 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.203086 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.296096 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.203776 10122 rpcz_store.cc:269] 0709 13:17:49.671723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.970014 (+298291us) service_pool.cc:224] Handling call
0709 13:17:51.115980 (+1145966us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54001c6e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:51.116132 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108358912}
I20260709 13:17:51.126554 13469 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:17:51.206326 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130594us
I20260709 13:17:51.297423 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:51.204183 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 96395.1us
I20260709 13:17:51.298041 10122 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:51.211918 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.214787 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104852us
W20260709 13:17:51.130832 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44531.7us
I20260709 13:17:51.299310 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:51.299606 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:51.223153 13503 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:17:51.237485 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.307992 10126 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:51.220386 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:51.240243 13496 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:17:51.182865 13345 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.244365 13414 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:51.245551 13369 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.688s	user 0.009s	sys 0.004s
W20260709 13:17:51.248494 13482 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:17:51.345463 13369 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.139360 13409 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.232008 13292 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.315061 13355 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:17:51.347962 13369 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.182050 13464 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.251573 10050 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:51.251904 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 145593us
I20260709 13:17:51.175451 13472 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.253237 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:51.253609 13431 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.255613 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:51.168708 13473 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.168247 13477 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 13:17:51.214481 13450 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.209756 13084 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.257810 13433 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:51.251271 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1551) took 1592 ms. Trace:
I20260709 13:17:51.262975 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:51.252408 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2791) took 1385 ms. Trace:
I20260709 13:17:51.210986 13365 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.250492 13364 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10.
I20260709 13:17:51.283356 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.336424 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.289466 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161615us
I20260709 13:17:51.192015 13491 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:51.203228 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.313982 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.316597 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.320079 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93153.6us
I20260709 13:17:51.324555 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-cebbf225b866442ab895577b0496de1b received from {username='slave'} at 127.0.0.1:50726
I20260709 13:17:51.333897 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.390179 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.386948 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:51.340361 13337 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.001s	user 0.011s	sys 0.002s
I20260709 13:17:51.213753 13480 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:17:51.344379 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.281037 13413 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:51.353559 10200 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:17:51.357386 13505 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:17:51.405441 10203 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:51.354112 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53450.3us
I20260709 13:17:51.406613 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:51.406873 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:51.384768 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:51.409005 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:51.384212 10124 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.197402 13123 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.385460 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:51.387863 10205 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:51.389816 10048 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:51.390486 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1549) took 1735 ms. Trace:
I20260709 13:17:51.338513 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:51.386541 10128 rpcz_store.cc:276] 0709 13:17:49.867188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.913502 (+1046314us) service_pool.cc:224] Handling call
0709 13:17:50.913932 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.914193 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.914218 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.914772 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.045321 (+130549us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:51.062463 (+ 17142us) log.cc:844] Serialized 14 byte log entry
0709 13:17:51.063393 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.063520 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.243198 (+179678us) raft_consensus.cc:1685] finished
0709 13:17:51.243224 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.252329 (+  9105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14734,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":414,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":15545,"thread_start_us":130083,"threads_started":1,"wal-append.queue_time_us":316596}
I20260709 13:17:51.429320 10122 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:51.430258 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:51.430923 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.431294 10128 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:17:51.432804 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:51.395444 10055 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:51.434996 10055 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:51.398846 10894 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:51.291810 13410 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:51.381021 10123 rpcz_store.cc:276] 0709 13:17:49.658453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.575188 (+916735us) service_pool.cc:224] Handling call
0709 13:17:50.600023 (+ 24835us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:50.798560 (+198537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.798597 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:50.849437 (+ 50840us) log.cc:844] Serialized 52 byte log entry
0709 13:17:51.005283 (+155846us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:17:51.005325 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.005404 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.250977 (+245573us) raft_consensus.cc:1685] finished
0709 13:17:51.251001 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.251201 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21631,"raft.queue_time_us":5312,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":47880,"thread_start_us":350256,"threads_started":2,"wal-append.queue_time_us":355863}
I20260709 13:17:51.350225 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.349074 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2795) took 1480 ms. Trace:
W20260709 13:17:51.352890 13418 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.033s	user 0.010s	sys 0.004s
W20260709 13:17:51.424417 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66201.1us
I20260709 13:17:51.410285 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:51.308426 13420 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:51.429466 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.356508 13362 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.401160 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-cebbf225b866442ab895577b0496de1b on tablet cebbf225b866442ab895577b0496de1b with peer db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:51.449208 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.450574 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66082.5us
I20260709 13:17:51.450990 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:51.449273 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:51.428850 10127 rpcz_store.cc:276] 0709 13:17:49.655369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.558237 (+902868us) service_pool.cc:224] Handling call
0709 13:17:50.558665 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.591757 (+ 33092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.591796 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.592306 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.199881 (+607575us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:17:51.207506 (+  7625us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:51.207575 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.207685 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.358694 (+151009us) raft_consensus.cc:1685] finished
0709 13:17:51.358733 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.390423 (+ 31690us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32593,"raft.queue_time_us":41301,"raft.run_cpu_time_us":3015,"raft.run_wall_time_us":594471,"spinlock_wait_cycles":11015936,"thread_start_us":470770,"threads_started":1,"wal-append.queue_time_us":650800}
I20260709 13:17:51.358395 13475 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.389654 13435 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.349438 13292 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:51.350116 13409 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:51.205586 13356 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.457218 10122 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:51.485517 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79036.7us
I20260709 13:17:51.354485 13422 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:51.460254 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 73376.6us
I20260709 13:17:51.293897 13398 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.295795 13352 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.461244 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:51.390074 13345 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:51.454624 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 154983us
I20260709 13:17:51.473105 13362 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.474694 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:51.406054 13337 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.479879 10130 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:51.486707 13413 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:51.486516 10019 leader_election.cc:400] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
I20260709 13:17:51.439309 10131 rpcz_store.cc:276] 0709 13:17:49.868567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.926451 (+1057884us) service_pool.cc:224] Handling call
0709 13:17:50.927066 (+   615us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:50.928065 (+   999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.928093 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:50.928783 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 13:17:51.078520 (+149737us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:51.078579 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.078680 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.348809 (+270129us) raft_consensus.cc:1685] finished
0709 13:17:51.348832 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.349005 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":178,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":148305,"threads_started":1,"wal-append.queue_time_us":198837}
I20260709 13:17:51.542553 10019 leader_election.cc:403] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:51.489775 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1676) took 1149 ms. Trace:
I20260709 13:17:51.543558 10047 rpcz_store.cc:276] 0709 13:17:50.340160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.345676 (+  5516us) service_pool.cc:224] Handling call
0709 13:17:51.489450 (+1143774us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b5400369054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:51.489678 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13906304}
I20260709 13:17:51.288230 13468 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.493077 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:51.300196 13462 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.472851 13364 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:51.467263 13394 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:51.472446 13477 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.402963 13464 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.473404 10124 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:51.523825 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1548) took 1868 ms. Trace:
I20260709 13:17:51.478556 10126 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:51.620714 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:17:51.621608 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 276) took 1097 ms. Trace:
I20260709 13:17:51.483747 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:51.536374 10122 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:51.622045 10126 rpcz_store.cc:276] 0709 13:17:50.523690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.619877 (+1096187us) service_pool.cc:224] Handling call
0709 13:17:51.620374 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.621250 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.621530 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:51.490631 10049 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:51.536834 13345 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.643095 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:51.645076 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 158685us
I20260709 13:17:51.646392 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:51.585078 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:51.649124 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64134us
I20260709 13:17:51.650313 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:51.583673 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109127us
W20260709 13:17:51.562260 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:51.582763 13437 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:51.516376 13491 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:51.516062 13172 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:51.593030 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 71555.9us
I20260709 13:17:51.592962 10196 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:51.350075 13469 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:51.482257 13475 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.603035 10130 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:51.485055 13360 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:17:51.486515 13435 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:51.614836 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 146059us
I20260709 13:17:51.551908 13380 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:51.541240 13413 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.015894 13413 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.016075 13516 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:52.016979 13516 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.616997 10199 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:51.618745 10129 rpcz_store.cc:276] 0709 13:17:49.654999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.557682 (+902683us) service_pool.cc:224] Handling call
0709 13:17:50.558087 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.581207 (+ 23120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.581241 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.610328 (+ 29087us) log.cc:844] Serialized 26 byte log entry
0709 13:17:50.853208 (+242880us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:50.853262 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:50.853348 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:51.523430 (+670082us) raft_consensus.cc:1685] finished
0709 13:17:51.523467 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:51.523720 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23193,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":20946,"thread_start_us":122526,"threads_started":1,"wal-append.queue_time_us":394057}
I20260709 13:17:51.552420 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:51.583382 13452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.006s	sys 0.032s
I20260709 13:17:51.643942 10202 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:51.806599 13418 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:51.667315 13437 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:51.642519 13292 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7.
W20260709 13:17:52.031291 13509 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.312s	user 0.000s	sys 0.002s
I20260709 13:17:51.560384 13427 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.216s
I20260709 13:17:52.032627 13427 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:52.033084 13427 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:51.777096 13451 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.375s	user 0.008s	sys 0.009s
I20260709 13:17:52.035526 13451 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.036677 13435 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:51.612720 13502 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:17:51.541877 13362 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:51.616715 13477 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:51.642764 13518 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:51.599606 13364 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:51.599263 13433 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:51.583775 13493 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.385s
I20260709 13:17:51.522398 13414 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:17:52.018095 13472 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:52.063089 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 412875us
I20260709 13:17:52.015937 13380 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:52.042228 13231 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.025621 13172 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:17:52.056192 13477 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.798179 13480 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.142s
W20260709 13:17:51.818513 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 357392us
I20260709 13:17:51.939687 13411 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.011772 13428 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:52.013096 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 289) took 1071 ms. Trace:
I20260709 13:17:51.961741 13491 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:51.840442 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 194177us
W20260709 13:17:52.040067 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.059s	sys 0.135s
I20260709 13:17:52.044518 10129 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:52.178610 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2799) took 1170 ms. Trace:
I20260709 13:17:52.179255 10129 rpcz_store.cc:276] 0709 13:17:51.008785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.018698 (+1009913us) service_pool.cc:224] Handling call
0709 13:17:52.044226 (+ 25528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.178196 (+133970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.178527 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.177377 10052 rpcz_store.cc:276] 0709 13:17:50.941327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.987704 (+ 46377us) service_pool.cc:224] Handling call
0709 13:17:51.227368 (+239664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.228139 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.012803 (+784664us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400136d50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:52.013008 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549910784}
I20260709 13:17:51.859614 13512 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.390s
I20260709 13:17:52.052125 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1631) took 1501 ms. Trace:
W20260709 13:17:52.021472 13452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.332s	user 0.007s	sys 0.034s
I20260709 13:17:52.213603 10051 rpcz_store.cc:276] 0709 13:17:50.536770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.900631 (+363861us) service_pool.cc:224] Handling call
0709 13:17:50.992643 (+ 92012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.014794 (+ 22151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.014850 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:51.015573 (+   723us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.185316 (+169743us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:51.185371 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.185444 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:52.037409 (+851965us) raft_consensus.cc:1685] finished
0709 13:17:52.037429 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.037581 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5646,"raft.queue_time_us":11105,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":10489,"thread_start_us":108503,"threads_started":1,"wal-append.queue_time_us":266205}
I20260709 13:17:52.059527 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.007586 13084 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:52.034917 13475 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:51.899662 10126 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:17:52.217826 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 277) took 1693 ms. Trace:
I20260709 13:17:52.218328 10126 rpcz_store.cc:276] 0709 13:17:50.523977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.627059 (+1103082us) service_pool.cc:224] Handling call
0709 13:17:51.848956 (+221897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.217451 (+368495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.217739 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.029721 13431 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.022086 13450 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.038709 13437 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:51.861696 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.034053 13427 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.056493 13518 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.057336 13337 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:52.084903 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:52.177942 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.059s	sys 0.135s
I20260709 13:17:52.017536 13516 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.178110 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:52.064457 13398 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.082298 13472 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.083742 13352 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:52.225885 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.030s	sys 0.087s
I20260709 13:17:52.081930 13473 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.305786 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.311208 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1640) took 1045 ms. Trace:
I20260709 13:17:52.176945 13418 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.312844 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:52.350558 10051 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:52.274562 10129 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:52.085716 13172 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:52.177219 13428 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:52.315651 10052 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:52.084905 13525 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.253s
I20260709 13:17:52.212777 13435 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:52.234143 13514 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.622s	user 0.000s	sys 0.002s
I20260709 13:17:52.212451 13491 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.226825 13123 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.086654 13231 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:52.337754 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.031s	sys 0.087s
I20260709 13:17:52.341409 13520 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:52.341593 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:52.342594 13515 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.523s	user 0.000s	sys 0.002s
I20260709 13:17:52.324514 13394 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.107944 13522 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:52.346204 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 261380us
I20260709 13:17:52.347754 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:52.344218 13513 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:17:52.352566 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1552) took 1331 ms. Trace:
I20260709 13:17:52.402832 10129 rpcz_store.cc:276] 0709 13:17:51.021432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.181434 (+1160002us) service_pool.cc:224] Handling call
0709 13:17:52.274354 (+ 92920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.352138 (+ 77784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.352481 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.357579 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1641) took 1091 ms. Trace:
I20260709 13:17:52.318094 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1553) took 1296 ms. Trace:
I20260709 13:17:52.404512 10052 rpcz_store.cc:276] 0709 13:17:51.265814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.181901 (+916087us) service_pool.cc:224] Handling call
0709 13:17:52.314904 (+133003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.357184 (+ 42280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.357489 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.404603 10126 rpcz_store.cc:276] 0709 13:17:51.021883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.231041 (+1209158us) service_pool.cc:224] Handling call
0709 13:17:52.317275 (+ 86234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.317568 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.317611 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.317678 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:52.317727 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.317812 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.317997 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.406539 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:52.320827 13411 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.407440 13411 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.324323 13451 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.323724 13518 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.272595 13526 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 13:17:52.380427 13427 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:52.392103 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.223829 13364 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.424814 13525 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:52.425781 13525 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:52.425011 13353 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.197s	user 0.012s	sys 0.000s
I20260709 13:17:52.426810 13353 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.355803 13398 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:52.430655 13398 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:52.416014 13433 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:52.431789 13519 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.434s	user 0.000s	sys 0.002s
I20260709 13:17:52.407699 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.433029 13521 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.158s
I20260709 13:17:52.434028 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:52.435061 10198 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:52.364013 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:52.436918 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.409024 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1556) took 1386 ms. Trace:
W20260709 13:17:52.438210 13458 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.329s	user 0.013s	sys 0.000s
I20260709 13:17:52.382107 13452 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:17:52.438251 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:52.444545 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44717.6us
W20260709 13:17:52.413959 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.007s	sys 0.017s
I20260709 13:17:52.366568 13468 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.345333 13472 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.368263 13345 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:52.388808 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 210792us
I20260709 13:17:52.381567 13464 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.422245 10129 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:52.356278 13380 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.362941 13462 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.438009 13529 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:52.413479 10131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.013s
I20260709 13:17:52.355389 13473 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.437703 10126 rpcz_store.cc:276] 0709 13:17:51.023070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.406423 (+1383353us) service_pool.cc:224] Handling call
0709 13:17:52.406808 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.407052 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.407081 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.408670 (+  1589us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:52.408725 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.408796 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.408955 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:52.446177 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.007s	sys 0.017s
W20260709 13:17:52.446579 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.010s	sys 0.025s
W20260709 13:17:52.473254 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.011s	sys 0.025s
I20260709 13:17:52.431957 13433 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.379166 13435 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.456050 13533 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:52.457031 13362 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.456895 13523 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
W20260709 13:17:52.443192 13517 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:17:52.455725 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1555) took 1432 ms. Trace:
I20260709 13:17:52.491490 10129 rpcz_store.cc:276] 0709 13:17:51.022660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.409573 (+1386913us) service_pool.cc:224] Handling call
0709 13:17:52.421871 (+ 12298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.454725 (+ 32854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.455046 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.475670 13345 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.454704 13337 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:17:52.466140 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1669) took 1238 ms. Trace:
I20260709 13:17:52.496898 10199 rpcz_store.cc:276] 0709 13:17:51.227907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.295093 (+ 67186us) service_pool.cc:224] Handling call
0709 13:17:52.465889 (+1170796us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400354654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:52.466067 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736939904}
W20260709 13:17:52.468472 10131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.006s	sys 0.013s
I20260709 13:17:52.361245 13418 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.467479 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:52.430779 13353 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:52.476382 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.637s (SENT) [suppressed 29 similar messages]
I20260709 13:17:52.404042 13394 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.465919 13360 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.475782 13452 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:52.447495 13458 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:52.483404 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.499909 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1559) took 1475 ms. Trace:
I20260709 13:17:52.527917 10126 rpcz_store.cc:276] 0709 13:17:51.024152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.477564 (+1453412us) service_pool.cc:224] Handling call
0709 13:17:52.498985 (+ 21421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.499320 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.499365 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.499433 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:52.499487 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.499580 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.499801 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.448694 13525 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.480086 13231 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.517998 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:52.518512 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64497.5us
I20260709 13:17:52.534014 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:52.480397 13421 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.465421 13428 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.480669 13529 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:52.524246 13478 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.998s	user 0.008s	sys 0.009s
I20260709 13:17:52.535873 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:52.552368 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:52.552893 10898 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:52.535923 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:52.537269 13472 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.530831 13360 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.431906 13398 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.538275 13452 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:52.543769 13497 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.483s	user 0.002s	sys 0.002s
I20260709 13:17:52.544610 13418 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.530498 13480 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:52.548486 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1634) took 2010 ms. Trace:
I20260709 13:17:52.486430 13516 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.555145 10900 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:17:52.556654 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:52.557006 13538 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:52.553424 10896 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:52.598686 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:52.619182 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:17:52.546675 13351 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.637693 13351 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.639266 13360 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.557950 13353 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:52.640806 13351 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.492936 13345 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.644239 13345 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.477197 13473 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.647236 13411 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.541057 13421 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:52.633659 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77105.7us
I20260709 13:17:52.649602 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:52.636384 13475 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.637995 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:52.625082 10054 rpcz_store.cc:276] 0709 13:17:50.538043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.928499 (+390456us) service_pool.cc:224] Handling call
0709 13:17:50.928809 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.013561 (+ 84752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.548086 (+1534525us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54002350d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:52.548396 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162980224}
I20260709 13:17:52.616782 13525 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:52.655790 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:52.656935 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:52.666687 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:52.666996 10051 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:52.657966 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.661162 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 263) took 2990 ms. Trace:
I20260709 13:17:52.668412 10125 rpcz_store.cc:276] 0709 13:17:49.670804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.600910 (+930106us) service_pool.cc:224] Handling call
0709 13:17:50.606528 (+  5618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:50.656775 (+ 50247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:50.657514 (+   739us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:50.685409 (+ 27895us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.012045 (+326636us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:51.012107 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.012190 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:52.542933 (+1530743us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:52.660646 (+117713us) raft_consensus.cc:1685] finished
0709 13:17:52.660682 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.661051 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24378240,"thread_start_us":254163,"threads_started":1,"wal-append.queue_time_us":355191}
I20260709 13:17:52.670758 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:52.676991 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:52.679782 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.681988 13529 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.682289 13352 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:17:52.673133 13413 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.657s	user 0.018s	sys 0.000s
I20260709 13:17:52.634544 13418 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:52.551235 13356 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 13:17:52.624734 13480 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:52.692098 13292 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:52.690546 10199 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:52.709189 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.710929 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:52.685731 13353 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:52.664538 13538 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.647823 13473 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.670589 13360 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:17:52.763808 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:52.670169 13512 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:52.768378 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2798) took 2899 ms. Trace:
I20260709 13:17:52.672008 13475 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.784730 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:52.758587 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1558) took 1734 ms. Trace:
I20260709 13:17:52.770082 10199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:52.787086 10199 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:52.788062 10199 raft_consensus.cc:614] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:52.786648 10127 rpcz_store.cc:276] 0709 13:17:51.023836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.461236 (+1437400us) service_pool.cc:224] Handling call
0709 13:17:52.461795 (+   559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.462137 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:52.505953 (+ 43816us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.757992 (+252039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.758080 (+    88us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:52.758155 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.758263 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.758484 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153349,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":202,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":6062,"thread_start_us":281562,"threads_started":2}
W20260709 13:17:52.777941 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 112786us
I20260709 13:17:52.789276 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1643) took 1523 ms. Trace:
I20260709 13:17:52.789897 10051 rpcz_store.cc:276] 0709 13:17:51.267314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.788338 (+1521024us) service_pool.cc:224] Handling call
0709 13:17:52.788754 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.788951 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.788984 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.789025 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:52.789063 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.789131 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.789223 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.782598 13480 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:52.772938 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.704253 13462 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:17:52.797099 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 134601us
I20260709 13:17:52.713490 13292 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.763460 13353 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:52.802907 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:52.804251 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 327959us
I20260709 13:17:52.522004 13533 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.806335 10129 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:52.713001 13518 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:52.811383 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:52.789480 13473 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.704532 13413 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.810109 10051 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:52.771441 13538 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:52.784649 13512 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.791919 13355 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:17:52.857252 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46384.5us
I20260709 13:17:52.856259 13411 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.791739 13351 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:52.753966 13437 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:52.808339 10096 connection.cc:256] RPC call timeout handler was delayed by -1.09850610400008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 13:17:52.811872 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 319) took 1488 ms. Trace:
I20260709 13:17:52.753453 13542 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:52.862998 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1644) took 1595 ms. Trace:
I20260709 13:17:52.857833 13480 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:52.817385 13353 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.809544 13292 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:17:52.862104 13540 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:17:52.919919 10199 rpcz_store.cc:276] 0709 13:17:51.323549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.803075 (+1479526us) service_pool.cc:224] Handling call
0709 13:17:52.803683 (+   608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.811315 (+  7632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.811368 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.811435 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:52.811508 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.811601 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.811783 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.975049 10051 rpcz_store.cc:276] 0709 13:17:51.267612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.790807 (+1523195us) service_pool.cc:224] Handling call
0709 13:17:52.809887 (+ 19080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.862561 (+ 52674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.862905 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.926051 10125 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:52.905745 13512 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:52.990955 10201 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:52.902449 13355 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:53.001461 13453 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.004781 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:52.859902 13458 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:17:52.828671 13462 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:52.921851 13516 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.874441 13473 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.018105 10123 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:52.921738 13345 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:52.978734 13351 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:52.983289 13411 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.036638 10123 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:53.037384 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1564) took 1993 ms. Trace:
I20260709 13:17:53.037719 10123 rpcz_store.cc:276] 0709 13:17:51.043967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.033315 (+1989348us) service_pool.cc:224] Handling call
0709 13:17:53.036451 (+  3136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.037114 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.037319 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:52.936754 13422 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.922101 13472 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.039417 13394 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:52.922962 13380 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:53.040613 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 581) took 2379 ms. Trace:
I20260709 13:17:53.041059 13351 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.040735 13380 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.034353 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:53.034632 10051 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:53.045454 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1645) took 1774 ms. Trace:
I20260709 13:17:53.045904 10051 rpcz_store.cc:276] 0709 13:17:51.271533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.993756 (+1722223us) service_pool.cc:224] Handling call
0709 13:17:53.034436 (+ 40680us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.045145 (+ 10709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.045391 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.028350 13435 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:52.922437 13529 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:53.050310 10123 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:17:53.052533 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.054042 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:53.054443 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2806) took 2010 ms. Trace:
I20260709 13:17:53.055083 10123 rpcz_store.cc:276] 0709 13:17:51.044407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.038553 (+1994146us) service_pool.cc:224] Handling call
0709 13:17:53.038913 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.053295 (+ 14382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.054355 (+  1060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.016994 13355 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.058831 13351 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.037397 13411 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.039435 13472 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.061066 10051 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:53.177220 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1646) took 1901 ms. Trace:
I20260709 13:17:53.177850 10051 rpcz_store.cc:276] 0709 13:17:51.275549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.046790 (+1771241us) service_pool.cc:224] Handling call
0709 13:17:53.060718 (+ 13928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.176775 (+116057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.177128 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.094378 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:53.118104 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2804) took 2107 ms (client timeout 1752 ms). Trace:
W20260709 13:17:53.182610 10130 rpcz_store.cc:269] 0709 13:17:51.010752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.282880 (+272128us) service_pool.cc:224] Handling call
0709 13:17:53.117775 (+1834895us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b540015a754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:53.117982 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212395904}
I20260709 13:17:53.099342 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1565) took 2047 ms. Trace:
I20260709 13:17:53.185957 10123 rpcz_store.cc:276] 0709 13:17:51.056663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.056178 (+1999515us) service_pool.cc:224] Handling call
0709 13:17:53.098499 (+ 42321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.098802 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.098845 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.098918 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:53.098975 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.099035 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.099224 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.023916 13542 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:53.041091 10899 rpcz_store.cc:276] 0709 13:17:50.661250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.662138 (+   888us) service_pool.cc:224] Handling call
0709 13:17:51.192969 (+530831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.193168 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.193193 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:51.193242 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:51.193281 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.193336 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.040351 (+1847015us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b5400662e50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:53.040503 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220096384}
I20260709 13:17:53.062601 13231 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:53.202495 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:53.107590 13546 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.204s	user 0.000s	sys 0.002s
I20260709 13:17:53.131549 13469 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.081161 13529 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.231659 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1647) took 1955 ms. Trace:
I20260709 13:17:53.232381 10051 rpcz_store.cc:276] 0709 13:17:51.275824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.189336 (+1913512us) service_pool.cc:224] Handling call
0709 13:17:53.230891 (+ 41555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.231146 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.231184 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.231254 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:53.231306 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.231387 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.231550 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.113368 13422 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:53.192672 13411 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:53.049896 13345 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.034205 13516 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.240839 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 278) took 2159 ms. Trace:
I20260709 13:17:53.248683 10123 rpcz_store.cc:276] 0709 13:17:51.081493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.222728 (+2141235us) service_pool.cc:224] Handling call
0709 13:17:53.239779 (+ 17051us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.239981 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.240009 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.240058 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:53.240093 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.240172 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.240288 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.235731 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:53.240135 13473 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.240058 13422 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.225590 13380 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.264261 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1648) took 1988 ms. Trace:
I20260709 13:17:53.265100 10051 rpcz_store.cc:276] 0709 13:17:51.276065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.234119 (+1958054us) service_pool.cc:224] Handling call
0709 13:17:53.253884 (+ 19765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.263668 (+  9784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.263728 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.263801 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:53.263872 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.263976 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.264171 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1759}
I20260709 13:17:53.227319 13231 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:53.248165 13477 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.023s	user 0.014s	sys 0.001s
I20260709 13:17:53.278720 10123 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:17:53.279826 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 279) took 2198 ms. Trace:
I20260709 13:17:53.280262 10123 rpcz_store.cc:276] 0709 13:17:51.081812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.250327 (+2168515us) service_pool.cc:224] Handling call
0709 13:17:53.278297 (+ 27970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.279410 (+  1113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.279746 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.241093 13518 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.198257 13351 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.241431 13413 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.257447 13435 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.257373 13394 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.268829 13437 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.277238 13231 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:53.324649 10051 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:53.238003 13469 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.239841 13411 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.332090 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 280) took 2250 ms. Trace:
I20260709 13:17:53.342201 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1649) took 2065 ms. Trace:
I20260709 13:17:53.353920 10123 rpcz_store.cc:276] 0709 13:17:51.082096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.281964 (+2199868us) service_pool.cc:224] Handling call
0709 13:17:53.331349 (+ 49385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.331604 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.331657 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.331717 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:53.331764 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.331837 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.332000 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.354027 10051 rpcz_store.cc:276] 0709 13:17:51.277149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.277653 (+2000504us) service_pool.cc:224] Handling call
0709 13:17:53.324368 (+ 46715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.341876 (+ 17508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.342124 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.360954 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 296) took 1121 ms. Trace:
I20260709 13:17:53.361496 10051 rpcz_store.cc:276] 0709 13:17:52.239258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.356511 (+1117253us) service_pool.cc:224] Handling call
0709 13:17:53.360274 (+  3763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.360506 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.360533 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.360584 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:53.360622 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.360684 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.360858 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:53.320501 13345 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.239264 13542 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.377856 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 292) took 2414 ms. Trace:
I20260709 13:17:53.403702 10048 rpcz_store.cc:276] 0709 13:17:50.963481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.282199 (+318718us) service_pool.cc:224] Handling call
0709 13:17:51.306925 (+ 24726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.428679 (+121754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.377347 (+1948668us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54002364d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:53.377730 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123391744}
I20260709 13:17:53.381076 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 315) took 2151 ms. Trace:
I20260709 13:17:53.408087 10201 rpcz_store.cc:276] 0709 13:17:51.229940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.313478 (+ 83538us) service_pool.cc:224] Handling call
0709 13:17:51.343833 (+ 30355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.993507 (+1649674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.380620 (+387113us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:53.380963 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3787379968}
I20260709 13:17:53.438786 13472 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.439775 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:53.442009 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:53.443907 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:53.330539 13394 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:53.446054 13551 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:17:53.445803 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:53.448916 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.010s	sys 0.040s
W20260709 13:17:53.450564 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.010s	sys 0.040s
W20260709 13:17:53.452740 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 762811us
I20260709 13:17:53.455294 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:53.464107 13555 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.465067 13555 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.470118 13555 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.471052 13555 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:53.471719 13555 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:53.473449 10201 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:53.479765 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.023s
W20260709 13:17:53.480403 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.006s	sys 0.023s
I20260709 13:17:53.431247 13533 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.490142 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:53.508051 13555 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.448661 13477 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:53.489704 10199 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.007s	sys 0.024s
W20260709 13:17:53.514127 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.520642 10126 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:17:53.474769 13462 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:53.535985 10199 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.007s	sys 0.024s
W20260709 13:17:53.536316 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.532400 10201 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:17:53.539918 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.541465 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1566) took 2467 ms. Trace:
I20260709 13:17:53.544025 10050 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:53.515034 13351 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:53.533842 13427 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.092s	user 0.012s	sys 0.003s
W20260709 13:17:53.530407 13480 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.008s
I20260709 13:17:53.550484 10202 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:53.552063 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:53.531610 13411 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:53.552392 13353 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.564s	user 0.018s	sys 0.013s
W20260709 13:17:53.572245 10196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1667) took 2339 ms (client timeout 3112 ms). Trace:
W20260709 13:17:53.574688 10196 rpcz_store.cc:269] 0709 13:17:51.226155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.226298 (+   143us) service_pool.cc:224] Handling call
0709 13:17:53.564955 (+2338657us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54002184d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:53.565186 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821738880}
W20260709 13:17:53.577858 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 122760us
I20260709 13:17:53.579649 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:53.557296 13427 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:53.555184 13394 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.557241 13554 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.568179 13469 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:53.587509 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:53.575215 13529 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.594641 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:53.576601 13422 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.581789 13351 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:53.610280 13351 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:53.611187 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1561) took 2574 ms. Trace:
I20260709 13:17:53.611515 13351 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.611835 10126 rpcz_store.cc:276] 0709 13:17:51.036326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.529407 (+1493081us) service_pool.cc:224] Handling call
0709 13:17:52.543626 (+ 14219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.521363 (+977737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.610750 (+ 89387us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54001261d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:53.611062 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237810560}
I20260709 13:17:53.613719 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:53.614533 10126 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:53.592067 13196 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:53.615458 13196 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:53.615983 13537 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:53.616638 13537 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:53.616526 13196 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.617966 13196 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.575575 13353 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.621335 13353 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:53.622409 13531 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.293s	user 0.003s	sys 0.000s
I20260709 13:17:53.622304 13353 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:53.602857 13365 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:53.600085 13411 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:53.609126 13464 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:17:53.579809 13352 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.025s	sys 0.033s
I20260709 13:17:53.623587 10051 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:53.567237 13437 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:17:53.634923 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.016s	sys 0.061s
W20260709 13:17:53.635829 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.016s	sys 0.061s
I20260709 13:17:53.581897 13458 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:17:53.628981 13292 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.002s
I20260709 13:17:53.618007 13537 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:53.632501 13491 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.241s	user 0.006s	sys 0.011s
I20260709 13:17:53.586002 13462 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 13:17:53.634270 13352 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.026s	sys 0.034s
W20260709 13:17:53.634748 13522 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.219s	user 0.004s	sys 0.023s
I20260709 13:17:53.629483 13394 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:53.721794 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 153779us
I20260709 13:17:53.631789 13464 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:53.628542 13411 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:17:53.646034 13452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.012s	sys 0.021s
I20260709 13:17:53.592643 13554 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.746628 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1639) took 2464 ms. Trace:
I20260709 13:17:53.747349 10047 rpcz_store.cc:276] 0709 13:17:51.265484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.545212 (+279728us) service_pool.cc:224] Handling call
0709 13:17:51.628591 (+ 83379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.645703 (+ 17112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.645768 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:51.775531 (+129763us) log.cc:844] Serialized 26 byte log entry
0709 13:17:51.957310 (+181779us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:51.957370 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:51.957461 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.621654 (+1664193us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:53.721155 (+ 99501us) raft_consensus.cc:1685] finished
0709 13:17:53.721190 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.729379 (+  8189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14450,"raft.queue_time_us":34486,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":12163,"spinlock_wait_cycles":245715200,"thread_start_us":26988,"threads_started":1,"wal-append.queue_time_us":267472}
W20260709 13:17:53.750633 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 170517us
I20260709 13:17:53.623601 13353 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.704092 13437 consensus_queue.cc:1243] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:17:53.762894 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0011625949998688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 13:17:53.751005 13452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.013s	sys 0.022s
I20260709 13:17:53.767575 10126 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.724534 13352 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:53.684187 13292 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.773015 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.744256 13411 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.739392 13522 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:53.754967 13561 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.757858 13554 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:53.783715 13536 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:53.723464 13491 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.771365 13537 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.751647 13394 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.779031 13477 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:53.790603 13559 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 13:17:53.796353 13538 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.828s	user 0.038s	sys 0.026s
I20260709 13:17:53.811620 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:17:53.812116 10169 connection.cc:256] RPC call timeout handler was delayed by -2.8124962629999573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:17:53.812901 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:53.813956 10169 connection.cc:256] RPC call timeout handler was delayed by -2.5425175949999357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 13:17:53.814482 10169 connection.cc:256] RPC call timeout handler was delayed by -2.5001962630001344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 13:17:53.804035 13525 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.130s	user 0.024s	sys 0.016s
I20260709 13:17:53.802304 13536 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:53.819802 13236 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.801542 13464 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.843794 13292 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.843535 13343 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:53.844648 13353 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.845007 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:53.847412 13491 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.850196 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 299) took 1329 ms. Trace:
I20260709 13:17:53.821435 13538 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:53.834906 13453 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.051s
I20260709 13:17:53.852389 10198 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
W20260709 13:17:53.880470 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 239) took 2630 ms (client timeout 1765 ms). Trace:
I20260709 13:17:53.879921 10048 rpcz_store.cc:276] 0709 13:17:52.520963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.458443 (+937480us) service_pool.cc:224] Handling call
0709 13:17:53.488145 (+ 29702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.488588 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.488634 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.489343 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.755409 (+266066us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:53.755475 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.755607 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.849938 (+ 94331us) raft_consensus.cc:1685] finished
0709 13:17:53.849962 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.850121 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69088,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":10549,"thread_start_us":232043,"threads_started":1,"wal-append.queue_time_us":296572}
I20260709 13:17:53.798056 13561 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:53.884917 10894 rpcz_store.cc:269] 0709 13:17:51.245957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.250281 (+  4324us) service_pool.cc:224] Handling call
0709 13:17:53.851985 (+2601704us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b540063ef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:53.876271 (+ 24286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40122112}
I20260709 13:17:53.871094 10126 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:53.890597 10897 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:17:53.897607 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:53.891225 13367 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:53.905400 13367 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.881248 13352 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:53.915903 13538 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.834964 13411 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.921288 13411 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:53.921427 13394 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.924717 13561 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:53.877858 13236 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:53.883869 13437 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:17:53.889781 13525 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:53.889582 13553 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:17:53.941250 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1633) took 3403 ms. Trace:
I20260709 13:17:53.950827 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 275) took 3415 ms. Trace:
I20260709 13:17:53.951488 10124 rpcz_store.cc:276] 0709 13:17:50.523386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.617811 (+1094425us) service_pool.cc:224] Handling call
0709 13:17:51.618281 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.955989 (+337708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.956043 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:51.956113 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:51.956164 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.938171 (+1982007us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54003d4854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:53.938254 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.938453 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775932672}
I20260709 13:17:53.941794 10056 rpcz_store.cc:276] 0709 13:17:50.537625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.924021 (+386396us) service_pool.cc:224] Handling call
0709 13:17:50.924463 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.127634 (+203171us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:51.129373 (+  1739us) log.cc:844] Serialized 14 byte log entry
0709 13:17:51.479854 (+350481us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:51.479923 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:51.479973 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.940944 (+2460971us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54000da054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:53.940996 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.941171 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90753,"prepare.queue_time_us":2006,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":8325,"raft.queue_time_us":4072,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":20813,"spinlock_wait_cycles":466577792,"thread_start_us":346381,"threads_started":1,"wal-append.queue_time_us":535489,"wal-append.run_cpu_time_us":4574,"wal-append.run_wall_time_us":1646508}
I20260709 13:17:53.894045 13464 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:53.965657 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.966794 10056 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:53.883574 13336 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
W20260709 13:17:53.951865 13231 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.003s
I20260709 13:17:53.971769 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.976312 10056 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:53.986732 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.989759 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.993577 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:53.995167 10056 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
W20260709 13:17:54.068137 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 189796us
I20260709 13:17:54.073799 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.074676 10056 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:53.985606 13336 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:54.108136 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.109380 10048 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:54.122836 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.124115 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:17:54.067103 13369 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.131950 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.132925 10048 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:53.982075 13231 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.140305 10056 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:54.142252 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.143407 10056 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:54.144901 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.146499 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.083599 13196 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.148204 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.147629 10056 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:54.131850 13369 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:54.162426 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:54.167253 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.168251 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.169113 10056 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:54.148079 13196 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:17:54.175807 13571 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:17:54.175662 13369 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.176719 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1560) took 3141 ms. Trace:
W20260709 13:17:54.170159 13556 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.275s	user 0.001s	sys 0.001s
I20260709 13:17:54.177205 13369 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.177186 10129 rpcz_store.cc:276] 0709 13:17:51.035514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.492778 (+1457264us) service_pool.cc:224] Handling call
0709 13:17:52.493198 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.858955 (+365757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.176426 (+1317471us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54001275d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.176643 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719564032}
I20260709 13:17:54.180269 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:54.172696 13512 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.152s	user 0.010s	sys 0.024s
I20260709 13:17:54.185333 13512 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.186458 13231 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.186697 13428 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:54.172403 13421 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.497s	user 0.035s	sys 0.000s
I20260709 13:17:54.188980 13421 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.195840 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.197314 10052 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:54.200296 10050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 294) took 2701 ms (client timeout 1911 ms). Trace:
W20260709 13:17:54.200922 10050 rpcz_store.cc:269] 0709 13:17:51.494674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.505953 (+ 11279us) service_pool.cc:224] Handling call
0709 13:17:54.195272 (+2689319us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.195471 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3504616192}
W20260709 13:17:54.195052 13355 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.995s	user 0.008s	sys 0.009s
I20260709 13:17:54.192273 13555 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.215441 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.218010 10050 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:17:54.218828 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 285) took 1194 ms. Trace:
I20260709 13:17:54.198378 13538 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.196960 13394 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:54.219596 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.235029 10050 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:54.219338 10126 rpcz_store.cc:276] 0709 13:17:53.024628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.890555 (+865927us) service_pool.cc:224] Handling call
0709 13:17:53.890976 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.891292 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:53.994094 (+102802us) log.cc:844] Serialized 14 byte log entry
0709 13:17:54.218204 (+224110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.218268 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:54.218318 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.218400 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.218576 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7941,"prepare.queue_time_us":114271,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":123,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":321128,"threads_started":2,"wal-append.queue_time_us":223727}
I20260709 13:17:54.219905 13426 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:54.227546 10129 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:54.252072 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:54.252868 10129 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:17:54.253939 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:53.542335 10130 rpcz_store.cc:276] 0709 13:17:51.074505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.197171 (+2122666us) service_pool.cc:224] Handling call
0709 13:17:53.197682 (+   511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.197955 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:53.236153 (+ 38198us) log.cc:844] Serialized 14 byte log entry
0709 13:17:53.541001 (+304848us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.541073 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:53.541128 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.541229 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:53.541393 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38595,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":962,"thread_start_us":304341,"threads_started":1}
I20260709 13:17:54.227797 13411 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:52.784812 10131 rpcz_store.cc:276] 0709 13:17:49.869444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.561352 (+1691908us) service_pool.cc:224] Handling call
0709 13:17:51.614011 (+ 52659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:51.614533 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:51.614583 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:51.634654 (+ 20071us) log.cc:844] Serialized 26 byte log entry
0709 13:17:52.498981 (+864327us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:52.767611 (+268630us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.767970 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.768052 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:52.768124 (+    72us) raft_consensus.cc:1685] finished
0709 13:17:52.768139 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.768305 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":916,"prepare.queue_time_us":142821,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":339518,"raft.run_cpu_time_us":718,"raft.run_wall_time_us":56513,"thread_start_us":853201,"threads_started":2,"wal-append.queue_time_us":732860}
I20260709 13:17:54.255901 13411 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.257550 10131 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:54.228576 13491 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.259595 13394 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.259714 13491 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.239674 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.229254 13522 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:54.263784 10050 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:54.244164 13337 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:54.219956 13555 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:54.248571 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2802) took 3238 ms (client timeout 2087 ms). Trace:
W20260709 13:17:54.270044 10122 rpcz_store.cc:269] 0709 13:17:51.010046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.430590 (+420544us) service_pool.cc:224] Handling call
0709 13:17:54.248268 (+2817678us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54002eccd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.248434 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57975040}
I20260709 13:17:54.249249 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.249651 10127 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:54.260218 13512 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:54.240706 13538 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.258926 10131 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:17:54.277025 10054 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:54.279804 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:54.279947 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.280881 10130 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:54.282071 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:54.285419 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 216128us
I20260709 13:17:54.258345 13411 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.250804 13426 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.287802 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.289284 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:54.289601 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:54.291595 10056 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:54.294202 10054 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:54.294394 10130 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:54.294528 10126 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:54.288215 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.296068 10130 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:54.265069 13421 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.288429 13537 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:17:54.297757 13537 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:54.298158 13537 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.295640 10050 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:17:54.300619 10130 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:54.295492 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:54.269418 13337 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.264812 13236 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:54.296564 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.214692 13355 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.299260 13537 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.305166 13476 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:54.306985 10130 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:54.308733 13394 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.308949 13555 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:54.312064 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.312458 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.317790 10205 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:17:54.317839 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:54.318775 10130 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:54.319001 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.320313 10130 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:54.323253 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.325652 10054 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:54.325397 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:54.302420 13435 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.004s
I20260709 13:17:54.328076 10130 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:17:54.332305 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.332356 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.332722 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.216512 13196 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.334190 10050 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.335011 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.337622 10204 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:54.337476 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:54.276038 13522 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.306981 13538 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:54.342572 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.335244 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.339632 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.341176 10203 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:54.344144 10054 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:54.346159 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.336547 13435 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:54.360896 13355 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:54.363399 13469 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.364148 13469 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.365952 13522 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.366632 13469 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.316099 13476 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.367610 10047 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:54.367962 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:54.366219 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.368629 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.383455 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:54.381696 10054 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:54.385262 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.388087 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.391360 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:54.393795 10054 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:54.379549 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 176821us
I20260709 13:17:54.392411 10131 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:54.367313 13537 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.394847 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1650) took 1869 ms. Trace:
I20260709 13:17:54.380445 13196 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.397948 13576 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:54.401249 10054 rpcz_store.cc:276] 0709 13:17:52.525466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.390757 (+1865291us) service_pool.cc:224] Handling call
0709 13:17:54.391257 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.394462 (+  3205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.394767 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.382930 13476 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.408252 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1654) took 1881 ms. Trace:
I20260709 13:17:54.412637 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.409647 10054 rpcz_store.cc:276] 0709 13:17:52.526846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.406894 (+1880048us) service_pool.cc:224] Handling call
0709 13:17:54.407420 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.407715 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:54.407759 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.407834 (+    75us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:54.407896 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.407984 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.408156 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:54.415835 10095 connection.cc:256] RPC call timeout handler was delayed by -1.7875041309998778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:17:54.417709 10054 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:54.418376 13522 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.418632 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 300) took 1887 ms. Trace:
I20260709 13:17:54.418962 13522 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.419211 10054 rpcz_store.cc:276] 0709 13:17:52.533300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.416987 (+1883687us) service_pool.cc:224] Handling call
0709 13:17:54.417330 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.418302 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.418551 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.420861 10054 raft_consensus.cc:1217] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-8.5]   Dedup: 4.3->[6.4-8.5]
I20260709 13:17:54.421573 13522 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:54.421756 10095 connection.cc:256] RPC call timeout handler was delayed by -1.7076055669999732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2615s for a response
I20260709 13:17:54.421583 10054 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [4.3-8.5]
I20260709 13:17:54.423380 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1655) took 1890 ms. Trace:
I20260709 13:17:54.423992 10054 rpcz_store.cc:276] 0709 13:17:52.532807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.420216 (+1887409us) service_pool.cc:224] Handling call
0709 13:17:54.420648 (+   432us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:17:54.422893 (+  2245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.423286 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.426406 13573 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:54.427544 13573 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:17:54.428082 13573 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:54.428601 13573 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.431834 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.442139 13461 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.450028 13538 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:54.445043 10200 raft_consensus.cc:606] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:54.451658 10200 raft_consensus.cc:614] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:54.445254 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 286) took 1256 ms. Trace:
I20260709 13:17:54.395888 13337 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.452394 10124 rpcz_store.cc:276] 0709 13:17:53.189042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.953511 (+764469us) service_pool.cc:224] Handling call
0709 13:17:53.954101 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.954675 (+   574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.954716 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.963234 (+  8518us) log.cc:844] Serialized 26 byte log entry
0709 13:17:54.316210 (+352976us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:54.316257 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.316339 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.444982 (+128643us) raft_consensus.cc:1685] finished
0709 13:17:54.445004 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.445177 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3374,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":8644,"spinlock_wait_cycles":1579776,"thread_start_us":342573,"threads_started":1,"wal-append.queue_time_us":423246}
I20260709 13:17:54.413615 13491 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.455309 13355 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.461470 13355 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.453557 13337 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.447642 13572 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:17:54.433686 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:54.422339 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2770520329997908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260709 13:17:54.404054 13576 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.468137 10054 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:54.464780 10050 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:17:54.472082 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.472061 10050 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:54.367411 13426 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.483526 10054 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:54.467445 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2603055669999321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31120000000000003s for a response
W20260709 13:17:54.487311 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2127859739998712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
W20260709 13:17:54.488044 10095 connection.cc:256] RPC call timeout handler was delayed by -1.0225041310000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2615s for a response
I20260709 13:17:54.471946 13576 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.430015 13573 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.489022 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:54.479655 13488 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 13:17:54.503178 13437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.013s	sys 0.007s
I20260709 13:17:54.505921 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:54.542232 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.529898 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 298) took 2137 ms. Trace:
W20260709 13:17:54.532975 10865 connection.cc:256] RPC call timeout handler was delayed by -1.685826360999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
I20260709 13:17:54.506634 13491 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:54.572497 13569 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.668s	user 0.004s	sys 0.008s
I20260709 13:17:54.579571 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2766) took 2167 ms. Trace:
I20260709 13:17:54.580250 10199 rpcz_store.cc:276] 0709 13:17:52.412340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.975875 (+563535us) service_pool.cc:224] Handling call
0709 13:17:52.976439 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.536995 (+560556us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:53.777415 (+240420us) log.cc:844] Serialized 14 byte log entry
0709 13:17:53.968584 (+191169us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.969970 (+  1386us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.970127 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:53.970161 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.970230 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:54.571433 (+601203us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:17:54.579087 (+  7654us) raft_consensus.cc:1685] finished
0709 13:17:54.579267 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.579472 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11864,"prepare.queue_time_us":357730,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":140958,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":72236,"spinlock_wait_cycles":99129088,"thread_start_us":938556,"threads_started":4,"wal-append.queue_time_us":111999}
I20260709 13:17:54.584679 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.587056 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.589198 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.591135 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.592980 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.505849 13552 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:17:54.604152 13552 consensus_queue.cc:1243] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:17:54.608381 13516 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.616834 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1626) took 4250 ms. Trace:
I20260709 13:17:54.617461 10049 rpcz_store.cc:276] 0709 13:17:50.366797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.692816 (+326019us) service_pool.cc:224] Handling call
0709 13:17:51.490362 (+797546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.626149 (+135787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.616561 (+2990412us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b5400369050. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.616744 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732688640}
I20260709 13:17:54.621098 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:54.621661 13476 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.622934 13471 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:54.623661 13471 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.624190 13471 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:54.573035 10897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 407) took 3633 ms (client timeout 2230 ms). Trace:
W20260709 13:17:54.627998 10897 rpcz_store.cc:269] 0709 13:17:50.939783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:50.940023 (+   240us) service_pool.cc:224] Handling call
0709 13:17:54.572741 (+3632718us) spinlock_profiling.cc:243] Waited 2.95 s on lock 0x7b540063fbd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.572905 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6774567296}
W20260709 13:17:54.628604 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.632424 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1642) took 3366 ms. Trace:
I20260709 13:17:54.632969 10052 rpcz_store.cc:276] 0709 13:17:51.265987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.410276 (+1144289us) service_pool.cc:224] Handling call
0709 13:17:54.195608 (+1785332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.199642 (+  4034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.632149 (+432507us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54002e8950. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.632326 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4034158464}
I20260709 13:17:54.634400 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.636236 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1563) took 3594 ms. Trace:
I20260709 13:17:54.513927 13426 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:54.636936 10127 rpcz_store.cc:276] 0709 13:17:51.041529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.794914 (+1753385us) service_pool.cc:224] Handling call
0709 13:17:54.249344 (+1454430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.275195 (+ 25851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.636020 (+360825us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002eccd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.636161 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3343744384}
I20260709 13:17:52.345146 10053 rpcz_store.cc:276] 0709 13:17:51.265650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.940121 (+674471us) service_pool.cc:224] Handling call
0709 13:17:51.952181 (+ 12060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:51.952491 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:52.066699 (+114208us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.310417 (+243718us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.310503 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:52.310571 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:52.310672 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:52.311068 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9939,"prepare.queue_time_us":384697,"raft.queue_time_us":27729,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":13197,"thread_start_us":316939,"threads_started":2}
W20260709 13:17:54.514748 13437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.014s	sys 0.007s
W20260709 13:17:54.594990 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.643203 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:54.643877 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.644608 13576 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.645206 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:54.645337 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.663177 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.662024 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2771) took 1987 ms. Trace:
W20260709 13:17:54.657958 10198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1679) took 2577 ms (client timeout 2267 ms). Trace:
I20260709 13:17:54.667497 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.665704 10900 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:54.668452 10899 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:17:54.560173 10051 rpcz_store.cc:276] 0709 13:17:52.392771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.378999 (+986228us) service_pool.cc:224] Handling call
0709 13:17:53.379992 (+   993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.632161 (+252169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.529436 (+897275us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001b7e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.529773 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559730944}
W20260709 13:17:54.663236 10020 connection.cc:256] RPC call timeout handler was delayed by -1.1114927399999033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:17:54.669329 10020 connection.cc:256] RPC call timeout handler was delayed by -1.034992740000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260709 13:17:54.669920 10051 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:17:54.575008 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 69147.8us
I20260709 13:17:54.596966 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.573621 13522 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:54.600332 10199 raft_consensus.cc:2410] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:17:54.636260 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 281) took 1718 ms. Trace:
W20260709 13:17:54.688824 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.666433 10197 rpcz_store.cc:276] 0709 13:17:52.674972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.009633 (+334661us) service_pool.cc:224] Handling call
0709 13:17:53.022454 (+ 12821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:53.062452 (+ 39998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.062495 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:53.062540 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:53.062587 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.661552 (+1598965us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b540035a2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.661626 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.661900 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91459456}
I20260709 13:17:54.672676 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:54.650171 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.691802 10056 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.695575 13369 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:17:54.696394 13369 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:54.696949 13369 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:54.653314 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 322) took 3327 ms. Trace:
I20260709 13:17:54.678458 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:54.699211 10203 rpcz_store.cc:276] 0709 13:17:51.326333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.985071 (+1658738us) service_pool.cc:224] Handling call
0709 13:17:52.994641 (+  9570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.365196 (+1370555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.652950 (+287754us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.653197 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087327104}
I20260709 13:17:54.704212 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.705475 10199 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:54.701107 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:54.679042 10049 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:54.709136 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1663) took 1024 ms. Trace:
I20260709 13:17:54.709690 10049 rpcz_store.cc:276] 0709 13:17:53.684896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.618632 (+933736us) service_pool.cc:224] Handling call
0709 13:17:54.619034 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.708693 (+ 89659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.709031 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:17:54.667804 10198 rpcz_store.cc:269] 0709 13:17:52.080673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.435732 (+355059us) service_pool.cc:224] Handling call
0709 13:17:54.657720 (+2221988us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.657865 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3238180096}
I20260709 13:17:54.713039 10198 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:17:54.715289 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.716743 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:54.682235 10053 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:17:54.541781 13475 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.680500 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.731434 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:54.732051 10051 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:17:54.732141 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.733179 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.733539 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:54.575420 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.515643 13573 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.706852 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.686313 10123 rpcz_store.cc:276] 0709 13:17:52.918032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.356511 (+438479us) service_pool.cc:224] Handling call
0709 13:17:53.474018 (+117507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:53.478240 (+  4222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:53.478299 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:53.491017 (+ 12718us) log.cc:844] Serialized 26 byte log entry
0709 13:17:53.712110 (+221093us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:17:53.756414 (+ 44304us) log.cc:844] Serialized 14 byte log entry
0709 13:17:53.756661 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:53.756761 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:53.992268 (+235507us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b18002021e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:54.593617 (+601349us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18002021e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:17:54.636087 (+ 42470us) raft_consensus.cc:1685] finished
0709 13:17:54.636109 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.636208 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44040,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":3346,"raft.queue_time_us":3016,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":20462,"spinlock_wait_cycles":11441920,"thread_start_us":187700,"threads_started":1,"wal-append.queue_time_us":259474}
I20260709 13:17:54.721272 13421 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:54.722085 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.652271 13464 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:17:54.723162 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.745368 10197 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:54.649813 13576 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:54.661546 13476 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:54.724143 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.647039 13172 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:17:54.660125 13570 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.763s	user 0.005s	sys 0.001s
I20260709 13:17:54.727491 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.728163 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.646812 13524 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:17:54.685196 13516 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.726580 13572 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:54.732844 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:54.770042 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49689.1us
W20260709 13:17:54.739984 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53372us
I20260709 13:17:54.770363 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:54.771507 10897 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.771868 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:54.751500 13437 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:17:54.773708 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:54.774499 10891 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:54.747555 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:54.778509 13512 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:54.778831 13471 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.779592 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.781971 10898 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:54.782348 13471 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:54.752374 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.784284 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.785028 10053 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:54.786114 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.786793 10053 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.791422 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:54.791527 13522 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:54.792413 13522 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:54.792946 13522 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:54.796427 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1638) took 3531 ms. Trace:
I20260709 13:17:54.796921 10055 rpcz_store.cc:276] 0709 13:17:51.265306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.437306 (+172000us) service_pool.cc:224] Handling call
0709 13:17:51.437637 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.037848 (+600211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:52.037878 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:52.037929 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:52.037972 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.796156 (+2758184us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400369554. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.796208 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.796356 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379778816}
I20260709 13:17:54.797955 13536 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.798496 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.799767 13576 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.801767 13437 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:54.795513 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.732825 13530 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.802913 13530 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.804258 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 13) took 1012 ms. Trace:
I20260709 13:17:54.804760 10200 rpcz_store.cc:276] 0709 13:17:53.791900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.801558 (+  9658us) service_pool.cc:224] Handling call
0709 13:17:54.804002 (+1002444us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000b3954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:54.804172 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454326656}
I20260709 13:17:54.804966 13510 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:54.807360 10895 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:54.808262 13364 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.729899 13411 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:54.749207 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.729561  9943 maintenance_manager.cc:419] P db0c98fae3aa4b45a39c8914c5e641cc: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033337
I20260709 13:17:54.768366 10123 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:54.701809 13508 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.740947 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.829113 10051 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.744215 13421 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:54.751524 13435 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:54.751901 13172 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.763031 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:54.808854 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:54.800338 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.810442 10055 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:54.576174 13491 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:54.877933 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.847429 10056 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:17:54.879534 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.879989 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.802819 10049 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:17:54.756031 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:54.825251 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:54.880842 10056 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.540083 13424 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:54.832343 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.833081 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.833859 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.842684 10053 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:54.910406 10198 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:54.801440 13576 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:54.913509 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.854089 10891 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:54.802479 13437 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.880332 10200 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.827347 10126 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
W20260709 13:17:54.881641 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 125702us
I20260709 13:17:54.917083 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.813864 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:54.806957 13530 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.885165 10123 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:54.886771 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:54.888554 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.889777 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:54.887701 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:54.897258 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.897982 10051 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:54.846608 13421 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:54.828344 13536 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:54.804425 13524 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:17:54.875813 13411 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:54.902107 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1667) took 1065 ms. Trace:
I20260709 13:17:54.883055 13491 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:54.823938 13516 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.883401 13554 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:54.907873 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.911554 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.911798 10053 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:54.919512 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:54.921558 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.918958 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.889796 13367 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:54.929589 10051 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:17:54.854272 13508 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:54.934495 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46923.5us
W20260709 13:17:54.933816 13398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.027s	sys 0.026s
I20260709 13:17:54.943554 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.908877 13424 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.943096 10049 rpcz_store.cc:276] 0709 13:17:53.836903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.889437 (+1052534us) service_pool.cc:224] Handling call
0709 13:17:54.901459 (+ 12022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.901679 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:54.901727 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:54.901775 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:54.901809 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.901871 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:54.902033 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:54.975443 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:54.947824 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.953150 10052 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
W20260709 13:17:54.975016 13398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.028s	sys 0.027s
W20260709 13:17:54.958238 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:54.958581 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:54.960779 10051 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:17:54.993269 13426 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:54.993875 13426 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:54.994135 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.995082 10203 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:54.980383 13451 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:17:54.996203 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:54.996722 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1562) took 3960 ms. Trace:
I20260709 13:17:54.996886 10203 raft_consensus.cc:606] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:54.994859 13426 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.997219 10125 rpcz_store.cc:276] 0709 13:17:51.036648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.670245 (+1633597us) service_pool.cc:224] Handling call
0709 13:17:52.670640 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:52.987459 (+316819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.996450 (+2008991us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:54.996652 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":586658560}
I20260709 13:17:54.998381 13426 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:54.962848 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 355873us
I20260709 13:17:55.003221 13437 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.004716 13437 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:55.004756 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.007668 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.008106 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.008767 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.009202 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.009550 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.014031 13369 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:54.923781 13172 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:54.941939 13538 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:17:54.957480 13493 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.552s	user 0.034s	sys 0.044s
I20260709 13:17:54.932024 13464 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:17:54.971817 10055 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:54.974558 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:54.987490 10049 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:54.995303 13419 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:54.952826 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 181006us
W20260709 13:17:55.029749 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 55352.2us
I20260709 13:17:55.030507 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:55.036576 10052 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:55.030507 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:55.033279 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:55.016319 10205 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:55.017880 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53193.3us
I20260709 13:17:55.017715 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1659) took 1378 ms. Trace:
I20260709 13:17:54.962798 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:54.963243 10056 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:54.978144 13469 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:55.026784 10897 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:55.097358 10198 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:17:55.120568 10055 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:55.099851 10053 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:17:55.100762 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.027083 13493 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.621s	user 0.035s	sys 0.044s
I20260709 13:17:54.973516 13554 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.126901 10049 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: []
W20260709 13:17:55.126825 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93587.9us
I20260709 13:17:55.128947 10198 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:55.125322 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:55.026252 13231 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:17:55.139599 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109130us
I20260709 13:17:55.153173 10897 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 6 LEARNER]: Advancing to term 7
I20260709 13:17:55.154948 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:55.141254 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:55.096618 13512 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:55.184744 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:55.096000 13394 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:17:55.183372 13538 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:55.133150 13476 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.194517 13538 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:55.194840 13394 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:55.195856 13538 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.028916 13356 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
I20260709 13:17:55.200778 13459 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:55.204159 13508 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.205056 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:55.025971 13461 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.126s
I20260709 13:17:55.152201 13231 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.129482 13471 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:55.015242 13020 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 13:17:55.096554 13172 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.178392 13554 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:55.179538 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:55.181020 13435 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:55.179531 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:55.212908 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:55.195930 13394 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.214102 10893 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:55.214403 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:55.215284 10892 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:55.211328 13525 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:17:55.216426 13604 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:55.204015 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:55.216992 13604 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.217406 13604 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:55.206614 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:55.169103 13589 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:17:55.167500 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:55.201382 13369 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:55.221133 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1661) took 1537 ms. Trace:
I20260709 13:17:55.221567 13369 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:55.222244 13369 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.224867 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:55.226974 10048 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:17:55.192936 10897 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:55.208799 10049 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:55.240958 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:55.241990 10049 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:55.210019 10200 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:17:55.210708 10900 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10.
I20260709 13:17:55.096796 13572 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 13:17:55.154140 13411 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:55.151894 13537 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:55.205798 13512 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:55.271366 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:55.236358 13475 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.252884 13522 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.286844 13522 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.288326 13411 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.289777 10205 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:55.249778 13464 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:55.291261 13464 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.292534 13369 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.292340 13464 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.293694 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:55.252518 13356 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:55.295228 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:55.295706 13356 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:55.296463 13356 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.297432 10896 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:17:55.302829 10201 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:17:55.303757 13196 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.304360 13196 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.127779 13424 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.250633 13020 consensus_queue.cc:1243] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:17:55.210904 13231 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:55.281441 10899 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:17:55.282327 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:55.223448 13555 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
W20260709 13:17:55.284220 13573 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
I20260709 13:17:55.238505 13418 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.271106 13537 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.271445 13512 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:55.323390 10126 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:55.262732 13596 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:55.276175 13491 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.342032 10197 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:17:55.343313 13596 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:55.352372 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:55.356307 13510 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:55.357036 13510 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.357184 10893 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:55.362076 13479 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:55.362644 13479 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.339722 13418 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.363749 13479 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.358026 13510 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.374007 13596 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:55.402112 13603 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
I20260709 13:17:55.401022 13537 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:55.432541 13421 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.468312 13574 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:55.468824 13574 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:55.469681 13421 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.444921 13596 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.448083 13491 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:55.470570 13574 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.472086 13574 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:55.471870 13512 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.473378 13424 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:55.432126 13418 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.464148 13469 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:55.475627 13516 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.475991 13367 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:55.459534 13604 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.476507 13367 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.477578 13367 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:55.461493 13595 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.494s	user 0.003s	sys 0.000s
I20260709 13:17:55.464161 13475 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:55.467211 13536 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 13:17:55.475795 13469 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.481570 13398 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:55.481280 13469 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.482184 13398 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:55.482797 13469 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.478803 13604 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.480358 13512 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.482353 13596 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:55.221781 10054 rpcz_store.cc:276] 0709 13:17:53.684278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.491501 (+807223us) service_pool.cc:224] Handling call
0709 13:17:54.492312 (+   811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:54.493534 (+  1222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:54.493582 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:54.495156 (+  1574us) log.cc:844] Serialized 525 byte log entry
0709 13:17:54.814546 (+319390us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:54.814591 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:54.814684 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:55.220783 (+406099us) raft_consensus.cc:1685] finished
0709 13:17:55.220817 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:55.221043 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":194,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"thread_start_us":318860,"threads_started":1,"wal-append.queue_time_us":452136}
W20260709 13:17:55.527297 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 302371us
I20260709 13:17:55.535838 10051 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:17:55.540253 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1673) took 1216 ms. Trace:
W20260709 13:17:55.541415 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1709) took 1455 ms (client timeout 1813 ms). Trace:
W20260709 13:17:55.544476 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 403032us
W20260709 13:17:55.545885 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.546175 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.548568 10198 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:55.556231 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:55.559900 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1688) took 1245 ms. Trace:
W20260709 13:17:55.566017 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1658) took 1926 ms (client timeout 1844 ms). Trace:
W20260709 13:17:55.567728 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1680) took 3355 ms (client timeout 2201 ms). Trace:
I20260709 13:17:55.572813 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:55.578387 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:55.584786 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:55.588948 10048 rpcz_store.cc:269] 0709 13:17:54.085969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.161690 (+ 75721us) service_pool.cc:224] Handling call
0709 13:17:55.530828 (+1369138us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400086e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.541062 (+ 10234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2442945920}
I20260709 13:17:55.542724 10053 rpcz_store.cc:276] 0709 13:17:54.323556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.956280 (+632724us) service_pool.cc:224] Handling call
0709 13:17:54.976703 (+ 20423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:55.121384 (+144681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.534319 (+412935us) spinlock_profiling.cc:243] Waited 62.1 ms on lock 0x7b5400136d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:17:55.539926 (+  5607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142856960}
I20260709 13:17:55.589666 10201 rpcz_store.cc:276] 0709 13:17:54.314245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.433381 (+119136us) service_pool.cc:224] Handling call
0709 13:17:55.559018 (+1125637us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001d0454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.559487 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922841856}
I20260709 13:17:55.657191 10204 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:55.658234 10201 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:17:55.590417 10202 rpcz_store.cc:269] 0709 13:17:52.212021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.436660 (+224639us) service_pool.cc:224] Handling call
0709 13:17:55.567090 (+3130430us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400359dd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.567432 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559232256}
W20260709 13:17:55.590090 10047 rpcz_store.cc:269] 0709 13:17:53.638955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:53.749986 (+111031us) service_pool.cc:224] Handling call
0709 13:17:55.565345 (+1815359us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400199454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.565718 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351228800}
I20260709 13:17:55.056572 10050 rpcz_store.cc:276] 0709 13:17:53.639356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.474851 (+835495us) service_pool.cc:224] Handling call
0709 13:17:54.475262 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:54.475520 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:54.814497 (+338977us) log.cc:844] Serialized 14 byte log entry
0709 13:17:55.017341 (+202844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:55.017396 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:55.017448 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.017520 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:55.017649 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":455760,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3315,"raft.queue_time_us":111,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7930,"thread_start_us":515916,"threads_started":2}
W20260709 13:17:55.673638 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.478825 13367 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:55.681373 10200 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:55.684990 10196 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
W20260709 13:17:55.723588 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.03206e+06us
I20260709 13:17:55.704715 13394 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:55.731693 13394 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.736153 13475 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:55.738302 10205 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:17:55.739289 10205 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 7
I20260709 13:17:55.739718 10205 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:17:55.740149 10205 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:55.711005 13596 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:55.745890 13605 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.746891 13596 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:55.749191 10895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 241) took 4494 ms (client timeout 2615 ms). Trace:
W20260709 13:17:55.749809 10895 rpcz_store.cc:269] 0709 13:17:51.262455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.287375 (+ 24920us) service_pool.cc:224] Handling call
0709 13:17:55.748574 (+4461199us) spinlock_profiling.cc:243] Waited 3.38 s on lock 0x7b54006ee454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.748868 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7768840960}
I20260709 13:17:55.751569 10203 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:55.753429 13555 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.756476 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:55.759395 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70544.4us
I20260709 13:17:55.773043 13355 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:55.750726 13596 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:55.734568 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 476139us
W20260709 13:17:55.792366 10056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.011s	sys 0.026s
I20260709 13:17:55.754546 13555 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:55.812892 10056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.011s	sys 0.027s
I20260709 13:17:55.824019 13480 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:17:55.828011 13510 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:17:55.828850 13510 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:55.829339 13524 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.840749 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2784) took 1005 ms. Trace:
I20260709 13:17:55.841259 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:55.841507 10197 rpcz_store.cc:276] 0709 13:17:54.834698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.886396 (+ 51698us) service_pool.cc:224] Handling call
0709 13:17:55.840075 (+953679us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:55.840466 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889362304}
I20260709 13:17:55.842882 13411 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:55.746935 13605 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.780496 13398 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:55.845850 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.848284 13394 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:55.862231 13479 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:55.848882 13196 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:55.861913 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:55.855729 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1681) took 1178 ms. Trace:
I20260709 13:17:55.862033 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:55.875720 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.883222 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:55.884734 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 158720us
I20260709 13:17:55.856163 13437 raft_consensus.cc:993] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
W20260709 13:17:55.858712 13231 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.518s	user 0.001s	sys 0.023s
W20260709 13:17:55.841293 13356 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.008s
I20260709 13:17:55.846518 13605 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:55.894707 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80098us
W20260709 13:17:55.896391 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55394.4us
I20260709 13:17:55.849505 13344 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:17:55.894268 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:55.891011 13411 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:55.897426 10197 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:55.870087 10056 rpcz_store.cc:276] 0709 13:17:54.677399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.070817 (+393418us) service_pool.cc:224] Handling call
0709 13:17:55.071466 (+   649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.114673 (+ 43207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:55.114720 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:55.130542 (+ 15822us) log.cc:844] Serialized 26 byte log entry
0709 13:17:55.818436 (+687894us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:17:55.818582 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.818754 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:55.855437 (+ 36683us) raft_consensus.cc:1685] finished
0709 13:17:55.855461 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:55.855648 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48588,"raft.queue_time_us":25891,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":15891,"thread_start_us":633149,"threads_started":1,"wal-append.queue_time_us":670719}
I20260709 13:17:55.892031 13475 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:55.908761 13355 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:55.916599 13536 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.918828 10048 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 4. Current term is 5. Ops: []
I20260709 13:17:55.933095 10198 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:55.920966 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:55.897855 13231 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.893857 13437 raft_consensus.cc:1081] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:17:55.919750 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:55.830590 13510 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:55.852217 13537 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:17:55.938009 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:55.938706 13614 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 13:17:55.938500 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:55.943994 10048 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:55.934214 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:55.899871 13356 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.897320 13364 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:17:55.931452 13355 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:55.962055 10891 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:17:55.962256 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:55.901633 13611 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:17:55.968993 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:55.957505 13480 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:17:55.967435 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:55.973676 10199 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:55.974938 10203 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:55.960597 13344 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:55.977918 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63203.8us
I20260709 13:17:55.945129 13411 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.980111 10124 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:55.945927 13536 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.994155 10128 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:55.995679 13427 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:55.996234 13512 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:55.996516 13427 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:55.997146 13512 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:55.998164 13196 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:55.998061 13451 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:55.999233 13596 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:55.995365 13394 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:56.000139 13555 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.001472 10897 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
W20260709 13:17:55.995606 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87726.3us
I20260709 13:17:55.997994 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.003669 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:55.999378 13451 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.010159 10125 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:56.010511 13355 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.012796 13451 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.001531 13418 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:17:56.017742 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:56.018527 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:56.021271 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.012414 13355 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.007966 13196 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.033016 13536 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:56.034070 13512 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.033819 13196 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.012133 13344 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.026429 13234 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:56.028012 13231 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.030432 13411 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.031579 10127 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:56.012421 13574 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.008s
I20260709 13:17:56.021359 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:56.037267 13620 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:17:56.038954 13424 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.565s	user 0.017s	sys 0.005s
I20260709 13:17:56.026428 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.035310 13427 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:56.066742 10047 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:56.067301 10198 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:56.069844 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64897.9us
I20260709 13:17:56.071662 13231 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:56.072045 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.073698 10122 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.076970 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 307) took 1242 ms. Trace:
I20260709 13:17:56.077543 10055 rpcz_store.cc:276] 0709 13:17:54.834072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.162564 (+328492us) service_pool.cc:224] Handling call
0709 13:17:55.227360 (+ 64796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.227723 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:55.227753 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:55.244222 (+ 16469us) log.cc:844] Serialized 26 byte log entry
0709 13:17:55.716584 (+472362us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:55.735246 (+ 18662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.762985 (+ 27739us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:56.076530 (+313545us) raft_consensus.cc:1685] finished
0709 13:17:56.076552 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.076748 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8723,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8608,"thread_start_us":154927,"threads_started":1,"wal-append.queue_time_us":436133}
I20260709 13:17:56.081158 10127 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:56.082685 13493 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:17:56.083693 13493 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.084853 13493 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.086198 13493 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.089820 10129 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:56.039615 13356 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:56.039981 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1682) took 1362 ms. Trace:
I20260709 13:17:56.035315 13605 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:56.040657 13476 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:56.047866 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.049966 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.054183 13464 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:56.054457 10125 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:56.055198 13398 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:56.055780 13172 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.831s	user 0.012s	sys 0.012s
I20260709 13:17:56.058257 10124 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:56.059634  9953 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:56.099236 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 110190us
W20260709 13:17:56.100613 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.105172 10126 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:56.097571 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.041853 13344 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:56.101167 13605 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.103787 13464 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.101763 13476 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.096700 10052 rpcz_store.cc:276] 0709 13:17:54.677722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:55.097581 (+419859us) service_pool.cc:224] Handling call
0709 13:17:55.162113 (+ 64532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:55.162487 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:55.162532 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:55.205198 (+ 42666us) log.cc:844] Serialized 26 byte log entry
0709 13:17:55.877204 (+672006us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:55.883976 (+  6772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:55.890069 (+  6093us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:56.039554 (+149485us) raft_consensus.cc:1685] finished
0709 13:17:56.039587 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:56.039787 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":499051,"threads_started":1,"wal-append.queue_time_us":713405}
I20260709 13:17:56.104928 10128 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:56.102546 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:56.121062 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72405.5us
W20260709 13:17:56.127393 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.124630 13464 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.129479 13411 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.133546 13476 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.135527 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:56.136307 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87903.2us
I20260709 13:17:56.136583 10129 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:56.139631 13476 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:56.120661 13605 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.145265 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.146239 10125 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.115723 13427 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:56.140430 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.140543 13231 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.129729 10124 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Committing config change with OpId 6.5: config changed from index 3 to 5, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:56.141141 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:56.148833 10198 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:56.147317 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.150106 10122 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:56.156409 10205 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:17:56.152258 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:56.158080 10129 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:56.155993 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2779) took 1322 ms. Trace:
I20260709 13:17:56.161087 10199 rpcz_store.cc:276] 0709 13:17:54.833363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:54.918564 (+ 85201us) service_pool.cc:224] Handling call
0709 13:17:56.154248 (+1235684us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:56.155912 (+  1664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353518336}
I20260709 13:17:56.147078 13427 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.174670 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.176043 10122 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:56.179526 13356 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.158440 10126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:17:56.180294 13364 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:56.187816 13364 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.188135 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:56.178169 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61613.4us
I20260709 13:17:56.188409 13364 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.193549 13510 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.195452 13196 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:56.197956 13555 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:56.198489 13555 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:56.173555 13479 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:17:56.199749 13555 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.201434 13555 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.169931 13574 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:56.208261  9953 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.185542 13596 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:17:56.218204 10126 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:56.220283 10126 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:56.217517 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50422.9us
I20260709 13:17:56.106788 13475 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.226248 13596 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.221742 13352 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:17:56.227968 13352 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:56.228480 13352 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:56.229718 13352 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:56.231344 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.231904 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:56.171725 13231 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.233536 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.237272 13451 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.237962 13231 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:56.234860 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.239542 10891 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:56.213141  9953 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:56.243445 13367 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.541s	user 0.019s	sys 0.001s
I20260709 13:17:56.244799 13424 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:56.246408 13424 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:56.246814 13424 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.218070 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:56.127804 10049 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Committing config change with OpId 6.5: config changed from index 3 to 5, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:56.248611 13427 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:56.258100 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:56.241869 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:56.257892 13344 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:56.263453 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.221094 13574 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.227057 13596 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:56.254915 10126 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:56.276397 13632 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:17:56.277685 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.276703 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:17:56.277652 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89141.2us
I20260709 13:17:56.273885 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:56.264454 13352 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.295939 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:17:56.285171 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67558.4us
I20260709 13:17:56.301224 13418 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:17:56.282320  9953 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.252664 13356 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.328014 10125 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:56.337373 10126 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:56.341913 13626 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 13:17:56.317577 13424 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:56.344314 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.309263 13427 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:56.351022 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 142641us
I20260709 13:17:56.353215 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.350306 13609 raft_consensus.cc:2764] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:56.327369 13512 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.357168 13172 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:56.368602 13627 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:56.388041 13627 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:56.388500 13355 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:56.389058 13355 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.389211 13627 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.389559 13411 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:56.369681 13616 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
I20260709 13:17:56.371718 13424 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:56.391654 13512 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.375831 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:56.395428 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 180405us
I20260709 13:17:56.396066 10205 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.368621 13588 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:17:56.367199 13451 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.384325 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:17:56.405220 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44232.6us
I20260709 13:17:56.407501 13574 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.368952 13623 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:56.389690 13627 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.394783 13512 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.411235 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:56.411567 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.423570 13623 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:17:56.424199 13627 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.382510 13605 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.425410 13634 raft_consensus.cc:2764] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:17:56.431308 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.424780 13630 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:56.436293 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 80463.4us
I20260709 13:17:56.427445  9953 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 5, term changed from 6 to 8, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to <none>, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:17:56.440476 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.390369 13355 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.424031 13356 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:56.450217 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 154383us
I20260709 13:17:56.445849 10205 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:17:56.451181 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 142708us
I20260709 13:17:56.426522 13427 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:17:56.455843 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:56.452702 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:56.460193 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48618.4us
I20260709 13:17:56.466615 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.466979 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.446081 13196 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:17:56.472663 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.470717  9953 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.448807 13630 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.419804 13588 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
W20260709 13:17:56.479631 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 82551.1us
I20260709 13:17:56.466310 13451 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:56.489028 10199 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:56.469961 13632 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:17:56.474406 13196 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.474313 13418 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:56.513346 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46781.1us
W20260709 13:17:56.502135 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 98318.6us
I20260709 13:17:56.509698 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:56.512472 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.501060 13234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50710 (request call id 8) took 19424 ms. Trace:
I20260709 13:17:56.489403 13634 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:17:56.522028 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.520661 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:56.530870 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:56.491942 13355 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.527462 10123 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:17:56.489373 13630 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.495219 13464 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.513495  9953 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.532050 13573 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:56.539762 10055 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:17:56.543727 10204 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:17:56.539929 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44292.8us
I20260709 13:17:56.514377 13196 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:56.546411 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47306.2us
I20260709 13:17:56.553258 10052 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:56.537601 13493 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.562458 13640 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:17:56.562799 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57508.1us
I20260709 13:17:56.512995 13364 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.542903 13573 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:56.587137 13464 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.564924 13623 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:56.576099 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 75577us
I20260709 13:17:56.543787 13418 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.542488 13632 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.583590 10047 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:56.599375 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85139.4us
W20260709 13:17:56.575645 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.576139 13637 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:17:56.567977 10019 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.752s (SENT)
I20260709 13:17:56.607383 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:56.590958  9951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 219) took 30207 ms (client timeout 15000 ms). Trace:
I20260709 13:17:56.586257 13364 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.600898 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.593027  9953 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.617429 10891 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:17:56.617650 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.565443 13493 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.620244 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:56.627755 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:56.589166 13573 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.567394 13640 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.575711 13537 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:56.632588 13588 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:56.642203 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:56.642887 13418 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:56.643611 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68757.6us
I20260709 13:17:56.643811 13344 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:56.655215 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.607841 13451 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:56.634244 13493 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.659652 10047 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:17:56.521116 13234 rpcz_store.cc:276] 0709 13:17:37.056910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:37.073676 (+ 16766us) service_pool.cc:224] Handling call
0709 13:17:37.449615 (+375939us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b54003d34a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f81520 00007fa27800ab42 00007fa278009800 00007fa27cfcad29 00007fa2765491d4 00007fa276549162 00007fa276549091 00007fa27654900c 00007fa2765482c2 00007fa27633f48c
0709 13:17:38.084348 (+634733us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b58008192d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277fff70a 00007fa27800076a 00007fa277fff91b 00007fa27800b03c 00007fa27800a6b3 00007fa27cfcad29 00007fa2765491d4 00007fa276549162
0709 13:17:38.822921 (+738573us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5800818fd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277fff70a 00007fa27800076a 00007fa277fff91b 00007fa27800b03c 00007fa27800a6b3 00007fa27cfcad29 00007fa2765491d4 00007fa276549162
0709 13:17:39.594689 (+771768us) log.cc:844] Serialized 499 byte log entry
0709 13:17:42.013542 (+2418853us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b58007361d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa277f5e43c 00007fa277f64d53 00007fa277f64cd9 00007fa277f64c69 00007fa277f64c31 00007fa277f63dfd 000000000038f354 000000000038f199 00007fa274f17475 00007fa274f197c1
0709 13:17:56.474157 (+14460615us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":172296,"raft.queue_time_us":26664,"raft.run_cpu_time_us":3592,"raft.run_wall_time_us":1067861,"spinlock_wait_cycles":2950883712,"thread_start_us":1640164,"threads_started":2,"wal-append.queue_time_us":1742695,"wal-append.run_cpu_time_us":3805,"wal-append.run_wall_time_us":1452882}
I20260709 13:17:56.640702 13634 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:17:56.665194 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 103306us
W20260709 13:17:56.681690 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61681.5us
W20260709 13:17:56.692770 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.652498 13588 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.684511 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:56.642891 13352 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:17:56.653201 13640 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:56.710659 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50018.1us
I20260709 13:17:56.655524 13344 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.671326 13456 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:17:56.731372 13418 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.732178 13418 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.705811  9953 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 7 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.734740 13418 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:56.735495  9953 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.736670 13464 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.737090 10052 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:56.660466 13635 raft_consensus.cc:2764] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:17:56.713809 13356 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:56.708660 13588 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.709379 13427 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:17:56.743741 13645 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:17:56.744031 13576 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:17:56.744885 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:56.748911 13451 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:56.608444  9951 rpcz_store.cc:269] 0709 13:17:26.383156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:26.396623 (+ 13467us) service_pool.cc:224] Handling call
0709 13:17:56.590563 (+30193940us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:48.944200 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:48.964993 (+ 20793us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:48.965055 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:17:48.965077 (+    22us) tablet.cc:662] Decoding operations
0709 13:17:49.283528 (+318451us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:49.283602 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:49.283636 (+    34us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:17:49.524819 (+241183us) tablet.cc:701] Row locks acquired
0709 13:17:49.524857 (+    38us) write_op.cc:260] PREPARE: finished
0709 13:17:49.525085 (+   228us) write_op.cc:270] Start()
0709 13:17:49.525264 (+   179us) write_op.cc:276] Timestamp: P: 1783603069525012 usec, L: 0
0709 13:17:49.525289 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:49.526198 (+   909us) log.cc:844] Serialized 6747 byte log entry
0709 13:17:50.803138 (+1276940us) op_driver.cc:464] REPLICATION: finished
0709 13:17:51.255151 (+452013us) write_op.cc:301] APPLY: starting
0709 13:17:51.255277 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:51.255850 (+   573us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:51.256031 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:51.269720 (+ 13689us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:51.270125 (+   405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:17:51.270174 (+    49us) write_op.cc:312] APPLY: finished
0709 13:17:51.270655 (+   481us) log.cc:844] Serialized 119 byte log entry
0709 13:17:51.272398 (+  1743us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:51.272728 (+   330us) write_op.cc:454] Released schema lock
0709 13:17:51.272855 (+   127us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:51.276705 (+  3850us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:51.276829 (+   124us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4540,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":429295,"apply.run_cpu_time_us":5892,"apply.run_wall_time_us":21382,"num_ops":17,"prepare.queue_time_us":475155,"prepare.run_cpu_time_us":31211,"prepare.run_wall_time_us":774449,"raft.queue_time_us":264219,"raft.run_cpu_time_us":18712,"raft.run_wall_time_us":506173,"replication_time_us":1205846,"thread_start_us":724017,"threads_started":4,"wal-append.queue_time_us":213654,"wal-append.run_cpu_time_us":30530,"wal-append.run_wall_time_us":2524491}]]}
W20260709 13:17:56.751911 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55304.1us
I20260709 13:17:56.675807 13605 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:56.754046 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.755820 10169 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10.
I20260709 13:17:56.769560 10169 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:56.786923 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:56.787694 10198 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:56.721246 13596 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.762848 13576 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:56.786044 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.797670  9953 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.800046 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.814153 10124 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:56.800606 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:56.801070 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:56.792366 13344 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.802160 13427 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:56.784474 13476 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10.
I20260709 13:17:56.803040 13356 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:56.779757 13634 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:56.800657 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:56.807385 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:56.835493 10054 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:56.795753 13480 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:17:56.812444 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.799507 13646 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:56.799786 13172 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:56.817960 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:56.820147 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:56.761473 13464 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.849299 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:56.832180 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:17:56.793733 13555 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.592s	user 0.016s	sys 0.000s
I20260709 13:17:56.825055 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:56.795377 13623 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.849189 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.859910 10123 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.849514 10891 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:56.852839 10201 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:56.843010 10019 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.087s (SENT)
I20260709 13:17:56.861124 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:56.830678 13427 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.834148 13356 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.873435 10130 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:17:56.875485 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.848634 13344 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.888417 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:56.888824 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 190099us
I20260709 13:17:56.882272 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:56.881819 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:56.893229 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.896800 10124 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:56.813050 13576 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:56.860250 13398 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:56.895778 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:56.912237 10127 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:56.875861 13635 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:17:56.838089 13418 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:17:56.913519 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:56.911463 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 129560us
I20260709 13:17:56.937628 10047 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:17:56.935391 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:56.932449 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52640.5us
I20260709 13:17:56.943192 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.879309 13555 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:17:56.901604 13398 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:56.956105 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.976759 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:56.955299 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.915452 13364 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:17:56.941169 13475 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:56.875156 13646 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:56.989959 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:56.993498 10126 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:56.933372 13654 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:17:56.992295 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.003778 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 160787us
I20260709 13:17:57.008134 10122 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:56.989738 13537 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:17:57.008895 13649 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
I20260709 13:17:56.981743 13398 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.019635 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:56.988709 13576 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:17:57.024538 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.026404 10128 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:57.033862 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.002044 13475 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:56.947331 13623 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.041098 10127 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.036880 10130 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:57.036921 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60316.3us
W20260709 13:17:57.050074 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 142064us
I20260709 13:17:57.052863 10124 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:57.011978 13661 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:17:57.036725 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 155459us
I20260709 13:17:57.007165 13666 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.070456 13666 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.071065 13666 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:56.976904 13630 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.072829 10125 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.074306 13630 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.024047 13635 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:57.069447 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.076161 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:17:57.021605 13654 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:57.078239 10129 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:57.068665 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.035156 13555 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:57.107483 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:57.044656 13475 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:57.110486 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.110857 10130 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
W20260709 13:17:57.113646 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:57.109666 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 273029us
I20260709 13:17:57.122321 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:57.106315 13196 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.001s
I20260709 13:17:57.121990 10128 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:17:57.115691 10124 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.084731 13596 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.066329 13661 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:57.143373 10126 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:57.115090 13670 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:57.150646 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.152433 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 212762us
I20260709 13:17:57.133612 13664 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:57.156711 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 80679.1us
I20260709 13:17:57.157584 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.130563 13630 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:17:57.160203 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.117947 13635 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:17:57.166767 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.168417 10127 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:57.167409 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.175410 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:57.176267 10047 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:57.110250 13573 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.176783 13476 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:57.182072 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74728.2us
I20260709 13:17:57.156399 13344 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:17:57.141630 13196 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.201803 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.181512 13660 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:17:57.141402 13555 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.198715 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:57.199329 10204 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:57.199661 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.200003 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:57.205057 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:57.209312 10129 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.184542 13464 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.193097 13344 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.223845 10202 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:17:57.224879 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 141207us
I20260709 13:17:57.216795 10123 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:17:57.222910 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46430.2us
I20260709 13:17:57.155418 13640 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.229673 13640 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.232041 13555 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:57.233496 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:57.234530 10047 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:17:57.205749 13670 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.192442 13596 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:17:57.242132 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:57.246186 10055 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:17:57.244289 10127 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:17:57.251837 10129 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:57.225404 13344 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:57.260120 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.265027 13646 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.265513 10897 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:57.256392 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:17:57.242218 13630 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:17:57.276077 10049 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:57.279613 10127 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:57.279858 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.283054 10051 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:57.288405 10129 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:17:57.287303 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:57.279989 13679 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:57.324841 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.279196 13630 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:17:57.325455 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.280316 13663 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.292109 13623 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.334765 10198 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:17:57.338632 10055 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:17:57.346586 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 163434us
I20260709 13:17:57.353233 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.328666 13630 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:57.334915 13627 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.877s	user 0.018s	sys 0.000s
I20260709 13:17:57.362560 10201 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:17:57.364212 10054 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.002s
I20260709 13:17:57.375553 10054 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:17:57.346160 13493 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.000s
W20260709 13:17:57.356060 13652 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.274s	user 0.001s	sys 0.001s
I20260709 13:17:57.376330 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:57.399225 10201 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:57.404542 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:57.408699 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:57.424547 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.436643 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:57.377612 13573 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:57.427717 10129 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6. Re-sending same reply.
I20260709 13:17:57.451092 10201 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:57.411689 10050 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:57.431869 13196 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:57.453902 13476 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:17:57.469475 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:57.452723 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:57.451773 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:57.372862 13627 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.390115 13493 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:57.459816 13658 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
W20260709 13:17:57.469450 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 112006us
I20260709 13:17:57.447686 13667 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.470590 10200 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:57.452663 10198 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:57.481315 10203 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:57.466221 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:57.505945 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:57.512140 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.473735 13685 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.513085 10123 raft_consensus.cc:2368] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:17:57.469036 13677 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:17:57.454516 13659 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:17:57.538576 10200 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:57.512457 13367 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.518116 13464 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:57.511984 13672 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:57.543732 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60684.1us
I20260709 13:17:57.523945 13644 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.548033 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:57.551237 10128 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:57.554590 10169 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:17:57.555830 10123 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:57.536511 13677 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:57.564271 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:57.540998 13367 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:57.545532 13464 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:57.564432 13684 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:17:57.594215 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.592357 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.596733 13659 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:17:57.601109 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.610394 10124 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.601354 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.597309 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.587148 13367 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.601023 10130 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:17:57.600878 13356 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.699s	user 0.005s	sys 0.010s
I20260709 13:17:57.332448 13679 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.608222 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.602772 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.567215 13682 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.158s
W20260709 13:17:57.603005 13668 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.250s	user 0.000s	sys 0.002s
I20260709 13:17:57.653975 13674 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:57.654758 13674 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:17:57.604029 13683 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:17:57.590605 13464 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:57.656229 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.657164 10127 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:57.597723 13588 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.822s	user 0.015s	sys 0.000s
I20260709 13:17:57.658780 13588 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.659788 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:57.609397 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 426349us
I20260709 13:17:57.661454 13588 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:57.602887 13690 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:17:57.623337 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 59220.4us
I20260709 13:17:57.662961 13690 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:57.663511 13690 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:57.663910 13690 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.665804 13696 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:57.666640 13696 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.666657 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:57.575323 13476 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:17:57.669154 13696 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:57.496942 13667 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.669302 10895 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEARNER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:17:57.672480 10898 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:17:57.673015 10898 pending_rounds.cc:85] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 7
I20260709 13:17:57.673271 10898 pending_rounds.cc:107] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:17:57.673542 10898 raft_consensus.cc:2889] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:57.623687 13695 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:17:57.626773 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.600664 13678 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:17:57.674753 10122 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:57.339618 13663 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.253916 13670 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.210799 13661 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.541913 13685 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.629822 13679 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.646826 13451 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:17:57.602991 13689 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:17:57.231679 13635 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.632802 13596 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:57.632241 13677 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.632928 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.637892 13475 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.639506 13627 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:57.639168 13356 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.641248 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.642210 13398 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.578s	user 0.004s	sys 0.007s
I20260709 13:17:57.601261 13688 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:17:57.644953 13655 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.705s	user 0.004s	sys 0.001s
I20260709 13:17:57.647866 13493 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:57.555625 13644 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.585496 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:17:57.552496 13672 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:57.651614 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:57.652793 13427 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.001s
I20260709 13:17:57.621932 10131 raft_consensus.cc:2376] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11. Re-sending same reply.
I20260709 13:17:57.620858 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.672544 13667 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:57.167922 13664 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:57.600386 10128 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:57.675727 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.681651 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.099787 10125 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:57.695294 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:57.233233 13654 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:57.726460 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 170509us
W20260709 13:17:57.669379 13576 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.640s	user 0.006s	sys 0.008s
W20260709 13:17:57.734563 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.119585 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:57.735863 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:57.650688 13344 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:57.746438 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:58.030056 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 205842us
I20260709 13:17:58.100642 10205 raft_consensus.cc:606] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:57.778211  9953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.014s	sys 0.012s
W20260709 13:17:58.101155 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.122802 10205 raft_consensus.cc:614] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:58.122793 10126 raft_consensus.cc:2410] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:17:58.123661  9953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.021s	sys 0.017s
I20260709 13:17:58.103360 10124 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:58.125067 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:57.833387 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.011s	sys 0.012s
I20260709 13:17:58.120335 10127 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:17:58.126511 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.018s	sys 0.015s
I20260709 13:17:58.129699 10056 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:17:57.771986 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:58.131737 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.133078 10131 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:58.097162 13666 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.025s	user 0.009s	sys 0.005s
I20260709 13:17:58.140537 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.117640 13627 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.145704 10122 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:58.098088 13661 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:57.734593 13196 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.144280 10050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.010s	sys 0.013s
I20260709 13:17:57.735371 13451 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:58.153720 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:17:58.161010 10050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.011s	sys 0.013s
I20260709 13:17:57.966135 13476 consensus_queue.cc:1243] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:17:58.180612 10200 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:17:58.068005 13682 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:57.933261 13555 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.014s	sys 0.012s
I20260709 13:17:58.166837 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.156507 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.042125 13701 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.445s
I20260709 13:17:58.156605 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.213725 10122 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:58.215047 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.215891 10122 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:17:58.181780 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.217365 10126 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7. Re-sending same reply.
I20260709 13:17:58.212779 10131 raft_consensus.cc:2376] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8. Re-sending same reply.
I20260709 13:17:58.181703 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:57.790869 13536 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.372s
I20260709 13:17:58.218544 10124 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:17:58.198242 10201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.031s	sys 0.020s
W20260709 13:17:58.219437 10201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.032s	sys 0.023s
I20260709 13:17:58.140561 13693 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.150262 13688 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:58.233650 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.242216 10125 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:58.238551 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.121460 13427 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:58.240654 10895 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.002s
I20260709 13:17:58.243507 10122 raft_consensus.cc:2376] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9. Re-sending same reply.
I20260709 13:17:58.245406 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.031774 13675 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.498s
I20260709 13:17:58.247008 10131 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:17:58.247193 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.248155 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:58.212544 13555 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.019s	sys 0.013s
I20260709 13:17:58.254591 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:17:58.256387 13665 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.257093 13665 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.142841 13398 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.265480 13398 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.265725 13718 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.142640 13596 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.266343 13718 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:17:58.267329 10130 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:17:58.268934 13718 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:58.270277 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 129958us
I20260709 13:17:58.271355 10095 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:58.272832 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:58.273216 13427 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.181497 13704 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:58.274456 13704 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.277657 13704 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:58.222254 13655 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:17:58.279991 13670 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.280364 13644 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.281189 13685 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:58.282047 13672 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.098491 13674 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.290725 13475 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.291327 13696 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:17:58.297652 10019 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.530s (SENT)
I20260709 13:17:58.218597 13689 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.300082 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.301076 10122 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:58.298445 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.302641 10124 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:58.303534 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.304790 10122 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:58.308173 13664 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.309358 10125 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:58.312603 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 347) took 1707 ms (client timeout 1790 ms). Trace:
I20260709 13:17:58.312992 13451 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:58.313347 10202 rpcz_store.cc:269] 0709 13:17:56.605313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:56.605588 (+   275us) service_pool.cc:224] Handling call
0709 13:17:58.312171 (+1706583us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540039dd54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:58.312488 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360604032}
I20260709 13:17:58.314350 13451 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:17:58.323489 13427 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:17:58.324079 13427 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:58.324997 13427 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:58.325943 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.326077 13427 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:17:58.328001 13596 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.332056 10128 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:58.334153 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.332885 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:58.334662 10127 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:58.336428 10128 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:58.162556 13627 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.342303 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.347640 10126 raft_consensus.cc:2376] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8. Re-sending same reply.
I20260709 13:17:58.348960 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.349923 10127 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:17:58.350605 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:58.352530 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 231193us
W20260709 13:17:58.353804 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.355768 10122 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:58.353991 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.359894 10124 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:58.156055 13666 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.372777 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.215775 13196 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.328759 13596 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:58.412632 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.347306 13689 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.323570 13356 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.343353 13718 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:58.413805 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.407191 13702 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.416329 10127 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:58.441545 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.442389 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:17:58.444401 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:58.467672 10124 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:17:58.461040 10128 raft_consensus.cc:2376] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8. Re-sending same reply.
W20260709 13:17:58.473594 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 175952us
I20260709 13:17:58.476454 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.370085 13664 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.441785 13464 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.785s	user 0.012s	sys 0.000s
I20260709 13:17:58.239761 13682 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.481294 10125 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:17:58.482422 10131 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:17:58.474699 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.538s (SENT) [suppressed 44 similar messages]
I20260709 13:17:58.443735 13667 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.512403 10128 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:17:58.466722 13670 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.513958 10126 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:17:58.516868 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:58.520336 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164898us
I20260709 13:17:58.477789 13665 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.533679 10125 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
W20260709 13:17:58.534358 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.253489 13693 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.480731 13704 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:17:58.560775 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44069.9us
I20260709 13:17:58.574214 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:17:58.574354 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99707.5us
W20260709 13:17:58.575359 10019 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.554s (SENT)
I20260709 13:17:58.576005 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:58.560393 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 287626us
W20260709 13:17:58.510831 13690 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.846s	user 0.013s	sys 0.000s
W20260709 13:17:58.508306 13630 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.122s	user 0.014s	sys 0.001s
I20260709 13:17:58.584510 10131 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:17:58.531644 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:58.585650 10122 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:58.265511 13688 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.526907 13713 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.603343 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2816) took 1269 ms. Trace:
I20260709 13:17:58.603453 10122 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:58.613618 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:58.614264 10125 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:17:58.614571 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.627175 10122 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:17:58.493587 13664 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.493927 13464 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:17:58.622972 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.535004 13654 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:58.560144 13356 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:58.485376 13702 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.625603 10196 rpcz_store.cc:276] 0709 13:17:57.327827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.328014 (+   187us) service_pool.cc:224] Handling call
0709 13:17:57.343666 (+ 15652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:57.515998 (+172332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:57.516046 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:57.539872 (+ 23826us) log.cc:844] Serialized 26 byte log entry
0709 13:17:57.737194 (+197322us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:57.737276 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:57.738708 (+  1432us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.596907 (+858199us) raft_consensus.cc:1685] finished
0709 13:17:58.596933 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.597144 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21686,"raft.queue_time_us":112551,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":5491,"thread_start_us":321386,"threads_started":2,"wal-append.queue_time_us":775235}
I20260709 13:17:58.527441 13667 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.660681 13356 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.560053 13670 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.635355 13704 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:17:58.661572 13712 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:17:58.672926 10897 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:17:58.676990 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102904us
W20260709 13:17:58.673198 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 97886.6us
W20260709 13:17:58.680406 10019 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.581s (SENT)
I20260709 13:17:58.608151 13630 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.602625 13690 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:58.693149 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161595us
I20260709 13:17:58.694913 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:58.697994 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.643337 13713 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.664150 13720 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:17:58.717581 10123 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:58.732087 13682 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.733043 13733 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.663302 13729 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:17:58.735047 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.000s	sys 0.022s
W20260709 13:17:58.735687 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.001s	sys 0.022s
I20260709 13:17:58.737324 13356 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.739091 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:58.720877 13664 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:58.740411 13666 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:58.743683 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.744910 13704 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:17:58.754026 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 600) took 1087 ms. Trace:
W20260709 13:17:58.756462 13711 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:17:58.754719 10895 rpcz_store.cc:276] 0709 13:17:57.666532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.667946 (+  1414us) service_pool.cc:224] Handling call
0709 13:17:57.668411 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:57.669076 (+   665us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:17:58.249047 (+579971us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.579282 (+330235us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:58.579811 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:17:58.579917 (+   106us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:58.581002 (+  1085us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.581308 (+   306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.581394 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:58.753678 (+172284us) raft_consensus.cc:1685] finished
0709 13:17:58.753708 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.753913 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":159484,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":6935,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":45341,"thread_start_us":117935,"threads_started":2,"wal-append.queue_time_us":441435}
I20260709 13:17:58.734059 13733 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.720132 13627 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.767198 13356 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:17:58.771746 13733 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.774627 13733 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.787658 13196 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.790510 13464 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:58.792316 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2849) took 1189 ms (client timeout 1581 ms). Trace:
I20260709 13:17:58.662154 13654 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.792671 13733 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:58.793510 13596 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:58.801870 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 165352us
W20260709 13:17:58.832432 10130 rpcz_store.cc:269] 0709 13:17:57.602782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.620482 (+ 17700us) service_pool.cc:224] Handling call
0709 13:17:58.792015 (+1171533us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540036b854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:58.792175 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388072832}
W20260709 13:17:58.830632 10863 connection.cc:256] RPC call timeout handler was delayed by -1.2160218809999606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2425s for a response
I20260709 13:17:58.846043 13737 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:17:58.848652 13451 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.002s
I20260709 13:17:58.848935 13693 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:58.842926 13596 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:17:58.852074 13367 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.176s	user 0.014s	sys 0.000s
I20260709 13:17:58.852779 13367 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:58.854027 13596 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.842988 13682 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:17:58.770823 13702 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:58.844630 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 105614us
W20260709 13:17:58.861474 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 181116us
W20260709 13:17:58.862258 10019 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.583s (SENT)
I20260709 13:17:58.862725 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:58.715999 13720 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.879696 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:58.880631 10047 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:17:58.880491 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.881397 13367 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.881484 10054 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:58.882001 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:58.882928 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.883373 13723 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:58.883893 10054 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:17:58.859495 13356 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:58.883534 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:58.885964 10049 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:58.888128 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:17:58.888762 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1703) took 1672 ms. Trace:
I20260709 13:17:58.889317 10050 rpcz_store.cc:276] 0709 13:17:57.216329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.222192 (+  5863us) service_pool.cc:224] Handling call
0709 13:17:57.242805 (+ 20613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:57.260356 (+ 17551us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:17:57.613335 (+352979us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.887930 (+1274595us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.888233 (+   303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:58.888302 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:58.888393 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.888492 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:58.888659 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":725665,"raft.queue_time_us":138178,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":256,"spinlock_wait_cycles":1597184,"thread_start_us":534060,"threads_started":1,"wal-append.queue_time_us":860510}
I20260709 13:17:58.891633  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:58.892211 13574 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.896961 13689 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:58.898185 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet 2c485d13f66d4ce8bdca72b60d242842: Illegal state: Replica d51ad94710c348889d417f7f00032948 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:17:58.858067 13690 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:58.901233 10897 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 599) took 1235 ms (client timeout 1507 ms). Trace:
W20260709 13:17:58.903420 13706 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:17:58.884352 13723 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:58.915777 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.910058 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2840) took 1345 ms. Trace:
I20260709 13:17:58.907505 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:17:58.932313 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 70079.8us
W20260709 13:17:58.938556 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67742.6us
W20260709 13:17:58.957185 10897 rpcz_store.cc:269] 0709 13:17:57.666109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.666294 (+   185us) service_pool.cc:224] Handling call
0709 13:17:58.900911 (+1234617us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54001e5854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:58.901087 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2292361472}
I20260709 13:17:58.945520 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:17:58.948113 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.965824 10130 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:58.962929 10123 rpcz_store.cc:276] 0709 13:17:57.565173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.585220 (+ 20047us) service_pool.cc:224] Handling call
0709 13:17:58.900206 (+1314986us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:58.909935 (+  9729us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544135424}
I20260709 13:17:58.974638 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:17:58.965660 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.987697 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:58.990159 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.869426 13720 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:58.913455 13630 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:58.994774 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.906913 13344 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:17:58.983561 10863 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.059s (SENT)
I20260709 13:17:58.912633 13456 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:58.998359 10054 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:17:58.956383 13692 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.962417 13627 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.912253 13588 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:17:59.027148 10049 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:59.007615 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:59.033643 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.034283 10123 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:17:59.000164 10019 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.660s (SENT)
I20260709 13:17:58.997910 13682 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:59.009390 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:58.965030 10125 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:58.974987 13723 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.051059 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:17:59.052093 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77596.2us
I20260709 13:17:59.052510 10054 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:17:59.044762 10048 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:17:59.046635 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:59.036187 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:58.965361 13537 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.937390 13702 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:17:59.054847 10053 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:59.002499 13196 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:59.072999 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.017318 13667 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:59.079753 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73788.7us
I20260709 13:17:59.082201 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:17:58.957070 13574 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.055526 13635 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.044159 13588 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:59.097404 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:17:59.102612 13663 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:59.103502 13663 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.104106 13663 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.106788 13737 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:17:59.107416 13737 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.107494 13690 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:59.108011 13737 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:59.068495 13356 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.113135 13576 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.054486 13729 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:59.114475 13729 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.022951 13456 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.119428 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:17:59.155393 13664 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:17:59.104768 13673 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.827s	user 0.007s	sys 0.040s
I20260709 13:17:59.155750 10049 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:59.059526 13718 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:59.059960 13573 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:59.071686 13719 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:59.172161 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.175341 10200 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:59.071410 13537 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.177202 13654 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:59.079387 13344 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:59.184201 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.156726 13664 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:17:59.179291 10056 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:59.115783 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:59.197329 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 197217us
W20260709 13:17:59.168648 13427 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.842s	user 0.011s	sys 0.010s
I20260709 13:17:59.050951 13627 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.204150 13627 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:59.212193 10049 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:17:59.201371 10131 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:59.201581 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.205323 10203 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:17:59.206538 10891 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 13:17:59.199143 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 585687us
I20260709 13:17:59.214520 10200 raft_consensus.cc:2410] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:17:59.098778 13735 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:17:59.080196 13196 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.043874 13692 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.060479 13682 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.156183 13733 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:17:59.211409 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 228243us
I20260709 13:17:59.169358 13718 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:17:59.208901 10019 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.669s (SENT)
I20260709 13:17:59.221567 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:17:59.075021 10201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.001s	sys 0.012s
I20260709 13:17:59.225368 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:17:59.211625 13706 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:17:59.200150 13664 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:59.252194 10202 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.251796 10125 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:17:59.251957 10201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.001s	sys 0.013s
I20260709 13:17:59.254082 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.254249 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:17:59.264334 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.264897 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.248906 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.259615 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:59.260336 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:59.208741 13719 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:17:59.267350 10203 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:59.268023 10054 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:59.270390 10200 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:17:59.207784 13713 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:59.275219 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.278131 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.180449 13573 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.162603 13635 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.207703 13344 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.283353 13719 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.335093 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:59.337146 10130 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:17:59.337527 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.338193 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:59.279838 13646 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:17:59.289734 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:59.294575 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1631) took 1635 ms. Trace:
I20260709 13:17:59.298774 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.340531 10205 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:17:59.298967 10200 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:17:59.300812 10123 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:17:59.303069 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 81636us
I20260709 13:17:59.284902 13733 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:17:59.273240 13234 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:17:59.330008 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.281234 10131 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:17:59.399050 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:59.340036 10127 rpcz_store.cc:276] 0709 13:17:57.659086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.479492 (+820406us) service_pool.cc:224] Handling call
0709 13:17:58.490737 (+ 11245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.491086 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:58.634494 (+143408us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.038882 (+404388us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.056898 (+ 18016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.056974 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:59.057042 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.057140 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.294271 (+237131us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54002ef250. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:17:59.294473 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10039,"prepare.queue_time_us":268542,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":85,"raft.queue_time_us":11285,"raft.run_wall_time_us":15211,"spinlock_wait_cycles":33595008,"thread_start_us":516079,"threads_started":2}
W20260709 13:17:59.407025 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.236505 13735 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.416647 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2837) took 1884 ms. Trace:
I20260709 13:17:59.417105 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:59.299499 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:17:59.299301 13680 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:17:59.433110 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79382us
W20260709 13:17:59.434111 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 225291us
W20260709 13:17:59.435155 10019 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.670s (SENT)
I20260709 13:17:59.435750 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:59.428750 10129 rpcz_store.cc:276] 0709 13:17:57.459847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.460012 (+   165us) service_pool.cc:224] Handling call
0709 13:17:57.469214 (+  9202us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:57.511440 (+ 42226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:57.511476 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:57.552072 (+ 40596us) log.cc:844] Serialized 52 byte log entry
0709 13:17:57.817224 (+265152us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:58.346686 (+529462us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.408574 (+ 61888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:58.408708 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:59.343451 (+934743us) raft_consensus.cc:1685] finished
0709 13:17:59.343475 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.343647 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":935522,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":375714,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":135572,"thread_start_us":652890,"threads_started":2,"wal-append.queue_time_us":945317}
I20260709 13:17:59.438771 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.341774 13646 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.439790 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.450565 10129 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:59.439643 10205 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:59.440091  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:59.255935 13588 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.443965 13646 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:17:59.444406 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:17:59.489506 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 53761.5us
I20260709 13:17:59.491998 13743 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:59.493002 13743 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.496244 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:17:59.341183 13456 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:59.399557 13690 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:17:59.497056 10122 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:17:59.497659 10202 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:17:59.508100 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:17:59.470311 13680 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.509861 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.510555 13721 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:17:59.510969 10205 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:17:59.511237 13721 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:17:59.512377 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:17:59.513006 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:17:59.513288 10205 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:17:59.513815 13680 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:59.514283 13537 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:59.512411 13721 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:17:59.527292 10203 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:59.497529 13689 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:17:59.528599 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.009s	sys 0.024s
I20260709 13:17:59.439061 13729 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:59.446934 13630 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:59.529174 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.009s	sys 0.024s
I20260709 13:17:59.530215 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:17:59.543653 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:17:59.545567 10130 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:17:59.536230 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 454089us
W20260709 13:17:59.541559 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 444292us
I20260709 13:17:59.492046 13746 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:17:59.443570 13574 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:59.488802 13664 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:17:59.783428 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.955587 10199 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.003s	sys 0.012s
W20260709 13:17:59.962433 10199 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.003s	sys 0.012s
W20260709 13:17:59.963292 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.954661 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 433030us
I20260709 13:17:59.559510 10202 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:17:59.965544 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:17:59.717152 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.497627 13744 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:17:59.605602 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 322) took 2185 ms. Trace:
W20260709 13:17:59.801909 13749 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:17:59.955430 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:17:59.948678 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 334451us
W20260709 13:17:59.823282 10898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.005s	sys 0.018s
W20260709 13:18:00.006376 10898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.005s	sys 0.022s
I20260709 13:18:00.006588 10122 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:17:59.682940 10200 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:17:59.597185 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162113us
I20260709 13:17:59.999632 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:00.007798 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:59.719070 13720 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.674s	user 0.016s	sys 0.004s
W20260709 13:17:59.644110 13661 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.458s	user 0.013s	sys 0.012s
W20260709 13:17:59.519474 13681 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.011s	sys 0.048s
I20260709 13:17:59.676260 13682 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:17:59.877321 13723 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.807s	user 0.019s	sys 0.001s
W20260709 13:18:00.012645 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.946993 13588 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:00.001219 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:00.016242 10019 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.685s (SENT)
I20260709 13:18:00.031363 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:17:59.550426 13663 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:59.677222 13635 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:59.591741 13630 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:17:59.825680 13743 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:17:59.642904 13735 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:17:59.677814 13344 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:17:59.642948 13196 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.026387 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.049923 10122 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:17:59.872740 13646 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:00.023872 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.036878 10130 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:17:59.548091 13451 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.489s
W20260709 13:18:00.021231 13681 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.012s	sys 0.053s
I20260709 13:17:59.733842 13725 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:00.067270 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58942.3us
W20260709 13:18:00.045065 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.011s	sys 0.017s
I20260709 13:18:00.013751 13574 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.035992 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:17:59.983006 10055 rpcz_store.cc:276] 0709 13:17:57.356369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.356767 (+   398us) service_pool.cc:224] Handling call
0709 13:17:57.376866 (+ 20099us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:57.377158 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:57.413700 (+ 36542us) log.cc:844] Serialized 14 byte log entry
0709 13:17:58.856151 (+1442451us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b1800457660. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:17:58.856459 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.540845 (+684386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.540937 (+    92us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:59.541008 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.541111 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:59.541333 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1454792,"prepare.queue_time_us":1473484,"prepare.run_cpu_time_us":4403,"prepare.run_wall_time_us":25644,"raft.queue_time_us":141148,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":17901,"spinlock_wait_cycles":2419072,"thread_start_us":699272,"threads_started":2,"wal-append.queue_time_us":53001,"wal-append.run_cpu_time_us":2613,"wal-append.run_wall_time_us":1212002}
I20260709 13:18:00.044138 13682 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.058107 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1652) took 1013 ms. Trace:
W20260709 13:18:00.065482 10197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.007s	sys 0.022s
I20260709 13:18:00.021988 13437 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.077730 10130 rpcz_store.cc:276] 0709 13:17:59.044663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.543275 (+498612us) service_pool.cc:224] Handling call
0709 13:18:00.058022 (+514747us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:00.034118 13669 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:00.068307 10019 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.717s (SENT)
I20260709 13:18:00.017979 13720 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.040122 13664 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.011341 13744 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:00.062583 10895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.029s	sys 0.038s
I20260709 13:18:00.090543 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.070559 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:00.079608 10894 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.015s	sys 0.031s
W20260709 13:18:00.092455 10894 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.015s	sys 0.032s
I20260709 13:18:00.092777 10130 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:00.038030 13723 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.027335 13231 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:18:00.094542 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1700) took 2889 ms. Trace:
I20260709 13:18:00.047207 13686 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:00.068729 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.011s	sys 0.018s
I20260709 13:18:00.049288 13692 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:00.082968 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59253.4us
I20260709 13:18:00.033401 13688 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:18:00.071202 10891 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:18:00.084249 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48365us
I20260709 13:18:00.116272 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:00.085973 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 60216.9us
I20260709 13:18:00.086546 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:00.007334 13746 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.083535 13437 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.120298 13646 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.135914 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:00.091089 10895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.030s	sys 0.038s
I20260709 13:18:00.139549 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.140318 10130 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:00.078852 10197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.022s
W20260709 13:18:00.058597 10205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.032s	sys 0.018s
W20260709 13:18:00.141085 10205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.032s	sys 0.018s
I20260709 13:18:00.141222 13744 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.141346 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.142253 10130 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:00.143445 13720 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.145570 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:00.146695 13752 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:00.147420 13752 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:00.148471 13752 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.082842 13682 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.153929 13682 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:00.068390 13737 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.960s	user 0.022s	sys 0.000s
I20260709 13:18:00.065035 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:00.080257 13574 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.098281 10122 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 5.
I20260709 13:18:00.081064 13669 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.202093 13669 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:00.202327 13680 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:00.204591 13723 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.089669 13476 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:00.111375 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.207396 13476 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:00.207888 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96647.6us
I20260709 13:18:00.208145 13476 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.097256 10052 rpcz_store.cc:276] 0709 13:17:57.205125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.218135 (+ 13010us) service_pool.cc:224] Handling call
0709 13:17:57.224945 (+  6810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:57.274376 (+ 49431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:57.274419 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:58.801647 (+1527228us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b2c000014c8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:17:58.834554 (+ 32907us) log.cc:844] Serialized 26 byte log entry
0709 13:17:59.290368 (+455814us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:17:59.296556 (+  6188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.296672 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.088731 (+792059us) raft_consensus.cc:1685] finished
0709 13:18:00.088753 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.094466 (+  5713us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27471,"raft.queue_time_us":95690,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":9704,"spinlock_wait_cycles":6155648,"thread_start_us":442466,"threads_started":1,"wal-append.queue_time_us":885667}
I20260709 13:18:00.211541 13720 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:00.212175 13720 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:00.213325 13720 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.136852 13676 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:00.214567 13676 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:00.215622 13676 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.216879 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.217556 10131 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.218055 13476 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.124260 13746 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.129112 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 294) took 1883 ms. Trace:
I20260709 13:18:00.131795 13630 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.047435 13689 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:00.158351 13737 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.143429 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:00.166025 13745 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.822s	user 0.002s	sys 0.000s
W20260709 13:18:00.171620 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 119903us
I20260709 13:18:00.240244 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.201339 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.241294 10050 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:00.241421 10122 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:00.207489 13670 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.089921 13344 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:00.241959 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.234874 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.232255 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.234508 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.235958 10130 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:00.222231 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 153947us
W20260709 13:18:00.245293 10019 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.720s (SENT)
I20260709 13:18:00.245783 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:00.197753 13682 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:00.198542 13741 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.700s	user 0.003s	sys 0.000s
I20260709 13:18:00.219421 10126 rpcz_store.cc:276] 0709 13:17:58.245399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.601086 (+355687us) service_pool.cc:224] Handling call
0709 13:17:58.608187 (+  7101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.049540 (+441353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.049596 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.069401 (+ 19805us) log.cc:844] Serialized 26 byte log entry
0709 13:17:59.727419 (+658018us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:59.727496 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.727625 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.128718 (+401093us) raft_consensus.cc:1685] finished
0709 13:18:00.128754 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.128998 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67738,"raft.queue_time_us":107626,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":68891,"thread_start_us":851401,"threads_started":2,"wal-append.queue_time_us":881664}
I20260709 13:18:00.200440 13574 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:00.229190 13757 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 13:18:00.210886 13751 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.439s	user 0.002s	sys 0.009s
I20260709 13:18:00.261937 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.262678 10055 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:00.167574 13692 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:18:00.256721 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:00.269906 10047 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.256837 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:00.258257 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 373877us
I20260709 13:18:00.260669 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.236526 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:00.270812 10054 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:00.271872 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.272841 10126 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.248208 13344 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.275857 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.276911 10052 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:00.283646 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.285917 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.271607 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:00.259404 13680 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:00.317037 10048 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.318893 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:00.319778 10054 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.309604 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.315944 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.325021 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:00.392290 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 147011us
I20260709 13:18:00.265901 13670 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:00.393457 10019 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.762s (SENT)
I20260709 13:18:00.342710 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:00.395615 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63425.3us
I20260709 13:18:00.394169 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:00.398339 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55740.2us
I20260709 13:18:00.396770 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.430488 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:00.399452 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:00.433674 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.431540 10048 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:00.278998 13735 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:00.432835 10122 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.445837 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:00.441433 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44781.3us
I20260709 13:18:00.442149 10055 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:00.281790 13344 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:00.449656 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.752s (SENT)
I20260709 13:18:00.450590 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:00.452544 13746 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:00.454726 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.798s (SENT)
I20260709 13:18:00.449730 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.455531 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:00.456394 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.456736 13746 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:00.457939 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.447005 10122 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.460691 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.461602 10122 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.444415 10052 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:00.470798 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2823) took 1171 ms. Trace:
I20260709 13:18:00.471572 10199 rpcz_store.cc:276] 0709 13:17:59.299346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.299614 (+   268us) service_pool.cc:224] Handling call
0709 13:17:59.335641 (+ 36027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.335881 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:17:59.962977 (+627096us) log.cc:844] Serialized 14 byte log entry
0709 13:18:00.470249 (+507272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:00.470334 (+    85us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:00.470414 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.470519 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.470703 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31733,"prepare.queue_time_us":784770,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"thread_start_us":1005927,"threads_started":2}
I20260709 13:18:00.391722 13630 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:00.391670 13196 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:00.488204 10047 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
W20260709 13:18:00.521839 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128438us
I20260709 13:18:00.522179 10122 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:18:00.525039 10054 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.283464 13689 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.196s
I20260709 13:18:00.437649 13691 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:00.526691 10054 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:00.526855 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:00.489933 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:00.530535 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.070s (SENT)
I20260709 13:18:00.531311 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:00.525198 10131 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.435639 13682 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:00.538429 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:00.539105 10865 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.029s	sys 0.068s
I20260709 13:18:00.541697 10049 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:00.542757 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:00.578991 10865 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.031s	sys 0.068s
W20260709 13:18:00.580405 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 568265us
W20260709 13:18:00.580727 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1728) took 1616 ms (client timeout 2099 ms). Trace:
I20260709 13:18:00.565267 10054 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:00.527592 13680 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:00.474896 13723 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:00.559655 10052 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:00.562700 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 357) took 2928 ms. Trace:
I20260709 13:18:00.586239 10201 rpcz_store.cc:276] 0709 13:17:57.625452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.625641 (+   189us) service_pool.cc:224] Handling call
0709 13:17:57.626044 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:58.220066 (+594022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.220115 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:58.294538 (+ 74423us) log.cc:844] Serialized 26 byte log entry
0709 13:17:59.263874 (+969336us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:17:59.263935 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.264043 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.163804 (+899761us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:00.520936 (+357132us) raft_consensus.cc:1685] finished
0709 13:18:00.520970 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.553741 (+ 32771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41059,"raft.queue_time_us":83126,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":26065,"spinlock_wait_cycles":391479680,"thread_start_us":1335527,"threads_started":2,"wal-append.queue_time_us":860294}
I20260709 13:18:00.437937 13670 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.444731 13735 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:00.581359 10056 rpcz_store.cc:269] 0709 13:17:58.964166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.003583 (+ 39417us) service_pool.cc:224] Handling call
0709 13:18:00.580420 (+1576837us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b54002e9354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:00.580626 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186724736}
I20260709 13:18:00.579332 10131 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:00.479046 13742 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.039s	user 0.001s	sys 0.015s
I20260709 13:18:00.583809 10122 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8. Re-sending same reply.
I20260709 13:18:00.535825 13691 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.540100 13746 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:00.551682 13744 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:00.609506 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78996.2us
I20260709 13:18:00.568594 13753 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.489050 13630 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:00.623274 10204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.040s	sys 0.084s
W20260709 13:18:00.643728 10204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.040s	sys 0.084s
I20260709 13:18:00.644773 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2829) took 1214 ms. Trace:
I20260709 13:18:00.578991 13682 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.644265 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.649300 10122 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:00.642911 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 395893us
W20260709 13:18:00.627416 13718 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.349s	user 0.016s	sys 0.001s
I20260709 13:18:00.585690 13196 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:00.618247 13735 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.591495 13723 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:00.623864 13691 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:00.645251 10204 rpcz_store.cc:276] 0709 13:17:59.430542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.457706 (+ 27164us) service_pool.cc:224] Handling call
0709 13:17:59.487172 (+ 29466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.487514 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:00.011224 (+523710us) log.cc:844] Serialized 14 byte log entry
0709 13:18:00.644284 (+633060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:00.644354 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:00.644407 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.644510 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.644687 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":633438,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"thread_start_us":1117620,"threads_started":2}
W20260709 13:18:00.632416 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.023s	sys 0.051s
I20260709 13:18:00.640072 13753 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.673188  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:00.669921 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.023s	sys 0.051s
I20260709 13:18:00.647332 13686 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:00.656525 13682 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.657992 13196 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:00.642596 13451 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.017s	sys 0.058s
I20260709 13:18:00.645488 13744 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.667405 13723 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:00.660287 13719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.018s	sys 0.024s
W20260709 13:18:00.705049 13766 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.417s	user 0.001s	sys 0.000s
W20260709 13:18:00.710323 13719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.023s	sys 0.026s
I20260709 13:18:00.691285 13721 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:00.701717 13451 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.017s	sys 0.058s
I20260709 13:18:00.677801 13753 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:00.702423 13765 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.443s	user 0.001s	sys 0.001s
W20260709 13:18:00.715077 13760 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.434s	user 0.000s	sys 0.001s
I20260709 13:18:00.715286 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 430) took 1434 ms. Trace:
I20260709 13:18:00.716575 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:00.717072 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 194402us
W20260709 13:18:00.706022 13664 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.616s	user 0.012s	sys 0.002s
I20260709 13:18:00.719780 10131 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:00.714763 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 292) took 2568 ms. Trace:
I20260709 13:18:00.664019 13669 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:00.753187 10019 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.968s (SENT)
I20260709 13:18:00.754696 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:00.744565 10122 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:00.748273 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 310) took 1674 ms. Trace:
I20260709 13:18:00.664566 13718 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:00.744221 10891 rpcz_store.cc:276] 0709 13:17:59.281164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.289366 (+  8202us) service_pool.cc:224] Handling call
0709 13:18:00.714985 (+1425619us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:00.715215 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680553856}
W20260709 13:18:00.742942 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 343522us
I20260709 13:18:00.757380 10131 rpcz_store.cc:276] 0709 13:17:59.074029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.542317 (+1468288us) service_pool.cc:224] Handling call
0709 13:18:00.748185 (+205868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:00.760283 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.761611 10131 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.748172 10128 rpcz_store.cc:276] 0709 13:17:58.145345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.533054 (+387709us) service_pool.cc:224] Handling call
0709 13:17:58.601317 (+ 68263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:58.794348 (+193031us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:17:58.801323 (+  6975us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.342643 (+541320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.342729 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:17:59.342777 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.712993 (+1370216us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001c8254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:00.713045 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.713248 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13012,"prepare.queue_time_us":7081,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3385,"raft.queue_time_us":69980,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":52995,"spinlock_wait_cycles":443283840,"thread_start_us":475931,"threads_started":1,"wal-append.queue_time_us":835861,"wal-append.run_cpu_time_us":1719,"wal-append.run_wall_time_us":1001717}
I20260709 13:18:00.759755 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:00.764184 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:00.786124 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 135657us
I20260709 13:18:00.791808 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2831) took 1301 ms. Trace:
W20260709 13:18:00.762952 13762 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:18:00.792548 10205 rpcz_store.cc:276] 0709 13:17:59.491002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.514294 (+ 23292us) service_pool.cc:224] Handling call
0709 13:17:59.514738 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.531450 (+ 16712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.531487 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.535646 (+  4159us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.142315 (+606669us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:00.142362 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.142431 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.791452 (+649021us) raft_consensus.cc:1685] finished
0709 13:18:00.791486 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.791714 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2362,"raft.queue_time_us":15663,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":60406,"thread_start_us":605497,"threads_started":1,"wal-append.queue_time_us":712769}
I20260709 13:18:00.797945 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.798566 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.798990 10128 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.799296 10122 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:00.727141 13721 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:00.802264 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.770345 13664 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:00.793324 13663 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:00.825414 13689 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:00.830052 13771 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:18:00.831087 13782 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:18:00.831880 13770 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:18:00.832618 13669 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.836009 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 431) took 1534 ms. Trace:
I20260709 13:18:00.850169 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 428) took 1629 ms. Trace:
I20260709 13:18:00.835677 13663 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.811991 13664 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:00.848466 10048 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:00.854082 10898 rpcz_store.cc:276] 0709 13:17:59.221182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.221343 (+   161us) service_pool.cc:224] Handling call
0709 13:17:59.255946 (+ 34603us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:17:59.269792 (+ 13846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.269851 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:17:59.283816 (+ 13965us) log.cc:844] Serialized 52 byte log entry
0709 13:18:00.007026 (+723210us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:00.007089 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.007222 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.849837 (+842615us) raft_consensus.cc:1685] finished
0709 13:18:00.849862 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.850070 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1688,"raft.queue_time_us":3943,"spinlock_wait_cycles":16918912,"thread_start_us":722656,"threads_started":1,"wal-append.queue_time_us":916115}
I20260709 13:18:00.859182 13663 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:00.853111 10894 rpcz_store.cc:276] 0709 13:17:59.301600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.443218 (+141618us) service_pool.cc:224] Handling call
0709 13:17:59.443802 (+   584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.444256 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.444301 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.444912 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.092942 (+648030us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:00.560805 (+467863us) log.cc:844] Serialized 14 byte log entry
0709 13:18:00.561120 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.561232 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.835719 (+274487us) raft_consensus.cc:1685] finished
0709 13:18:00.835741 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.835923 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":598431,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3797,"raft.queue_time_us":2236,"raft.run_cpu_time_us":367,"raft.run_wall_time_us":21323,"thread_start_us":1111830,"threads_started":2,"wal-append.queue_time_us":783038}
I20260709 13:18:00.860420 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1729) took 1856 ms. Trace:
I20260709 13:18:00.899834 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.863420 13670 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.902863 13792 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:00.864797 13778 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:18:00.865983 13689 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:00.802902 13790 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:18:00.872882 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 603) took 1697 ms. Trace:
I20260709 13:18:00.880539 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.843638 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.942265 10122 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:00.899737 10053 rpcz_store.cc:276] 0709 13:17:59.003883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.072373 (+ 68490us) service_pool.cc:224] Handling call
0709 13:17:59.116778 (+ 44405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.179339 (+ 62561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.179390 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.227007 (+ 47617us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.101061 (+874054us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:00.101102 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.101176 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.860133 (+758957us) raft_consensus.cc:1685] finished
0709 13:18:00.860158 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.860344 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1695,"raft.queue_time_us":10435,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":12733,"thread_start_us":809813,"threads_started":1,"wal-append.queue_time_us":1019434}
I20260709 13:18:00.942752 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:00.895453 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84726.2us
I20260709 13:18:00.901247 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.880630 13744 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.905366 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.905846 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:00.948148 10129 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.906985 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:00.937654 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:00.894107 13664 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:00.940071 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.906226 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.948323 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.949141 10204 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:00.951378 10202 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:00.947369 10197 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:00.941596 10897 rpcz_store.cc:276] 0709 13:17:59.175045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.177178 (+  2133us) service_pool.cc:224] Handling call
0709 13:17:59.177869 (+   691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.197975 (+ 20106us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:17:59.202999 (+  5024us) log.cc:844] Serialized 14 byte log entry
0709 13:17:59.976484 (+773485us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b18003da980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:18:00.444045 (+467561us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:00.491292 (+ 47247us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.491531 (+   239us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:00.741146 (+249615us) log.cc:844] Serialized 14 byte log entry
0709 13:18:00.741638 (+   492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.741836 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.872487 (+130651us) raft_consensus.cc:1685] finished
0709 13:18:00.872521 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.872773 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3468,"prepare.queue_time_us":229781,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":2451,"raft.queue_time_us":24133,"raft.run_cpu_time_us":527,"raft.run_wall_time_us":18550,"spinlock_wait_cycles":18878976,"thread_start_us":702922,"threads_started":2,"wal-append.queue_time_us":1464691}
I20260709 13:18:00.953887 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:00.954152 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.955217 10203 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:00.955390 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 202241us
I20260709 13:18:00.906248 13630 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:00.956866 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.997s (SENT)
I20260709 13:18:00.938894 13689 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.955278 10122 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.959830 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:00.961714 10199 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:00.965405 13721 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:00.962363 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.966485 10129 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:00.967730 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.959666 13785 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:00.968490 13785 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:00.969916 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.963724 13795 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:00.970801 13795 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:00.971946 13795 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.969709 13785 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:00.973017 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 172547us
I20260709 13:18:00.973400 13795 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:00.974040 13785 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.975147 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.975548 13790 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:00.975945 10123 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:00.976140 13790 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:00.976636 13790 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:00.977257 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.978017 10123 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:00.968650 10129 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:00.981092 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.975872 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.983280 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.984308 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.984778 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 360) took 1884 ms. Trace:
I20260709 13:18:00.984619 10128 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:00.985258 10122 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:00.988543 13476 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:00.986613 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:00.990662 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:00.985677 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1656) took 1022 ms. Trace:
I20260709 13:18:00.985258 10196 rpcz_store.cc:276] 0709 13:17:59.100129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.100299 (+   170us) service_pool.cc:224] Handling call
0709 13:17:59.105493 (+  5194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:59.488874 (+383381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.488914 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:59.488978 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:17:59.489018 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.984271 (+1495253us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:00.984493 (+   222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.984702 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881054592}
I20260709 13:18:00.991329 10127 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:00.984992 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:00.997115 13574 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.000s
I20260709 13:18:00.997710 13574 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:00.992652 10128 rpcz_store.cc:276] 0709 13:17:59.963356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.982723 (+1019367us) service_pool.cc:224] Handling call
0709 13:18:00.985592 (+  2869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.001277 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:00.979050 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.005705 10123 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:00.987720 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1642) took 2072 ms. Trace:
I20260709 13:18:01.005919 10127 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:01.007025 10129 rpcz_store.cc:276] 0709 13:17:58.914902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.986567 (+2071665us) service_pool.cc:224] Handling call
0709 13:18:00.986911 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:00.987195 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:00.987225 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:00.987276 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:00.987316 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.987378 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.987487 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.010550 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1647) took 2031 ms. Trace:
I20260709 13:18:01.010236 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1648) took 2016 ms. Trace:
I20260709 13:18:01.011580 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.007699 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.011246 10127 rpcz_store.cc:276] 0709 13:17:58.978987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.008998 (+2030011us) service_pool.cc:224] Handling call
0709 13:18:01.009785 (+   787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.010089 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.010128 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.010193 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:01.010244 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.010320 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.010463 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20480}
I20260709 13:18:01.012178 10129 rpcz_store.cc:276] 0709 13:17:58.993990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.009259 (+2015269us) service_pool.cc:224] Handling call
0709 13:18:01.009649 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.009890 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.009916 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.009960 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:01.009996 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.010050 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.010168 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.014299 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1644) took 2055 ms. Trace:
I20260709 13:18:01.014730 10128 rpcz_store.cc:276] 0709 13:17:58.959007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.000216 (+2041209us) service_pool.cc:224] Handling call
0709 13:18:01.013747 (+ 13531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.013986 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.014043 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.014083 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:01.014106 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.014149 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.014240 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180992}
W20260709 13:18:01.015364 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54331.7us
I20260709 13:18:00.999711 13689 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:01.020814 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.020898 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.013038 10898 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:01.024549 10196 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:01.025954 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.023705 10897 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:01.030392 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:01.031387 10129 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:01.041090 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.045629 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.049490 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:01.050534 10122 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:18:01.049489 10199 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:01.032024 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet 806f35e66c234d569cbafd8b3981518f: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 5
I20260709 13:18:01.057166 13646 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:01.057876 13646 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:01.031522 10128 raft_consensus.cc:2376] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8. Re-sending same reply.
I20260709 13:18:01.048936 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.064958 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.044997 10123 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:01.072844 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1654) took 1872 ms. Trace:
I20260709 13:18:01.073230 10123 rpcz_store.cc:276] 0709 13:17:59.200534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.022677 (+1822143us) service_pool.cc:224] Handling call
0709 13:18:01.044376 (+ 21699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.072557 (+ 28181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.072784 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:01.071846 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet dfacd39092d24ba5a33d0cd9451b91a1: Illegal state: Replica d51ad94710c348889d417f7f00032948 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:01.068100 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 308) took 1994 ms. Trace:
I20260709 13:18:01.075232 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:01.017112 13796 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:01.076901 13753 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.078975 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1655) took 1863 ms. Trace:
I20260709 13:18:01.089648 10123 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
W20260709 13:18:01.077387 13344 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.003s
I20260709 13:18:01.084162 10128 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8. Re-sending same reply.
I20260709 13:18:01.093021 13663 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.094285 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:01.146584 10125 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:18:01.148058 10125 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:01.096601 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.097219 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:01.098192 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:01.150094 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52990.2us
I20260709 13:18:01.099174 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.089444 10129 rpcz_store.cc:276] 0709 13:17:59.073313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.022276 (+1948963us) service_pool.cc:224] Handling call
0709 13:18:01.022951 (+   675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.051181 (+ 28230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.068026 (+ 16845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.100167 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.101011 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.103443 13794 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:01.108701 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 151891us
I20260709 13:18:01.109746 13737 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.119038 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.120507 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.124083 13573 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.124455 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.126197 10126 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:01.127146 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.127676 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.128677 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.129606 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:01.130093 13780 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.373s	user 0.001s	sys 0.001s
I20260709 13:18:01.129627 13718 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.130884 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.180492 13744 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:01.180809 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.181144 13744 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.181594 13790 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.182148 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.138362 10196 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:01.184118 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.184965 10196 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:01.185909 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.187204 10128 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:01.187816 13574 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.188053 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1666) took 1032 ms. Trace:
I20260709 13:18:01.188484 10128 rpcz_store.cc:276] 0709 13:18:00.155931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.146259 (+990328us) service_pool.cc:224] Handling call
0709 13:18:01.164529 (+ 18270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.187713 (+ 23184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.187981 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.189026 13744 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.159637 13667 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.173382 13794 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:01.190078 13667 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:01.191190 13667 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.139438 10122 rpcz_store.cc:276] 0709 13:17:59.215879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.077905 (+1862026us) service_pool.cc:224] Handling call
0709 13:18:01.078335 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.078564 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.078599 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.078655 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:01.078695 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.078757 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.078905 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.153831 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.192883 10897 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.156692 13344 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.195518 13344 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:01.195879 13691 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.001s
I20260709 13:18:01.196367 13691 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.162256 13723 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.202028 13663 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.203558 10200 raft_consensus.cc:2368] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:01.203806 13681 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:01.204226 13681 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:01.205272 13681 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.206132 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.206583 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.207453 10205 raft_consensus.cc:2368] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:18:01.209443 10202 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:01.135567 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 140424us
I20260709 13:18:01.159089 10898 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:01.159493 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:01.212100 10123 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:01.160547 10198 raft_consensus.cc:2368] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
W20260709 13:18:01.170907 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43897.7us
I20260709 13:18:01.153544 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:01.213651 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:01.214774 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:01.157204 10019 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.644s (SENT)
I20260709 13:18:01.215440 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:01.215636 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.206885 10200 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:01.176522 13664 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.139806 13796 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.218343 10197 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:01.191061 13794 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.160938 13682 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.210750 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:01.220476 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 67002.7us
I20260709 13:18:01.170881 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1662) took 1121 ms. Trace:
I20260709 13:18:01.220997 10196 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:01.178740 13234 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:01.211967 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:01.206754 13681 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.231104 13682 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.231948 13574 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:01.232488 13574 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:01.232966 13574 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:01.221347 10126 rpcz_store.cc:276] 0709 13:18:00.049962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.125425 (+1075463us) service_pool.cc:224] Handling call
0709 13:18:01.125787 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.170526 (+ 44739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.170809 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:01.236272 10126 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:18:01.236742 10126 pending_rounds.cc:85] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
I20260709 13:18:01.237085 10126 pending_rounds.cc:107] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:18:01.237365 10126 raft_consensus.cc:2889] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:01.242043 10126 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:01.243578 13723 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.244632 13753 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.248564 13691 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.250442 13790 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.251539 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.252605 10204 raft_consensus.cc:2410] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:01.256412 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.259620 10895 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:01.260383 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.227845 10095 leader_election.cc:400] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:01.224977 13234 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.261093 10095 leader_election.cc:403] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:01.224749 13796 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.225236 13680 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:01.216500 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:01.267808 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51356us
I20260709 13:18:01.269708 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:01.273061 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 115933us
I20260709 13:18:01.269047 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.279145 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:01.223615 10199 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:01.281620 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.264698 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.285501 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.226577 13720 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:01.255738 13663 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:01.256031 13344 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.293990 10894 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:01.323426 10126 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:01.323398 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.263935 13234 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.265182 13690 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:01.334750 10200 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:01.291661 13691 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:01.355482 13691 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:01.356272 13691 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.323351 13720 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:01.357131 13720 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.341403 13686 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:01.289390 13682 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:01.263933 13725 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
W20260709 13:18:01.337247 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 73949.4us
I20260709 13:18:01.292814 13753 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:01.336858 13661 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.323994 13723 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.361754 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:01.370555 13196 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.002s
I20260709 13:18:01.369692 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.380089 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:01.377786 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 273071us
I20260709 13:18:01.400575 10200 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:01.361052 13789 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:01.353353 13680 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.347014 13344 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:01.363365 13740 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.928s	user 0.001s	sys 0.004s
I20260709 13:18:01.428984 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:01.396390 10205 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:01.323650 13663 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.363191 13476 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:18:01.396790 10199 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:01.389643 10894 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:01.380085 13686 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.407936 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:01.447685 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 192533us
I20260709 13:18:01.409507 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:01.363395 13690 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:01.385705 13682 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.386471 13752 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.390704 13753 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.426584 10130 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:01.414238 13723 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:01.441020 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:01.452823 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:01.427479 13735 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.758s	user 0.015s	sys 0.000s
I20260709 13:18:01.427687 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:01.466930 10020 connection.cc:256] RPC call timeout handler was delayed by -1.5194928159999108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:18:01.467960 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:01.428793 13789 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:01.440637 13663 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.412410 13196 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.437578 13536 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.459439 13682 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.474395 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.492241 10048 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:01.398320 13661 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.471493 13737 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:01.488724 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59919.8us
I20260709 13:18:01.499081 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:01.489357 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.489368 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.501566 10051 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:01.489605 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:01.502556 10894 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:01.467453 13725 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:01.518731 10201 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
W20260709 13:18:01.489837 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80456.8us
I20260709 13:18:01.490254 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.521404 10056 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
W20260709 13:18:01.490836 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.424986 13680 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.540091 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 288) took 3878 ms. Trace:
I20260709 13:18:01.497028 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.469731 13753 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.511484 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1651) took 2488 ms. Trace:
I20260709 13:18:01.473520 10897 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:01.453819 13689 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.471662 13800 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.530967 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.553357 10049 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8. Re-sending same reply.
I20260709 13:18:01.469540 13752 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.537514 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.558554 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.540864 10124 rpcz_store.cc:276] 0709 13:17:57.682736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.480352 (+797616us) service_pool.cc:224] Handling call
0709 13:17:58.508968 (+ 28616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:58.654441 (+145473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:58.654491 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:58.664486 (+  9995us) log.cc:844] Serialized 26 byte log entry
0709 13:17:59.056485 (+391999us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:17:59.056554 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:59.056672 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.202966 (+2146294us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:01.527321 (+324355us) raft_consensus.cc:1685] finished
0709 13:18:01.527360 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.539974 (+ 12614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96744,"raft.queue_time_us":225428,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":102362,"spinlock_wait_cycles":530852736,"thread_start_us":449137,"threads_started":2,"wal-append.queue_time_us":582220}
I20260709 13:18:01.545372 10127 rpcz_store.cc:276] 0709 13:17:59.023612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.013224 (+1989612us) service_pool.cc:224] Handling call
0709 13:18:01.016249 (+  3025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.016487 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:01.359503 (+343016us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.510906 (+151403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:01.510991 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:01.511065 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.511205 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.511387 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6252,"prepare.queue_time_us":397137,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3407,"raft.queue_time_us":47085,"thread_start_us":454121,"threads_started":2}
I20260709 13:18:01.471732 13735 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.520445 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.561182 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.495842 13536 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:01.568534 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 69629.3us
I20260709 13:18:01.564728 10054 raft_consensus.cc:2410] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:01.565979 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:01.564020 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43736.2us
I20260709 13:18:01.569492 10051 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:01.569790 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:01.570674 10048 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:01.571245 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.573319 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.574316 10055 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:01.554464 13776 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.605820 10203 raft_consensus.cc:2368] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:01.572069 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.607453 10048 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:01.552830 13744 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.607759 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.608304 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.576828 13746 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.609426 13630 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:18:01.680122 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:01.680972 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.682188 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:01.609877 13772 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.796s	user 0.003s	sys 0.001s
W20260709 13:18:01.609480 13804 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:01.610997 13234 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:01.615815 13805 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:01.696393 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.696811 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.697913 10127 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:01.621066 13536 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.622339 13789 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:01.628400 13806 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:18:01.604202 13800 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.640823 13196 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.641101 13752 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.643270 13680 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:01.723381 13661 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.724570 13234 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.725152 13809 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:01.725867 13809 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.645077 13689 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:01.728310 13809 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:01.568702 13721 raft_consensus.cc:2764] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:01.552838 13344 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.615085 13691 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:01.573984 13795 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.600s	user 0.014s	sys 0.003s
I20260709 13:18:01.731079 13646 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:01.655231 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47011.1us
I20260709 13:18:01.733114 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.733246 13800 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.656049 10054 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.656641 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:01.736554 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80025.4us
I20260709 13:18:01.650801 13735 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.675211 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:01.739643 13815 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:01.740790 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65726.9us
I20260709 13:18:01.702343 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.742002 10124 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.742846 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.743361 13752 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:01.744225 10095 leader_election.cc:400] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:01.744567 10095 leader_election.cc:403] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:18:01.742816 13815 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:01.749478 13646 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:01.768798 13691 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:01.775852 13744 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:01.777330 13744 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.777982 13744 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:01.782557 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47479.9us
I20260709 13:18:01.784399 13667 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:01.785249 13667 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:01.562597 13723 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.605068 13725 raft_consensus.cc:2764] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:18:01.711560 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:01.714483 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.793709 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.794718 10047 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:01.565126 13690 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:01.735776 13800 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:01.736680 13536 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:01.795755 13536 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:01.796228 13536 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.739270 13692 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:01.705020 10130 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:01.788937 13774 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:01.789361 13746 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.797818 10048 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:01.707341 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.698491 13510 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:01.787724 13231 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:01.809799 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54827.4us
I20260709 13:18:01.829613 10200 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:01.829929 10205 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:01.830447 10196 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:01.817368 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.819636 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:01.832262 10205 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:01.832597 10048 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.834263 10127 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:01.822992 10054 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:01.824120 10124 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:18:01.825259 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.825848 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:01.836483 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:01.836752 10051 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:01.837427 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.844923 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:01.846489 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.844929 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.848773 10201 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.845288 10204 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:01.852027 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.853340 10205 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:01.816308 13574 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.583s	user 0.016s	sys 0.000s
I20260709 13:18:01.849934 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.871250 10130 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:01.871380 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:01.856026 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.873905 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.873027 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:01.875088 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 406426us
I20260709 13:18:01.877080 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:01.877012 10125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.006s
I20260709 13:18:01.826915 13510 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:01.894019 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.904378 10204 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:01.900815 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.896508 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:01.901321 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.902485 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 316) took 1220 ms. Trace:
I20260709 13:18:01.911290 10128 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:01.901225 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:01.905328 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:01.786607 13667 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:01.851754 13719 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.888911 10054 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:01.923870 10201 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:01.929039 10048 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:01.897446 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.911199 10125 rpcz_store.cc:276] 0709 13:18:00.682233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.149140 (+466907us) service_pool.cc:224] Handling call
0709 13:18:01.154103 (+  4963us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:01.173405 (+ 19302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.173446 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:01.179810 (+  6364us) log.cc:844] Serialized 52 byte log entry
0709 13:18:01.333791 (+153981us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:01.879234 (+545443us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.900719 (+ 21485us) log.cc:844] Serialized 14 byte log entry
0709 13:18:01.901916 (+  1197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:01.902056 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:01.902122 (+    66us) raft_consensus.cc:1685] finished
0709 13:18:01.902142 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:01.902375 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27098,"prepare.queue_time_us":164777,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":10386,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":15944,"thread_start_us":667162,"threads_started":2,"wal-append.queue_time_us":258438}
I20260709 13:18:01.935523 13800 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:01.936154 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63234.3us
I20260709 13:18:02.024590 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.024820 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.909219 13510 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:02.020565 13804 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:18:01.885357 13574 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:02.076864 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 241977us
W20260709 13:18:02.207758 13796 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.924s	user 0.015s	sys 0.008s
I20260709 13:18:02.021406 13682 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.238529 10130 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:02.199888 13792 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:02.220331 13720 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.863s	user 0.010s	sys 0.014s
I20260709 13:18:02.195284 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:01.893288 13811 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.143s
I20260709 13:18:02.034461 13721 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:18:02.196024 10047 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:18:02.196625 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 452538us
I20260709 13:18:01.823978 13476 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:18:02.208513 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.935931 10891 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:02.208760 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:02.280279 10125 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:01.894315 13737 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:02.214792 10122 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:02.332091 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53381.9us
I20260709 13:18:02.218752 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:02.333524 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 114870us
I20260709 13:18:02.336037 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:01.999763 13196 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:18:02.206202 13752 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:02.207054 13809 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:02.238425 10128 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:02.120702 10052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.031s	sys 0.065s
W20260709 13:18:02.349803 10052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.031s	sys 0.068s
I20260709 13:18:02.240062 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:02.218369 13753 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.009s
W20260709 13:18:02.212559 13663 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.009s
I20260709 13:18:02.207351 13669 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:02.246654 10055 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.643s	user 0.014s	sys 0.010s
I20260709 13:18:02.003722 13820 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 13:18:02.248445 10126 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:02.450282 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 220159us
I20260709 13:18:02.248624 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:01.935902 13802 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.287s
I20260709 13:18:01.908113 13686 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.251821 10201 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:02.256845 10095 leader_election.cc:400] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:02.258816 13574 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:02.259498 10196 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:02.260388 13828 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:02.346599 13692 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:02.471026 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.255697 13789 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.207223 13719 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.263588 13231 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:02.264635 13437 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:02.264781 10123 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:02.488826 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:02.489645 10123 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:02.256448 13720 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.492623 10197 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:02.451072 13663 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.493832 13720 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:02.268287 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:02.460747 13669 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:02.497187 13669 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:02.306115 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.468175 13735 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:02.499954 13735 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:02.500675 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.501356 13735 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.319283 10198 raft_consensus.cc:2368] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12.
I20260709 13:18:02.320179 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.503567 10047 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.503767 13737 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:02.246634 13796 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:02.342247 10056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.042s	sys 0.054s
I20260709 13:18:02.351418 10049 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.353811 10125 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:02.355491 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.278959 13800 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:02.356544 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:02.263687 13811 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:02.444196 10055 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:01.937649 13744 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:02.451664 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.456736 10122 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:02.464375 10095 leader_election.cc:403] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:02.255319 13792 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.473522 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.448077 13794 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:02.479724 13808 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.946s	user 0.006s	sys 0.003s
W20260709 13:18:02.533455 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.016s	sys 0.044s
W20260709 13:18:02.583539 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.016s	sys 0.044s
W20260709 13:18:02.583961 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 707299us
W20260709 13:18:02.533813 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 177379us
I20260709 13:18:02.587143 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.587945 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:02.446987 13809 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:02.517243 10047 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:02.539944 10056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.047s	sys 0.063s
I20260709 13:18:02.543519 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1729) took 1003 ms. Trace:
W20260709 13:18:02.604899 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 253) took 1509 ms (client timeout 1826 ms). Trace:
I20260709 13:18:02.604835 10055 rpcz_store.cc:276] 0709 13:18:01.540825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.552080 (+ 11255us) service_pool.cc:224] Handling call
0709 13:18:02.543055 (+990975us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:02.553795 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:02.581740 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 325076us
W20260709 13:18:02.605394 10891 rpcz_store.cc:269] 0709 13:18:01.095624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.099822 (+  4198us) service_pool.cc:224] Handling call
0709 13:18:02.604703 (+1504881us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54001e64d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:02.604841 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437529984}
I20260709 13:18:02.606935 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:02.584884 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:02.538529 10048 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:02.509270 13231 consensus_queue.cc:1243] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:02.452345 13753 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.712544 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.713073 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.640899 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:02.713769 10047 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:02.660128 10126 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:02.712648 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.533546 13692 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:02.715554 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 130825us
I20260709 13:18:02.713472 10055 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:02.539222 13733 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:02.541893 13811 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:02.470717 13774 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:18:02.756713 10051 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:02.551255 13437 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:02.759574 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.777702 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.778231 10048 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:02.778370 10054 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:02.760501 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119674us
I20260709 13:18:02.764775 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.779870 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.770587 10049 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:02.772511 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.758493 13720 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:02.509668 13789 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.781734 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.621560 13824 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:02.796519 10047 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:02.555982 13800 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.798271 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.820073 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:02.780591 10050 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:02.801982 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.796177 13733 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:02.804019 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 317) took 2110 ms. Trace:
I20260709 13:18:02.763348 13826 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:02.782032 13646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.010s	sys 0.025s
I20260709 13:18:02.816900 10126 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:02.826521 10055 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:02.827872 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.797715 10049 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:02.588888 13794 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:02.839843 13834 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:18:02.797185 13720 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:02.842283 13733 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:02.828920 10129 rpcz_store.cc:276] 0709 13:18:00.693794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.154265 (+460471us) service_pool.cc:224] Handling call
0709 13:18:01.154612 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.162765 (+  8153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.162821 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:01.219150 (+ 56329us) log.cc:844] Serialized 26 byte log entry
0709 13:18:01.398022 (+178872us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:01.702342 (+304320us) log.cc:844] Serialized 14 byte log entry
0709 13:18:02.056781 (+354439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.056882 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:02.262808 (+205926us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b180023b000. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:02.432313 (+169505us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:02.803667 (+371354us) raft_consensus.cc:1685] finished
0709 13:18:02.803701 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.803927 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43909,"prepare.queue_time_us":171581,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":13675,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":119602,"spinlock_wait_cycles":21851904,"thread_start_us":742621,"threads_started":3,"wal-append.queue_time_us":230357}
I20260709 13:18:02.843220 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1756) took 1051 ms. Trace:
W20260709 13:18:02.844913 13536 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.048s	user 0.012s	sys 0.002s
I20260709 13:18:02.845517 13789 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:02.846722 13510 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:18:02.717128 13196 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.183s
I20260709 13:18:02.847043 13686 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:02.999176 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:02.914537 10054 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:03.000245 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1738) took 1059 ms. Trace:
I20260709 13:18:02.851580 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:02.966648 10897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.004s	sys 0.009s
I20260709 13:18:02.878134 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:03.003360 10897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.006s	sys 0.009s
I20260709 13:18:03.003623 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:03.004277 10047 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:02.876027 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 424832us
W20260709 13:18:02.967280 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.004s	sys 0.009s
W20260709 13:18:03.009123 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.004s	sys 0.011s
I20260709 13:18:02.957151 10051 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:18:03.009589 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 676220us
I20260709 13:18:03.000741 10054 rpcz_store.cc:276] 0709 13:18:01.941503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.713471 (+771968us) service_pool.cc:224] Handling call
0709 13:18:03.000159 (+286688us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:03.002115 10052 raft_consensus.cc:2376] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11. Re-sending same reply.
I20260709 13:18:03.033794 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.034631 10048 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:03.034703 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.035454 10055 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:03.037358 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.000308 10056 rpcz_store.cc:276] 0709 13:18:01.791796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.795259 (+  3463us) service_pool.cc:224] Handling call
0709 13:18:01.795725 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:01.796164 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:01.796201 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:01.796876 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.604514 (+807638us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:02.604560 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:02.604642 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:02.842918 (+238276us) raft_consensus.cc:1685] finished
0709 13:18:02.842941 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:02.843142 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2902,"raft.queue_time_us":8537,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11746,"thread_start_us":807194,"threads_started":1,"wal-append.queue_time_us":784529}
I20260709 13:18:03.042559 10047 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:03.046790 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:03.045194 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.048264 10054 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:03.047010 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.049300 10047 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:03.001294 13646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.017s	sys 0.045s
I20260709 13:18:03.053645 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.056778 10048 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:18:03.004930 13752 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.031s	sys 0.027s
I20260709 13:18:03.057765 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.054777 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.069511 10052 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:03.069902 10049 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:18:02.892485 13808 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:03.072702 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.031913 13720 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:03.001999 13682 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:02.997570 13718 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:03.075053 10050 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:03.044018 13789 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:03.072907 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 466063us
W20260709 13:18:03.072016 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.900063 13824 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.180s
I20260709 13:18:03.034346 13663 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:03.075565 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.054175 13818 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:03.094376 10054 raft_consensus.cc:2376] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:18:03.078615 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.083750 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.096416 10050 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
W20260709 13:18:03.068750 13752 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.031s	sys 0.027s
I20260709 13:18:03.089591 10055 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
W20260709 13:18:03.077883 13746 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.233s	user 0.012s	sys 0.004s
I20260709 13:18:03.070942 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.033265 13510 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.098224 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.091730 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.102003 10055 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:03.086905 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:03.109633 10052 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:03.076143 13733 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:03.033265 13536 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:03.108090 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.013s	sys 0.015s
I20260709 13:18:03.083895 13718 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:03.123106 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.013s	sys 0.015s
I20260709 13:18:03.090955 13789 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:03.130256 10047 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:03.125663 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:02.966818 13692 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.035897 13826 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:03.133188 13820 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.013s	sys 0.027s
W20260709 13:18:03.171605 13820 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.014s	sys 0.027s
I20260709 13:18:03.170125 13718 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.167423 13733 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.176350 13733 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:03.170274 13794 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:03.166601 13789 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:03.153301 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.179391 10050 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:03.100905 13752 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:03.186100 13752 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.093536 13646 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:18:03.222751 10096 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.000s	sys 0.024s
W20260709 13:18:03.226399 10096 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.000s	sys 0.024s
W20260709 13:18:03.226804 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 663815us
I20260709 13:18:03.223868 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.231833 10055 raft_consensus.cc:2376] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11. Re-sending same reply.
I20260709 13:18:03.196693 13792 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:03.196563 13733 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:03.209252 10204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.002s	sys 0.021s
W20260709 13:18:03.362974 10204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.003s	sys 0.022s
I20260709 13:18:03.109915 13682 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:03.110018 13746 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:03.102002 13818 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.190433 13843 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:03.109825 13824 consensus_queue.cc:1243] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:18:03.354072 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.011s	sys 0.023s
I20260709 13:18:03.033829 13811 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.364042 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2845) took 1491 ms. Trace:
W20260709 13:18:03.511565 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.011s	sys 0.095s
W20260709 13:18:03.573024 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.011s	sys 0.101s
I20260709 13:18:03.573122 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:03.569757 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:03.543617 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.016s	sys 0.022s
W20260709 13:18:03.577991 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.016s	sys 0.023s
W20260709 13:18:03.580765 10096 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.519s (SENT)
W20260709 13:18:03.581892 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.527s (SENT) [suppressed 43 similar messages]
I20260709 13:18:03.575198 10049 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:03.364284 10202 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:03.569861 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:03.583576 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.017s	sys 0.025s
I20260709 13:18:03.599517 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:03.594872 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.604130 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 837807us
I20260709 13:18:03.604486 10052 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:03.605515 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.584582 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1757) took 1250 ms. Trace:
W20260709 13:18:03.582911 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:03.609628 10897 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.000s	sys 0.014s
W20260709 13:18:03.610078 10897 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.001s	sys 0.014s
I20260709 13:18:03.611310 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1752) took 1349 ms. Trace:
I20260709 13:18:03.611955 10052 rpcz_store.cc:276] 0709 13:18:02.261655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.599216 (+1337561us) service_pool.cc:224] Handling call
0709 13:18:03.611212 (+ 11996us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:03.602542 13437 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.008s	sys 0.015s
I20260709 13:18:03.445235 13820 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:03.493255 13774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.019s	sys 0.083s
W20260709 13:18:03.634388 13774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.021s	sys 0.092s
I20260709 13:18:03.635171 13774 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:03.363298 13794 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.639503 13794 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.642239 13794 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:03.583680 10204 rpcz_store.cc:276] 0709 13:18:01.872561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.921770 (+ 49209us) service_pool.cc:224] Handling call
0709 13:18:01.922241 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.922495 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:02.247557 (+325062us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.363598 (+1116041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:03.363665 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:03.363730 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.363814 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.363976 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":569412,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"thread_start_us":1168108,"threads_started":2}
W20260709 13:18:03.620126 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.564s (SENT)
I20260709 13:18:03.606606 10049 rpcz_store.cc:276] 0709 13:18:02.334497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.109492 (+774995us) service_pool.cc:224] Handling call
0709 13:18:03.584505 (+475013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:03.427238 13733 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:03.458433 13682 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:03.581943 13719 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:03.631011 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.571641 13800 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:03.645182 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:03.645449 10056 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:03.569381 13718 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:03.631503 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.597440 13826 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:03.655086 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.662436 10052 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:03.663477 10049 raft_consensus.cc:2368] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
W20260709 13:18:03.665848 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 274604us
I20260709 13:18:03.594635 13790 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:03.609891 13792 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:03.684226 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:03.700666 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1765) took 1149 ms. Trace:
I20260709 13:18:03.701160 10054 rpcz_store.cc:276] 0709 13:18:02.551778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.605383 (+1053605us) service_pool.cc:224] Handling call
0709 13:18:03.700578 (+ 95195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:03.643342 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:03.649181 13820 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:03.620548 13824 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:03.727725 13840 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.727898 13510 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:03.631585 13437 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.008s	sys 0.015s
I20260709 13:18:03.729573 13437 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:03.730904 13437 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.699940 13794 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.727959 13824 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.735793 13794 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.425113 13789 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.643810 13746 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:03.696484 13733 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.709017 13682 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:03.714104 13720 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.000s
I20260709 13:18:03.714538 13792 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.706638 13790 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.717386 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1696) took 1900 ms. Trace:
W20260709 13:18:03.632718 10129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.009s	sys 0.008s
W20260709 13:18:03.721894 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 116975us
I20260709 13:18:03.647094 13774 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:18:03.719180 10052 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:03.846343 10129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.011s	sys 0.013s
I20260709 13:18:03.846513 10125 rpcz_store.cc:276] 0709 13:18:01.817367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.535916 (+718549us) service_pool.cc:224] Handling call
0709 13:18:02.536349 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:03.044348 (+507999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.044385 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:03.071556 (+ 27171us) log.cc:844] Serialized 26 byte log entry
0709 13:18:03.583103 (+511547us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:03.583175 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.583264 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.717132 (+133868us) raft_consensus.cc:1685] finished
0709 13:18:03.717160 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.717321 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14482,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":64529,"thread_start_us":946149,"threads_started":2,"wal-append.queue_time_us":616418}
W20260709 13:18:03.724588 13667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.006s	sys 0.011s
I20260709 13:18:03.855356 10204 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:03.771886 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 151803us
I20260709 13:18:03.847839 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:03.851922 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:03.866762 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 435) took 2026 ms. Trace:
I20260709 13:18:03.684963 13855 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:03.868505 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:03.840127 13860 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:03.871896 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 296303us
I20260709 13:18:03.874177 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:03.874691 10095 leader_election.cc:400] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:18:03.874956 10125 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:03.723869 13821 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.462s	user 0.000s	sys 0.010s
I20260709 13:18:03.867262 10897 rpcz_store.cc:276] 0709 13:18:01.839927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.840146 (+   219us) service_pool.cc:224] Handling call
0709 13:18:01.840601 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:01.840909 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:02.121513 (+280604us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.610699 (+1489186us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.866347 (+255648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:03.866412 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:03.866461 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.866541 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.866693 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1127849,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"raft.queue_time_us":555502,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":147,"thread_start_us":1895904,"threads_started":5,"wal-append.queue_time_us":297936,"wal-append.run_cpu_time_us":1374,"wal-append.run_wall_time_us":1008305}
I20260709 13:18:03.869530 13828 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:03.738898 13536 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:03.707266 13719 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:03.870173 10054 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:03.783421 13848 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:03.879616 10204 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 7
I20260709 13:18:03.738334 13794 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:03.887802 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1776) took 1649 ms. Trace:
I20260709 13:18:03.866084 13746 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:03.865525 13667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.011s	sys 0.011s
I20260709 13:18:03.861526 10052 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:03.898082 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1758) took 2105 ms. Trace:
I20260709 13:18:03.898579 10052 rpcz_store.cc:276] 0709 13:18:01.792350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.860660 (+2068310us) service_pool.cc:224] Handling call
0709 13:18:03.861155 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.897737 (+ 36582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.898014 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:03.899853 10865 connection.cc:256] RPC call timeout handler was delayed by -2.050788189000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:18:03.871013 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 637477us
I20260709 13:18:03.894107 10054 rpcz_store.cc:276] 0709 13:18:02.237573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.726790 (+1489217us) service_pool.cc:224] Handling call
0709 13:18:03.886308 (+159518us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:03.873781 10096 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.638s (SENT)
I20260709 13:18:03.891088 10204 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:18:03.878095 13776 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:03.901335 10204 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:03.708083 13846 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.198s
I20260709 13:18:03.901809 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:03.916808 10125 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:03.728477 13840 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.882462 13855 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:03.922535 13818 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:03.887554 13789 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:03.923588 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 329) took 1643 ms. Trace:
I20260709 13:18:03.927013 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1759) took 2134 ms. Trace:
I20260709 13:18:03.944131 10054 rpcz_store.cc:276] 0709 13:18:01.792618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.925417 (+2132799us) service_pool.cc:224] Handling call
0709 13:18:03.925832 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.926443 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.926477 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:03.926525 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:03.926714 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.926789 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.926923 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3165}
I20260709 13:18:03.928144 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1695) took 2111 ms. Trace:
I20260709 13:18:03.943989 10129 rpcz_store.cc:276] 0709 13:18:02.279813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.007449 (+727636us) service_pool.cc:224] Handling call
0709 13:18:03.007829 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:03.008201 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.008234 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:03.034091 (+ 25857us) log.cc:844] Serialized 26 byte log entry
0709 13:18:03.852730 (+818639us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:03.852775 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.852849 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.923200 (+ 70351us) raft_consensus.cc:1685] finished
0709 13:18:03.923226 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.923498 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187746,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":180,"thread_start_us":768656,"threads_started":1,"wal-append.queue_time_us":840944}
I20260709 13:18:03.864097 13720 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.946189 10130 rpcz_store.cc:276] 0709 13:18:01.817088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.497772 (+680684us) service_pool.cc:224] Handling call
0709 13:18:02.498304 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.498928 (+   624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.498963 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.540986 (+ 42023us) log.cc:844] Serialized 26 byte log entry
0709 13:18:03.573652 (+1032666us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:03.573710 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.573818 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.927894 (+354076us) raft_consensus.cc:1685] finished
0709 13:18:03.927916 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.928073 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8744,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":51646,"thread_start_us":814717,"threads_started":1,"wal-append.queue_time_us":1313731}
I20260709 13:18:03.953012 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:03.889845 13851 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:03.952149 13846 consensus_queue.cc:1243] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:03.961145 13746 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.883903 13860 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.963841 13855 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:03.965794 10054 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:03.902984 13820 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.969560 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2841) took 2114 ms. Trace:
I20260709 13:18:03.991144 10196 rpcz_store.cc:276] 0709 13:18:01.855029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.904094 (+ 49065us) service_pool.cc:224] Handling call
0709 13:18:01.904452 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.466364 (+561912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.969152 (+1502788us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:03.969455 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815922048}
I20260709 13:18:03.992156 13860 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:03.974184 13848 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:03.994678 10125 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:03.995030 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1762) took 2201 ms. Trace:
I20260709 13:18:03.996430 10054 rpcz_store.cc:276] 0709 13:18:01.793347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.945957 (+2152610us) service_pool.cc:224] Handling call
0709 13:18:03.965427 (+ 19470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.994714 (+ 29287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.994964 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:03.997941 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1763) took 2204 ms. Trace:
W20260709 13:18:03.969842 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96153.4us
I20260709 13:18:03.892820 13719 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:03.976683 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
I20260709 13:18:04.000869 10095 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:04.002525 10095 leader_election.cc:400] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:04.003566 13848 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.002960 10095 leader_election.cc:403] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:18:04.007295 10125 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:04.008489 13846 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:03.977993 13733 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:03.981393 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:04.016888 13733 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:04.017531 13846 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:04.017669 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:04.017987 13846 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:04.019204 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:04.018916 13846 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.020157 13846 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:03.998199 10054 rpcz_store.cc:276] 0709 13:18:01.793638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.997268 (+2203630us) service_pool.cc:224] Handling call
0709 13:18:03.997567 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.997707 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.997722 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:03.997751 (+    29us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:03.997770 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.997805 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.997894 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:03.999703 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.731s (SENT)
I20260709 13:18:04.024744 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.026153 10054 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:18:04.026546 10096 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.806s (SENT)
W20260709 13:18:04.027665 10096 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.822s (SENT)
I20260709 13:18:03.965890 13842 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:04.029245 13842 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:04.028473 13789 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:04.030737 10205 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:18:04.030776 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.032445 13789 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:04.032553 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:04.033582 10205 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:04.034374 10096 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.826s (SENT)
I20260709 13:18:04.034773 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:04.035368 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.885s (SENT)
I20260709 13:18:04.035677 10205 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:04.035871 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1764) took 2242 ms. Trace:
I20260709 13:18:04.036358 10054 rpcz_store.cc:276] 0709 13:18:01.793912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.024750 (+2230838us) service_pool.cc:224] Handling call
0709 13:18:04.025248 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.026946 (+  1698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.035797 (+  8851us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.038698 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1765) took 2244 ms. Trace:
I20260709 13:18:04.053617 10054 rpcz_store.cc:276] 0709 13:18:01.794175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.037426 (+2243251us) service_pool.cc:224] Handling call
0709 13:18:04.037897 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.038153 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.038205 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.038275 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:04.038338 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.038425 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.038614 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.056545 13720 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:04.035892 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:04.045639 10095 connection.cc:256] RPC call timeout handler was delayed by -2.4358098879999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:18:04.048753 10897 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:04.060173 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.092484 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.096290 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.098121 10054 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:04.098152 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1766) took 2303 ms. Trace:
I20260709 13:18:04.099701 10053 rpcz_store.cc:276] 0709 13:18:01.794457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.058271 (+2263814us) service_pool.cc:224] Handling call
0709 13:18:04.097563 (+ 39292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.097783 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.097809 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.097868 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:04.097900 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.097955 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.098072 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.101269 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65966.2us
W20260709 13:18:04.102130 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.890s (SENT)
W20260709 13:18:04.103102 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.904s (SENT)
I20260709 13:18:04.103716 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:04.103910 10095 connection.cc:256] RPC call timeout handler was delayed by -2.1595098880000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 13:18:04.104439 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1769) took 2292 ms. Trace:
W20260709 13:18:04.106839 10096 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.911s (SENT)
I20260709 13:18:04.107070 10054 rpcz_store.cc:276] 0709 13:18:01.812339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.060888 (+2248549us) service_pool.cc:224] Handling call
0709 13:18:04.097794 (+ 36906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.098629 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.104354 (+  5725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.108415 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:04.110715 13871 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:18:04.112771 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1697) took 2295 ms. Trace:
I20260709 13:18:04.113276 10123 rpcz_store.cc:276] 0709 13:18:01.817620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.570423 (+752803us) service_pool.cc:224] Handling call
0709 13:18:02.639773 (+ 69350us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:03.151660 (+511887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.151950 (+   290us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:03.596076 (+444126us) log.cc:844] Serialized 78 byte log entry
0709 13:18:03.690952 (+ 94876us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:03.691014 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.691145 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.112444 (+421299us) raft_consensus.cc:1685] finished
0709 13:18:04.112469 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.112661 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":394269,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":15188,"thread_start_us":495557,"threads_started":2,"wal-append.queue_time_us":269718}
I20260709 13:18:04.114379 13849 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.115442 13826 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.115204 13849 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.116009 13826 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.117805 13849 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.118185 13826 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.118499 13849 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:04.118757 13826 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.108558 13718 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:04.120182 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3340215929999886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:18:04.121093 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:04.126191 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.936s (SENT)
I20260709 13:18:04.133767 10053 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:04.139443 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.126472 10054 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:04.139931 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1736) took 2300 ms. Trace:
I20260709 13:18:04.140348 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1777) took 1885 ms. Trace:
I20260709 13:18:04.140264 10053 rpcz_store.cc:276] 0709 13:18:01.839370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.100569 (+2261199us) service_pool.cc:224] Handling call
0709 13:18:04.133610 (+ 33041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.139633 (+  6023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.139869 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.140728 10054 rpcz_store.cc:276] 0709 13:18:02.255093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.121603 (+1866510us) service_pool.cc:224] Handling call
0709 13:18:04.124855 (+  3252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.140101 (+ 15246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.140278 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.131394 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1745) took 1872 ms (client timeout 2343 ms). Trace:
W20260709 13:18:04.142161 10056 rpcz_store.cc:269] 0709 13:18:02.259355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.056002 (+796647us) service_pool.cc:224] Handling call
0709 13:18:04.131071 (+1075069us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b540010c2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:04.131310 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1205744896}
I20260709 13:18:04.153353 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1700) took 1815 ms. Trace:
I20260709 13:18:04.154165 10123 rpcz_store.cc:276] 0709 13:18:02.338040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.152123 (+1814083us) service_pool.cc:224] Handling call
0709 13:18:04.152598 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.152877 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.152917 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:04.152981 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:04.153031 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.153107 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.153263 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.157697 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:04.162891 10053 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:04.166457 10056 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:04.168051 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1762) took 1691 ms. Trace:
I20260709 13:18:04.104872 13789 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:04.128549 13867 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.251s	user 0.000s	sys 0.000s
I20260709 13:18:04.129307 13845 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:04.127032 13848 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
I20260709 13:18:04.191854 13792 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.120973 13719 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.177775 13796 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.180568 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:04.133872 13733 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:04.122512 13824 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.119313 13826 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:04.185811 13876 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:04.171492 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1743) took 1912 ms. Trace:
I20260709 13:18:04.156697 13744 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.213397 10053 rpcz_store.cc:276] 0709 13:18:02.258620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.141138 (+1882518us) service_pool.cc:224] Handling call
0709 13:18:04.162633 (+ 21495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.171022 (+  8389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.171402 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:04.168732 10056 rpcz_store.cc:276] 0709 13:18:02.476600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.161133 (+1684533us) service_pool.cc:224] Handling call
0709 13:18:04.161596 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.167620 (+  6024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.167965 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.205986 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 80430.9us
W20260709 13:18:04.215597 10096 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.956s (SENT)
I20260709 13:18:04.217454 13840 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:04.218938 13878 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:04.193241 13845 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:04.223914 13864 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.324s	user 0.000s	sys 0.003s
W20260709 13:18:04.225147 13869 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 13:18:04.203445 13790 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.227353 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 330) took 1947 ms. Trace:
I20260709 13:18:04.227751 10126 rpcz_store.cc:276] 0709 13:18:02.280280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.012235 (+731955us) service_pool.cc:224] Handling call
0709 13:18:03.056952 (+ 44717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:03.057414 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:03.570834 (+513420us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.842647 (+271813us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:03.850807 (+  8160us) log.cc:844] Serialized 26 byte log entry
0709 13:18:03.851295 (+   488us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:03.852297 (+  1002us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.853121 (+   824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.853209 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.227101 (+373892us) raft_consensus.cc:1685] finished
0709 13:18:04.227155 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.227290 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":347,"prepare.queue_time_us":655202,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":3547,"raft.queue_time_us":141065,"raft.run_cpu_time_us":514,"raft.run_wall_time_us":16965,"thread_start_us":661487,"threads_started":2,"wal-append.queue_time_us":356671}
I20260709 13:18:04.229195 13826 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.209895 13719 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:04.173786 10054 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:04.194768 13746 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:04.120499 13718 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.119092 13849 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.214546 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:04.238584 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.990s (SENT)
W20260709 13:18:04.248921 10096 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.062s (SENT)
W20260709 13:18:04.250082 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.107s (SENT)
W20260709 13:18:04.251149 10096 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.185s (SENT)
W20260709 13:18:04.252149 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.210s (SENT)
I20260709 13:18:04.241730 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1744) took 1982 ms. Trace:
I20260709 13:18:04.252832 10054 rpcz_store.cc:276] 0709 13:18:02.258971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.141431 (+1882460us) service_pool.cc:224] Handling call
0709 13:18:04.173401 (+ 31970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.241380 (+ 67979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.241656 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:04.241082 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 103926us
I20260709 13:18:04.259171 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.237828 13719 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:04.267583 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:04.268874 10200 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:04.272790 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.276252 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.277164 10891 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:04.260311 13752 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:04.248703 13746 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:04.279557 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:04.271317 13344 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.259078 13826 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:18:04.299885 10895 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEARNER]: Advancing to term 8
W20260709 13:18:04.301590 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.225s (SENT)
W20260709 13:18:04.310956 10096 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.343s (SENT)
I20260709 13:18:04.324888 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:04.329973 10895 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 8 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:04.299052 13886 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.299854 13888 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:04.296025 13725 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:18:04.339280 13881 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:04.339843 13725 consensus_queue.cc:1243] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:04.339725 13889 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.341274 13746 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.345639 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.346674 10054 raft_consensus.cc:2410] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:04.347783 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.349350 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.351514 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.352092 13828 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.352689 13828 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.353219 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.354094 10056 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:04.355054 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.355551 13835 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.356058 10056 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:04.356226 13835 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.357343 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.357735 13828 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.358685 10056 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:04.359949 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.360600 10056 raft_consensus.cc:2376] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:18:04.350848 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.361933 10054 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:18:04.354084 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1720) took 3230 ms (client timeout 1519 ms). Trace:
W20260709 13:18:04.365216 10203 rpcz_store.cc:269] 0709 13:18:01.123939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.124105 (+   166us) service_pool.cc:224] Handling call
0709 13:18:04.353852 (+3229747us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:04.353999 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997064064}
I20260709 13:18:04.312665 13752 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:04.370112 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.370416 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:04.372990 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.306619 13344 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.367913 13848 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:04.406920 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:04.386682 13882 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:18:04.384542 13664 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:04.411373 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48959.5us
I20260709 13:18:04.384894 13752 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.406309 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.417765 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.412039 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:04.413153 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.415702 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:04.420974 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:04.421301 10170 connection.cc:256] RPC call timeout handler was delayed by -1.6494816550000451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 13:18:04.422531 10170 connection.cc:256] RPC call timeout handler was delayed by -1.6138015019998875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 13:18:04.423014 10170 connection.cc:256] RPC call timeout handler was delayed by -1.5913015019998511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 13:18:04.387147 13846 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:04.425076 10170 connection.cc:256] RPC call timeout handler was delayed by -1.5644831009999507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 13:18:04.425398 13846 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:04.425707 10170 connection.cc:256] RPC call timeout handler was delayed by -1.5557520809998096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:18:04.426442 10170 connection.cc:256] RPC call timeout handler was delayed by -1.5360816550000891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:18:04.427032 10170 connection.cc:256] RPC call timeout handler was delayed by -1.4393831009999758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
I20260709 13:18:04.426522 13846 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.442766 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.444375 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.409595 13790 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.429575 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:04.332432 13719 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:04.451547 13884 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:18:04.452073 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.452734 13846 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:04.448576 10900 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:04.427855 10170 connection.cc:256] RPC call timeout handler was delayed by -1.3710437169997931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 13:18:04.452901 10891 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:04.449375 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:04.454488 10170 connection.cc:256] RPC call timeout handler was delayed by -1.2010921739999958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 13:18:04.455892 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.456561 10054 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:04.492529 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:04.485657 13835 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:04.499406 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.502573 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:04.505632 13885 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:18:04.505838 13838 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:04.506603 13838 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:04.507941 13838 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.509503 13838 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:04.515554 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.518798 10054 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:04.524564 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:04.530088 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:04.536885 10019 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.850s (SENT)
I20260709 13:18:04.545256 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:04.511003 13860 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:04.458055 13790 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.550208 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.517748 13718 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.519821 13669 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:04.567029 10019 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.024s (SENT)
I20260709 13:18:04.567682 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:04.569007 10019 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.126s (SENT)
I20260709 13:18:04.569517 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:04.570468 13796 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:04.571379 13796 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:04.572163 13895 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:18:04.572256 13796 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.528903 13849 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:04.573868 13893 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:18:04.542064 13234 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:04.585636 10056 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.608332 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:04.609171 10130 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:04.611871 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1770) took 2771 ms (client timeout 1786 ms). Trace:
W20260709 13:18:04.612412 10051 rpcz_store.cc:269] 0709 13:18:01.840647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.857528 (+ 16881us) service_pool.cc:224] Handling call
0709 13:18:04.611564 (+2754036us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54001b7454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:04.611729 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173063424}
I20260709 13:18:04.614261 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.615095 10051 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:04.584041 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:04.633956 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50019.3us
I20260709 13:18:04.579785 13725 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:04.553531 13886 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:18:04.552994 13889 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:18:04.591395 13718 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:04.625155 13899 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:18:04.656008 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1785) took 2109 ms. Trace:
I20260709 13:18:04.656266 10130 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:04.657444 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.656551 10053 rpcz_store.cc:276] 0709 13:18:02.546530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.214481 (+1667951us) service_pool.cc:224] Handling call
0709 13:18:04.245251 (+ 30770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.245703 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.245738 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.249166 (+  3428us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.331404 (+ 82238us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:04.331449 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.331521 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.655730 (+324209us) raft_consensus.cc:1685] finished
0709 13:18:04.655754 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.655932 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1377,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9098,"thread_start_us":41731,"threads_started":1,"wal-append.queue_time_us":135029}
I20260709 13:18:04.530962 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:04.638095 13234 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:04.581837 10054 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:04.645902 13849 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:04.585253 13669 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:04.660142 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 129244us
I20260709 13:18:04.682588 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:04.661469 10130 raft_consensus.cc:2368] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:18:04.643529 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.642474 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.664143 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.679791 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.687526 10053 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:04.688306 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.688948 10054 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:04.690212 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.690941 10051 raft_consensus.cc:2376] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6. Re-sending same reply.
I20260709 13:18:04.690397 13234 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.691987 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.692790 10051 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:04.693720 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.694386 10051 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:04.695602 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.696908 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.697940 10051 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:04.662479 13737 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:04.658813 13733 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:04.699349 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.655036 13725 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.700136 10051 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:04.686098 13839 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:04.701488 13839 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:04.701963 13839 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:04.699810 13733 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.704008 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:04.704958 10130 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:04.705368 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.707278 10051 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:04.709038 13746 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:04.705163 13737 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:04.709282 13790 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.701846 13725 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:04.708841 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66482us
I20260709 13:18:04.665638 13849 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:04.714711 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:04.713517 13894 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:18:04.721628 10900 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:04.713438 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.719823 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.723335 10130 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:04.729382 10051 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:04.711995 13896 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:04.729935 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.712934 13737 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:04.733878 10051 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:04.724738 13849 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:04.749418 10053 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:04.749946 10053 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:04.750732 10053 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.751929 10053 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:04.756597 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1701) took 2411 ms. Trace:
I20260709 13:18:04.729354 13344 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:04.757123 10123 rpcz_store.cc:276] 0709 13:18:02.345225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.157470 (+1812245us) service_pool.cc:224] Handling call
0709 13:18:04.192609 (+ 35139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:04.280527 (+ 87918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.280582 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.293124 (+ 12542us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.362925 (+ 69801us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:04.362969 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.363050 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.756321 (+393271us) raft_consensus.cc:1685] finished
0709 13:18:04.756345 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.756523 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13303,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":119018,"threads_started":2,"wal-append.queue_time_us":248543}
I20260709 13:18:04.758760 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.761755 10900 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:04.762331 10054 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:04.768621 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:04.770582 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 201583us
W20260709 13:18:04.771638 10019 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.171s (SENT)
I20260709 13:18:04.772176 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:04.781610 13809 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.783855 13737 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.784030 13839 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.791532 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:04.792547 10123 raft_consensus.cc:2376] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8. Re-sending same reply.
I20260709 13:18:04.799201 10053 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:18:04.799408 10900 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:04.802253 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.802709 10053 pending_rounds.cc:85] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:18:04.803078 10053 pending_rounds.cc:107] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:18:04.800696 10126 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:04.803584 10053 raft_consensus.cc:2889] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:04.804330 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.803011 10054 raft_consensus.cc:2376] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9. Re-sending same reply.
I20260709 13:18:04.804431 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.805445 10126 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:04.805796 10123 raft_consensus.cc:2410] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:04.807333 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1704) took 1209 ms. Trace:
I20260709 13:18:04.744995 13669 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.807904 10127 rpcz_store.cc:276] 0709 13:18:03.598077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.188441 (+590364us) service_pool.cc:224] Handling call
0709 13:18:04.188808 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:04.211397 (+ 22589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:04.211446 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:04.212267 (+   821us) log.cc:844] Serialized 52 byte log entry
0709 13:18:04.479472 (+267205us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:04.479532 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.479639 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.806974 (+327335us) raft_consensus.cc:1685] finished
0709 13:18:04.807007 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.807259 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16721,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":176,"thread_start_us":241593,"threads_started":2,"wal-append.queue_time_us":342467}
I20260709 13:18:04.806797 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:04.814040 13902 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:18:04.814689 13733 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.799723 13809 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:04.816649 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45075.2us
I20260709 13:18:04.821321 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:04.821161 10019 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.228s (SENT)
I20260709 13:18:04.819382 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:04.827605 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:04.828962 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.834092 10900 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:04.835502 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.837191 10123 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:04.833703 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.838542 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.839648 10126 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:04.841114 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.842386 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:04.842988 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.843042 10127 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:04.843864 10891 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:04.841624 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.847476 10056 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:04.849009 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.849589 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.850102 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.851374 10020 connection.cc:256] RPC call timeout handler was delayed by -1.4659173840002495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
I20260709 13:18:04.822098 13839 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:04.853211 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.829304 13888 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:04.873104 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.874140 10894 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:04.850102 10123 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:04.857458 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.855489 13905 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.880015 13905 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.883175 10127 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:04.886742 13905 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.848798 13725 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.887801 13725 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.890479 13725 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:04.859268 10019 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.307s (SENT)
I20260709 13:18:04.892479 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:04.894007 10019 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.379s (SENT)
I20260709 13:18:04.894582 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:04.860558 13790 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:04.863037 10131 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:04.875906 10054 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:04.895709 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1706) took 1296 ms (client timeout 1661 ms). Trace:
W20260709 13:18:04.896397 10125 rpcz_store.cc:269] 0709 13:18:03.598849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.936297 (+337448us) service_pool.cc:224] Handling call
0709 13:18:04.895470 (+959173us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:04.895632 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34814336}
I20260709 13:18:04.905572 10054 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:04.909889 10125 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:04.910954 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:04.912025 10054 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:18:04.907773 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.913528 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:04.915616 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44209.6us
I20260709 13:18:04.914821 10054 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:04.919693 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:04.919420 13796 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.921121 13809 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:04.922479 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:04.923352 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1689) took 3255 ms. Trace:
I20260709 13:18:04.922640 10131 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:04.923861 10128 rpcz_store.cc:276] 0709 13:18:01.667956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.349500 (+681544us) service_pool.cc:224] Handling call
0709 13:18:02.453902 (+104402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.454284 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.454311 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:04.604767 (+2150456us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b2c0011b9a8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:04.605517 (+   750us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.800075 (+194558us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:04.800129 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.800243 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.922947 (+122704us) raft_consensus.cc:1685] finished
0709 13:18:04.922972 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.923280 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5241,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":4263,"spinlock_wait_cycles":6152960,"thread_start_us":194018,"threads_started":1,"wal-append.queue_time_us":235107}
W20260709 13:18:04.914095 13909 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.059s	user 0.000s	sys 0.002s
I20260709 13:18:04.925882 10131 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:04.894045 13792 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:04.930202 13865 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:04.930761 13865 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:04.931941 13790 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:04.932427 13865 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.933542 13809 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:04.923465 10049 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:04.933970 13865 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:04.934461 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1778) took 1846 ms (client timeout 2343 ms). Trace:
W20260709 13:18:04.934859 10049 rpcz_store.cc:269] 0709 13:18:03.087711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.684340 (+596629us) service_pool.cc:224] Handling call
0709 13:18:04.922325 (+1237985us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:04.934390 (+ 12065us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4540800}
I20260709 13:18:04.933985 13809 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:04.941825 13809 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.944120 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2840) took 3089 ms. Trace:
I20260709 13:18:04.944661 10202 rpcz_store.cc:276] 0709 13:18:01.854703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.854955 (+   252us) service_pool.cc:224] Handling call
0709 13:18:01.893978 (+ 39023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.585927 (+1691949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.943740 (+1357813us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540039ec54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:04.944035 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3380868480}
I20260709 13:18:04.948737 13842 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:04.953212 13845 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:04.963477 13845 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:04.963828 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 380) took 1062 ms. Trace:
I20260709 13:18:04.966117 13849 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:04.967521 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73552.6us
I20260709 13:18:04.964480 10201 rpcz_store.cc:276] 0709 13:18:03.901878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.920162 (+ 18284us) service_pool.cc:224] Handling call
0709 13:18:03.920681 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:03.993684 (+ 73003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:03.993720 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:03.994228 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:18:04.129817 (+135589us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:04.129871 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:04.963253 (+833382us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:04.963323 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:04.963392 (+    69us) raft_consensus.cc:1685] finished
0709 13:18:04.963414 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:04.963743 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6854,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6554,"spinlock_wait_cycles":166822272,"thread_start_us":135108,"threads_started":1,"wal-append.queue_time_us":199591}
W20260709 13:18:04.981213 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.982071 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.983368 10096 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.670s (SENT)
I20260709 13:18:04.984038 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:04.984864 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:04.988468 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:04.989827 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:04.992229 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:04.994686 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.968919 13753 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:04.971916 13846 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.005s
I20260709 13:18:04.969141 13809 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:05.006258 10130 raft_consensus.cc:2410] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:05.006392 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:04.976723 13849 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:04.993633 13865 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:04.998934 13796 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:04.999598 13721 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:05.021493 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.014420 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:05.004679 13838 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:04.993533 13723 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:04.994059 13719 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:05.029316 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:05.031175 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.000533 13234 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:18:05.008782 13686 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:05.003218 13725 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:18:05.019847 13815 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.766s	user 0.010s	sys 0.026s
I20260709 13:18:05.038489 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:05.027513 13838 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.024837 13860 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.020069 13796 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:05.036962 13875 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:05.051251 13815 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.800s	user 0.011s	sys 0.027s
I20260709 13:18:05.021505 13753 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:18:05.067586 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:05.041980 13839 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.068789 13841 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:05.093345 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 95006us
I20260709 13:18:05.072822 13838 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.073495 13860 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.078955 13796 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.034104 13721 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:05.043233 13234 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:05.068329 13819 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:05.112344 13825 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 2.853s	user 0.005s	sys 0.014s
I20260709 13:18:05.112496 13721 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:05.094990 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.095530 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 126969us
I20260709 13:18:05.096599 13875 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:05.116631 10019 connection.cc:256] RPC call timeout handler was delayed by -4.938784256999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2685s for a response
I20260709 13:18:05.117118 13875 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:05.117374 10019 connection.cc:256] RPC call timeout handler was delayed by -4.7366624870001033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 13:18:05.119069 10019 connection.cc:256] RPC call timeout handler was delayed by -4.6943624870000349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:18:05.121748 10019 connection.cc:256] RPC call timeout handler was delayed by -4.3640624869999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
I20260709 13:18:05.097642 13786 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:05.032851 13573 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:05.096621 13692 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:05.028183 13669 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:05.069900 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:05.105736 13860 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.132954 13786 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:05.135574 13786 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:05.097190 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.111161 13819 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:05.141873 13725 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:05.143167 13905 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:05.143003 13725 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.143775 13905 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:05.143920 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.144672 10047 raft_consensus.cc:2376] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8. Re-sending same reply.
W20260709 13:18:05.145264 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1775) took 2096 ms (client timeout 2185 ms). Trace:
I20260709 13:18:05.144958 13905 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.145833 13725 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.147327 10129 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:05.149787 13819 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:05.145660 10047 rpcz_store.cc:269] 0709 13:18:03.048676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.732592 (+683916us) service_pool.cc:224] Handling call
0709 13:18:05.143744 (+1411152us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:05.145195 (+  1451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4613376}
W20260709 13:18:05.148783 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 81301.8us
W20260709 13:18:05.128914 10019 connection.cc:256] RPC call timeout handler was delayed by -4.2838210869999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 13:18:05.164327 10019 connection.cc:256] RPC call timeout handler was delayed by -4.2496210869999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
W20260709 13:18:05.164865 10019 connection.cc:256] RPC call timeout handler was delayed by -3.9922506550001344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:18:05.165259 10019 connection.cc:256] RPC call timeout handler was delayed by -3.9841210869999486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.231s for a response
W20260709 13:18:05.165633 10019 connection.cc:256] RPC call timeout handler was delayed by -3.8545506550001392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 13:18:05.166019 10019 connection.cc:256] RPC call timeout handler was delayed by -3.8176506550000795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 13:18:05.166383 10019 connection.cc:256] RPC call timeout handler was delayed by -3.3362624870001127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.321s for a response
W20260709 13:18:05.166762 10019 connection.cc:256] RPC call timeout handler was delayed by -3.3227624870000909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3225s for a response
W20260709 13:18:05.167173 10019 connection.cc:256] RPC call timeout handler was delayed by -3.3056624870000633s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3244s for a response
W20260709 13:18:05.167553 10019 connection.cc:256] RPC call timeout handler was delayed by -3.2803506550001202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23290000000000002s for a response
I20260709 13:18:05.146328 13905 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:05.178536 10891 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:05.128670 13841 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:05.141896 13692 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:05.189347 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 158274us
W20260709 13:18:05.191422 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.171892 13810 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:05.194885 13810 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:05.200547 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.271458 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.277302 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82913.8us
I20260709 13:18:05.278849 13573 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.281698 13573 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:05.284584 13573 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:05.284454 13838 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.265169 13815 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:18:05.285919 13815 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:05.288339 10129 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:05.265523 13810 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.287858 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:05.299721 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1705) took 1701 ms (client timeout 1768 ms). Trace:
I20260709 13:18:05.300055 10047 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:05.300170 10129 rpcz_store.cc:269] 0709 13:18:03.598571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.947340 (+348769us) service_pool.cc:224] Handling call
0709 13:18:05.069600 (+1122260us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:05.299649 (+230049us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11494144}
I20260709 13:18:05.186815 13669 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:18:05.190578 13924 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
W20260709 13:18:05.318704 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.335808 13753 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:05.336385 13841 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.338644 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2839) took 3484 ms. Trace:
I20260709 13:18:05.339162 10197 rpcz_store.cc:276] 0709 13:18:01.854420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.854606 (+   186us) service_pool.cc:224] Handling call
0709 13:18:01.854991 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.493276 (+638285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.338334 (+2845058us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:05.338556 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465646464}
W20260709 13:18:05.351306 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.334403 10047 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:18:05.361863 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:05.363829 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1758) took 3023 ms (client timeout 2916 ms). Trace:
W20260709 13:18:05.364392 10055 rpcz_store.cc:269] 0709 13:18:02.340838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.360722 (+3019884us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:05.363743 (+  3021us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:05.286790 13838 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:05.372709 13811 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:05.395305 10051 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:05.366482 13721 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:05.400475 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.370674 13819 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.392146 13810 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
W20260709 13:18:05.407791 13877 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.098s	user 0.021s	sys 0.036s
W20260709 13:18:05.408973 13877 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.021s	sys 0.037s
I20260709 13:18:05.410138 13819 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:05.414099 13796 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:05.417671 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.404668 13721 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:05.420671 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:05.391330 13860 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.425032 13844 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:05.425639 13844 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:05.424791 13905 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:05.428726 10198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2836) took 3577 ms (client timeout 2130 ms). Trace:
W20260709 13:18:05.429420 10198 rpcz_store.cc:269] 0709 13:18:01.852356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.855676 (+  3320us) service_pool.cc:224] Handling call
0709 13:18:05.428423 (+3572747us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54001883d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:05.428622 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025613696}
I20260709 13:18:05.431550 13231 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:18:05.432020 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 373) took 3603 ms (client timeout 1782 ms). Trace:
I20260709 13:18:05.432327 10196 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:05.438395 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1793) took 1076 ms. Trace:
I20260709 13:18:05.438912 10056 rpcz_store.cc:276] 0709 13:18:04.362290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.405947 (+ 43657us) service_pool.cc:224] Handling call
0709 13:18:05.438172 (+1032225us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400235fd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:05.438314 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385671040}
I20260709 13:18:05.420399 13721 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:05.423372 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.452180 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 171386us
W20260709 13:18:05.453572 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.453548 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 334) took 1076 ms. Trace:
I20260709 13:18:05.454452 10127 rpcz_store.cc:276] 0709 13:18:04.376800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.844150 (+467350us) service_pool.cc:224] Handling call
0709 13:18:04.852218 (+  8068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:04.883939 (+ 31721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.446408 (+562469us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b540014c654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:05.453476 (+  7068us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70477568}
W20260709 13:18:05.432463 10199 rpcz_store.cc:269] 0709 13:18:01.829008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.846088 (+ 17080us) service_pool.cc:224] Handling call
0709 13:18:05.431737 (+3585649us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54001ff254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:05.431929 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4226388864}
I20260709 13:18:05.451924 13725 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.459654 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:05.460024 13725 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.462203 13796 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:05.452621 13845 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.463577 13845 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.464108 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:05.464427 13725 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.465873 13845 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.461666 13630 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.467419 13630 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.461740 13574 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:05.464811 10900 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.475302 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:05.476135 10127 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:05.472517 13574 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:05.478516 13574 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.480893 13737 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:05.482231 13860 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:05.483364 13630 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.485451 10900 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:05.487159 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:05.487802 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:05.495016 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:05.496482 10129 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:05.484469 13880 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.332s	user 0.001s	sys 0.003s
I20260709 13:18:05.499940 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.500200 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:05.500972 10123 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.502426 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:05.505220 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:05.505327 10051 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:05.509282 10051 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:05.513931 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.520025 10199 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:05.514129 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.522082 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.523877 10198 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:05.524443 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124066us
I20260709 13:18:05.525125 10199 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:05.426573 13844 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.527216 10049 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:18:05.454273 13848 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:05.527796 10123 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:05.513068 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:05.537398 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.538910 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.539841 10049 pending_rounds.cc:85] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 3
I20260709 13:18:05.540421 10049 pending_rounds.cc:107] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:05.540796 10049 raft_consensus.cc:2889] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.540192 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:05.543085 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.551004 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:05.552135 10199 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:05.551679 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.554306 10203 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:05.562418 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:05.554626 10202 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:05.586777 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.555016 10200 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:18:05.508004 13796 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:05.591440 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.555195 10053 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:05.582042 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:05.545905 10047 pending_rounds.cc:85] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 2
I20260709 13:18:05.588271 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.593310 10047 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:05.484987 13845 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.555739 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:05.590174 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.597397 10050 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:05.593853 10047 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.593667 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.594427 10204 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:05.597834 10047 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:05.598220 10047 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:05.605473 10050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1776) took 2548 ms (client timeout 2210 ms). Trace:
I20260709 13:18:05.554353 13792 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.608348 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:05.614571 10203 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:05.615367 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.616374 13932 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:05.616361 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:05.585466 13790 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.610788 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:05.588568 13844 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.620968 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:05.608722 13785 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.009s	sys 0.035s
I20260709 13:18:05.527081 13905 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:18:05.531584 13630 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 13:18:05.630033 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48147.8us
W20260709 13:18:05.631338 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.535725 13810 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
W20260709 13:18:05.609149 10050 rpcz_store.cc:269] 0709 13:18:03.056968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.740136 (+683168us) service_pool.cc:224] Handling call
0709 13:18:05.555563 (+1815427us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:05.605350 (+ 49787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4470144}
I20260709 13:18:05.585007 13848 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:05.643599 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 81260us
I20260709 13:18:05.620419 13932 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:05.649070 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:05.617976 13792 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:05.634124 13785 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.459s	user 0.010s	sys 0.035s
I20260709 13:18:05.656803 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:05.660271 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.672514 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.682145 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:05.683071 10050 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.607753 13845 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:05.698014 13938 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:18:05.697773 13905 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:05.698640 13905 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:05.633900 13836 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:18:05.699791 13905 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.673312 13792 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:05.647383 13930 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:05.634886 13840 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:18:05.684037 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:05.655051 13932 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:05.710227 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58558.9us
I20260709 13:18:05.711486 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.687978 13725 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.726580 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:05.675424 13924 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:18:05.741587 10050 raft_consensus.cc:1217] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-10.6]   Dedup: 4.3->[6.4-10.6]
I20260709 13:18:05.731932 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:05.814112 13723 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:18:05.802153 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:05.817915 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:05.822386 10050 pending_rounds.cc:85] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 3
I20260709 13:18:05.824093 10050 pending_rounds.cc:107] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:05.826100 10050 raft_consensus.cc:2889] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:05.702455 13860 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
W20260709 13:18:05.830942 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99085.3us
I20260709 13:18:05.731402 13367 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:05.790104 13905 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:05.839107 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.841734 10126 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:05.845108 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:05.821125 13810 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:05.827435 13921 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:05.718353 13855 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:05.873842 13842 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.841526 13367 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:05.841938 13723 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:05.892293 10048 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:18:05.895779 13933 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:18:05.899191 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 108855us
I20260709 13:18:05.905704 10894 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:05.909870 13944 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:05.910513 13810 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:05.916786 13848 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.925727 13845 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.930454 13930 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:05.865638 13921 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.973352 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:05.879180 13951 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:05.947856 13690 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.961862 13838 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.986397 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:05.988162 13367 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.989459 13367 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:05.993950 13690 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:05.972484 13800 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:18:05.989957 13930 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:05.996380 13838 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:05.996954 13930 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:06.000021 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.002244 13951 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.002959 13951 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:06.006114 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 339) took 1627 ms (client timeout 2107 ms). Trace:
I20260709 13:18:06.007185 13810 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.007930 13810 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.009641 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1743) took 1998 ms. Trace:
I20260709 13:18:06.010612 13810 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:05.979943 13921 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:06.017158 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:06.017524 10131 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:06.017940 10126 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 5.
I20260709 13:18:06.018708 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.019500 10204 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:06.020164 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.021340 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 442) took 1105 ms. Trace:
I20260709 13:18:06.021986 10894 rpcz_store.cc:276] 0709 13:18:04.915717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.919419 (+  3702us) service_pool.cc:224] Handling call
0709 13:18:06.021100 (+1101681us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b540064e1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:06.021257 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264949504}
I20260709 13:18:06.010242 10196 rpcz_store.cc:276] 0709 13:18:04.011471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.011660 (+   189us) service_pool.cc:224] Handling call
0709 13:18:05.431994 (+1420334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.433206 (+  1212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.009328 (+576122us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001ff250. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:06.009546 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3265300352}
I20260709 13:18:06.025198 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:06.026154 10127 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:06.006827 10130 rpcz_store.cc:269] 0709 13:18:04.378236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.729601 (+351365us) service_pool.cc:224] Handling call
0709 13:18:06.005784 (+1276183us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54003d4854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:06.006015 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632379648}
I20260709 13:18:06.063638 10122 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:06.070669 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:06.071522 10126 raft_consensus.cc:2376] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9. Re-sending same reply.
I20260709 13:18:06.073277 10130 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:06.103945 10127 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:18:06.104137 10168 connection.cc:256] RPC call timeout handler was delayed by -1.3725744170001235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 13:18:06.107481 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:06.108560 10122 raft_consensus.cc:2376] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9. Re-sending same reply.
I20260709 13:18:06.107563 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:06.113046 10131 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:06.114352 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:06.128209 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:06.166875 13951 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:06.196049 13925 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.886s	user 0.005s	sys 0.000s
I20260709 13:18:06.199970 13946 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:06.233218 13932 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.237496 13932 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.064740 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:06.300657 10054 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:06.450727 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 385893us
I20260709 13:18:06.452870 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.321601 10128 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:06.324069 10123 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:18:06.456221 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:06.348568 13725 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.186116 13785 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:06.458638 13785 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:06.458349 13725 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.355403 13826 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.461990 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.462255 13826 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.462515 13785 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.463407 10054 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:06.464280 13725 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.376667 10131 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:18:06.409570 13786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.020s	sys 0.019s
I20260709 13:18:06.409226 13646 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.464959 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:06.417510 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.016s	sys 0.001s
W20260709 13:18:06.466041 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.016s	sys 0.001s
I20260709 13:18:06.465868 13646 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.417518 13951 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.466586 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:06.467651 13951 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.469050 10124 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:06.470180 10127 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:06.470569 13646 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:06.223615 13844 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.003s
I20260709 13:18:06.471271 13959 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:06.472600 13959 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.473449 10130 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:06.474035 10130 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:06.475404 13682 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:06.476253 13951 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:06.477142 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51170.7us
W20260709 13:18:06.433480 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 175009us
I20260709 13:18:06.479817 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:06.482939 13961 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.484431 13949 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:06.484997 13949 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:06.457406 10128 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:18:06.358645 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.483018 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.489470 10128 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:18:06.490161 10128 pending_rounds.cc:85] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 3
I20260709 13:18:06.490283 10054 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
W20260709 13:18:06.465426 13786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.021s	sys 0.019s
W20260709 13:18:06.401564 13796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.021s	sys 0.005s
W20260709 13:18:06.409590 13792 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.695s	user 0.015s	sys 0.002s
I20260709 13:18:06.491179 13924 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:06.476061 10130 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.492197 13792 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.493069 10130 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.493520 13752 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:06.494061 13752 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:06.376361 10198 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:06.491722 13796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.022s	sys 0.006s
I20260709 13:18:06.495630 13796 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:06.456614 10123 pending_rounds.cc:85] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:18:06.496528 10123 pending_rounds.cc:107] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:18:06.497004 10123 raft_consensus.cc:2889] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.484038 13961 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.482120 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:06.490490 10128 pending_rounds.cc:107] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:06.450436 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:06.495721 13752 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.504846 13796 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.508214 10128 raft_consensus.cc:2889] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.527503 13723 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:06.531900 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.532697 13932 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:06.533263 13932 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.533854 13932 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:06.534425 13951 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:06.535415 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.535668 13951 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.536715 10051 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:06.538362 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.531900 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.539434 13951 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.533175 10198 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:18:06.541738 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.534547 13692 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.557389 10126 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:06.550307 13826 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.568183 13752 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:06.569005 13922 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.287s	user 0.003s	sys 0.003s
W20260709 13:18:06.571018 13935 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.781s	user 0.005s	sys 0.000s
I20260709 13:18:06.486815 13949 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.544157 13792 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:06.584458 10054 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:06.586695 13231 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.004s
W20260709 13:18:06.589923 13940 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.858s	user 0.000s	sys 0.004s
I20260709 13:18:06.581959 10051 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:06.617717 13961 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:06.618000 13949 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:06.624922 13959 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.626295 10196 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [7.5-9.6]
W20260709 13:18:06.626807 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.613907 10127 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:18:06.627147 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 145199us
I20260709 13:18:06.629072 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:06.630506 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:06.612932 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88597.9us
I20260709 13:18:06.642917 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.644721 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:06.644863 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.648448 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.660027 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:06.660866 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.661612 10051 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:18:06.662305 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.663245 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46246.1us
I20260709 13:18:06.664726 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:06.666003 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:06.670054 13646 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:06.679237 13826 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.679481 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.680172 13826 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:06.685009 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.685951 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.689971 13959 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:06.692076 13939 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.868s	user 0.002s	sys 0.002s
I20260709 13:18:06.680173 10049 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:06.658717 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.702050 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1813) took 1158 ms. Trace:
I20260709 13:18:06.662840 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:06.707892 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.670852 13646 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:06.717432 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.719852 10054 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:06.718892 13725 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.726037 13725 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.727969 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:06.730194 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76614.2us
I20260709 13:18:06.715811 13646 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:06.731608 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1815) took 1184 ms. Trace:
I20260709 13:18:06.732014 13924 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:06.732986 13924 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:06.739953 10196 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:06.741935 10891 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:06.743927 13943 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:06.744611 13943 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:06.748210 13838 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.690783 13959 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:06.749348 13953 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.553s	user 0.000s	sys 0.002s
W20260709 13:18:06.751235 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1781) took 4279 ms (client timeout 2208 ms). Trace:
I20260709 13:18:06.751554 13959 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:06.751760 10048 rpcz_store.cc:269] 0709 13:18:02.471866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.057489 (+585623us) service_pool.cc:224] Handling call
0709 13:18:06.750706 (+3693217us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54002350d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:06.750937 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6452827392}
I20260709 13:18:06.753620 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:06.763172 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111380us
W20260709 13:18:06.730744 13963 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.215s	user 0.001s	sys 0.002s
I20260709 13:18:06.732077 10052 rpcz_store.cc:276] 0709 13:18:05.546838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.547042 (+   204us) service_pool.cc:224] Handling call
0709 13:18:05.547484 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.547938 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.547972 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.548504 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:18:05.634808 (+ 86304us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:05.634892 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:05.634981 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.691044 (+1056063us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:06.703178 (+ 12134us) raft_consensus.cc:1685] finished
0709 13:18:06.703203 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.731399 (+ 28196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33638,"raft.queue_time_us":3282,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":34301,"spinlock_wait_cycles":14583552,"thread_start_us":85879,"threads_started":1,"wal-append.queue_time_us":160121}
I20260709 13:18:06.702613 10049 rpcz_store.cc:276] 0709 13:18:05.544161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.549583 (+  5422us) service_pool.cc:224] Handling call
0709 13:18:05.550065 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.698677 (+1148612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.701598 (+  2921us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:06.701940 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546646272}
I20260709 13:18:06.760692 10056 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:06.738636 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:06.709161 13826 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.734407 13924 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.779525 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:06.799077 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1812) took 1259 ms. Trace:
I20260709 13:18:06.799592 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.799893 10891 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:06.799642 10055 rpcz_store.cc:276] 0709 13:18:05.539605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.539791 (+   186us) service_pool.cc:224] Handling call
0709 13:18:05.590352 (+ 50561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.779275 (+188923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.779326 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.780097 (+   771us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.466822 (+686725us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:06.466886 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.466980 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:06.729480 (+262500us) spinlock_profiling.cc:243] Waited 879 us on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:06.798613 (+ 69133us) raft_consensus.cc:1685] finished
0709 13:18:06.798637 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.798988 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32533,"raft.queue_time_us":60860,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":18736,"spinlock_wait_cycles":2022656,"thread_start_us":846578,"threads_started":2,"wal-append.queue_time_us":735185}
I20260709 13:18:06.802177 10051 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:06.791635 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:06.803996 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65643.2us
I20260709 13:18:06.804800 10055 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:06.805881 10051 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:18:06.806453 10051 pending_rounds.cc:85] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:18:06.804065 10196 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:06.806718 10055 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:06.807350 10048 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:18:06.808872 13367 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.819s	user 0.017s	sys 0.007s
I20260709 13:18:06.809572 10055 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:06.806799 10051 pending_rounds.cc:107] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:06.811528 10051 raft_consensus.cc:2889] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:06.813530 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:06.822047 13810 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:06.804997 13725 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:06.823524 10055 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:06.824640 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:06.825551 10900 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:06.825865 10131 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:06.822763 13810 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:06.828864 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:06.829835 10126 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:06.831132 10130 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
W20260709 13:18:06.789934 10019 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.642s (SENT)
I20260709 13:18:06.792773 13826 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:06.844555 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50367.7us
I20260709 13:18:06.840013 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.836992 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.831771 10130 pending_rounds.cc:85] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
W20260709 13:18:06.848086 13921 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.824s	user 0.021s	sys 0.003s
I20260709 13:18:06.848791 13921 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.852005 13810 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.853466 13810 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:06.857481 13983 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.858325 13932 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:06.884115 13980 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:06.859661 10894 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 11. Ops: []
I20260709 13:18:06.860750 10205 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:06.867354 13826 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:06.886157 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104256us
I20260709 13:18:06.867933 13776 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:06.887312 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:06.868938 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:06.876595 13845 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.888502 13967 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:06.888466 13845 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.889125 10052 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:18:06.830480 13959 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:06.876962 13725 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.891929 13725 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:06.892499 10130 pending_rounds.cc:107] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:18:06.892885 10130 raft_consensus.cc:2889] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:06.880370 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 84241.6us
I20260709 13:18:06.881193 13573 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:06.858523 13983 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.887480 13776 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.875700 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.936861 13776 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:06.965993 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.967622 10054 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:18:06.941238 13845 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:06.971676 13949 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:06.972959 10202 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:06.972865 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:06.973461 13984 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:06.973562 13823 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:06.955776 13932 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:06.859743 13943 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:06.979463 10019 leader_election.cc:400] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
I20260709 13:18:07.015815 10199 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:06.966773 13921 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.020746 10054 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:06.987756 13725 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.010826 13823 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:07.013623 13826 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.016525 13845 raft_consensus.cc:476] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Not starting pre-election -- already a leader
I20260709 13:18:07.035461 10054 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:07.041531 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.038733 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:07.029321 13943 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.043808 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:07.047274 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.033468 13725 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:07.041869 13984 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.036893 13845 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.049113 13932 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:07.092612 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:07.092897 13984 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.051558 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:07.095625 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.096024 13367 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:07.099210 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:07.099591 13367 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.103036 13367 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:07.107724 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 154237us
I20260709 13:18:07.100176 10054 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:07.101001 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1839) took 1051 ms. Trace:
I20260709 13:18:07.110740 13921 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.109913 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.109882 10055 rpcz_store.cc:276] 0709 13:18:06.049544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.824448 (+774904us) service_pool.cc:224] Handling call
0709 13:18:06.824905 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.825820 (+   915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.825850 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.826459 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.983495 (+157036us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:06.983541 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.983627 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.100726 (+117099us) raft_consensus.cc:1685] finished
0709 13:18:07.100757 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.100931 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":175,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"thread_start_us":156574,"threads_started":1,"wal-append.queue_time_us":271501}
I20260709 13:18:07.117770 13984 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.117441 13921 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.123262 10055 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:07.123442 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1837) took 1107 ms. Trace:
W20260709 13:18:07.129355 13979 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:18:07.130915 13983 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.037388 13826 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:18:07.136248 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.132148 10051 rpcz_store.cc:276] 0709 13:18:06.016119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.812962 (+796843us) service_pool.cc:224] Handling call
0709 13:18:06.813430 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.813891 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.813920 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.814569 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.943513 (+128944us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:06.943586 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.943703 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.123187 (+179484us) raft_consensus.cc:1685] finished
0709 13:18:07.123211 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.123367 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":128365,"threads_started":1,"wal-append.queue_time_us":210954}
I20260709 13:18:07.094465 13932 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.134792 10205 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:07.113979 13367 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:07.141930 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 127250us
I20260709 13:18:07.150367 10055 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:07.144433 13961 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:07.152374 13961 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.140318 13752 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:07.120002 13845 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.093988 13725 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.141494 13932 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:07.170672 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.212894 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.213564 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.206310  9953 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 117) took 18868 ms (client timeout 15000 ms). Trace:
I20260709 13:18:07.215754 10054 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:07.025422  9942 maintenance_manager.cc:643] P db0c98fae3aa4b45a39c8914c5e641cc: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 12.101s	user 0.211s	sys 0.119s Metrics: {"bytes_written":29933,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":150962,"dirs.run_cpu_time_us":637,"dirs.run_wall_time_us":38267,"drs_written":1,"lbm_read_time_us":501,"lbm_reads_lt_1ms":4,"lbm_write_time_us":718608,"lbm_writes_10-100_ms":15,"lbm_writes_gt_100_ms":2,"lbm_writes_lt_1ms":16,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":97336,"threads_started":2,"update_count":11,"wal-append.queue_time_us":58432}
I20260709 13:18:07.146489 13826 consensus_queue.cc:1243] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:18:07.219094 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109297us
I20260709 13:18:07.218215 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.202591 13752 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:07.230342 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:07.235502 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.247295 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.220623 13646 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.254608 10891 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:01.548084  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:07.255404 13990 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.004s
I20260709 13:18:07.200860 13984 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.256857 13646 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.256593 13984 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:07.226951 13987 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
W20260709 13:18:07.259411 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.262663 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1838) took 1244 ms. Trace:
I20260709 13:18:07.262475 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.288270 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.290148 10055 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:07.290940 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.272780 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:07.293877 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.294476 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.295369 10895 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:07.260025 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.262884 13646 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.261019 13961 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.262198 13905 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.309888 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.284937 10048 rpcz_store.cc:276] 0709 13:18:06.018551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.824291 (+805740us) service_pool.cc:224] Handling call
0709 13:18:06.824775 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.825605 (+   830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.825638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.826529 (+   891us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.965565 (+139036us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:06.965635 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.967654 (+  2019us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.262347 (+294693us) raft_consensus.cc:1685] finished
0709 13:18:07.262368 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.262582 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"thread_start_us":138501,"threads_started":1,"wal-append.queue_time_us":203077}
I20260709 13:18:07.308861 10055 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:07.259056 13984 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.304095 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:07.315945 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.316918 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 224481us
I20260709 13:18:07.317983 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.321707 13826 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:07.322939 13991 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.324000 13826 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:07.320569 13930 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.324872 13991 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.325165 13930 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.325845 13826 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:07.215314  9953 rpcz_store.cc:269] 0709 13:17:48.337970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.355161 (+ 17191us) service_pool.cc:224] Handling call
0709 13:18:07.205953 (+18850792us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:58.607715 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:58.723981 (+116266us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:58.724075 (+    94us) write_op.cc:435] Acquired schema lock
0709 13:17:58.724098 (+    23us) tablet.cc:662] Decoding operations
0709 13:17:58.725935 (+  1837us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:58.726064 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:58.726120 (+    56us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:17:58.765312 (+ 39192us) tablet.cc:701] Row locks acquired
0709 13:17:58.765385 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:17:58.765693 (+   308us) write_op.cc:270] Start()
0709 13:17:58.765890 (+   197us) write_op.cc:276] Timestamp: P: 1783603078765594 usec, L: 0
0709 13:17:58.765922 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:17:58.769896 (+  3974us) log.cc:844] Serialized 4362 byte log entry
0709 13:18:00.129906 (+1360010us) op_driver.cc:464] REPLICATION: finished
0709 13:18:00.759474 (+629568us) write_op.cc:301] APPLY: starting
0709 13:18:00.785309 (+ 25835us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:00.886902 (+101593us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:00.887139 (+   237us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:01.145014 (+257875us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:01.198182 (+ 53168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:18:01.198238 (+    56us) write_op.cc:312] APPLY: finished
0709 13:18:01.198703 (+   465us) log.cc:844] Serialized 83 byte log entry
0709 13:18:01.399717 (+201014us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:01.399956 (+   239us) write_op.cc:454] Released schema lock
0709 13:18:01.400040 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:01.400571 (+   531us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:01.400602 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":186475,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":585626,"apply.run_cpu_time_us":58228,"apply.run_wall_time_us":660280,"num_ops":11,"prepare.queue_time_us":1307969,"prepare.run_cpu_time_us":16485,"prepare.run_wall_time_us":453771,"raft.queue_time_us":912517,"raft.run_cpu_time_us":27117,"raft.run_wall_time_us":431410,"replication_time_us":1363753,"thread_start_us":1999151,"threads_started":5,"wal-append.queue_time_us":720437,"wal-append.run_cpu_time_us":27169,"wal-append.run_wall_time_us":2790003}]]}
I20260709 13:18:07.322188  9943 maintenance_manager.cc:419] P db0c98fae3aa4b45a39c8914c5e641cc: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7870 bytes on disk
I20260709 13:18:07.314169 13646 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.320891 13944 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.322062 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.328272 13810 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:07.361706 13992 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.264s	user 0.003s	sys 0.000s
W20260709 13:18:07.363220 13977 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.511s	user 0.004s	sys 0.000s
W20260709 13:18:07.364475 13986 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 13:18:07.366731 13960 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.168s	user 0.005s	sys 0.000s
I20260709 13:18:07.368479 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:07.370373 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:07.332880 13991 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.371557 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:07.328689 13776 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.338405 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1804) took 1835 ms. Trace:
W20260709 13:18:07.362165 13982 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.389s	user 0.003s	sys 0.000s
I20260709 13:18:07.341747  9942 maintenance_manager.cc:643] P db0c98fae3aa4b45a39c8914c5e641cc: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":516,"lbm_reads_lt_1ms":4}
I20260709 13:18:07.373632 10056 rpcz_store.cc:276] 0709 13:18:05.502541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.502695 (+   154us) service_pool.cc:224] Handling call
0709 13:18:06.759906 (+1257211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.769352 (+  9446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.338013 (+568661us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002350d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:07.338298 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2879574784}
I20260709 13:18:07.368468 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 394) took 1259 ms. Trace:
I20260709 13:18:07.302948 13845 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.390023 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.261838 13752 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.390703 10198 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
W20260709 13:18:07.381384 13987 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:07.389307 13776 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:07.394068 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 173622us
W20260709 13:18:07.395408 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 172108us
I20260709 13:18:07.357112 13231 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:07.380836 13785 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:07.353624 13725 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.380872 10204 rpcz_store.cc:276] 0709 13:18:06.109017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.110046 (+  1029us) service_pool.cc:224] Handling call
0709 13:18:06.110439 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.113607 (+  3168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.113664 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.505397 (+391733us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:06.506130 (+   733us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.718903 (+212773us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:06.760840 (+ 41937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.760972 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.357919 (+596947us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:07.367924 (+ 10005us) raft_consensus.cc:1685] finished
0709 13:18:07.367964 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.368173 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146650,"raft.queue_time_us":22299,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":5835,"spinlock_wait_cycles":88000256,"thread_start_us":207732,"threads_started":1,"wal-append.queue_time_us":341008}
I20260709 13:18:07.329326 13690 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.393497 13961 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:07.396095 13686 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:07.406049 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:07.382895 13855 consensus_queue.cc:1175] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 10 status { last_received { term: 9 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:07.409762 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:07.418730 10051 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.421907 13993 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.412789 13967 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:07.400895 13231 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:07.299774 13823 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:07.449994 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 187660us
I20260709 13:18:07.407423 13921 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.456276 13691 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:07.402045 13646 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.458380 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 365) took 1545 ms. Trace:
I20260709 13:18:07.401604 13905 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.458859 10052 rpcz_store.cc:276] 0709 13:18:05.912650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.799004 (+886354us) service_pool.cc:224] Handling call
0709 13:18:06.799479 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.889597 (+ 90118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.457305 (+567708us) spinlock_profiling.cc:243] Waited 89.3 ms on lock 0x7b540010bdd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:07.458291 (+   986us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205410176}
W20260709 13:18:07.462311 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90874.7us
I20260709 13:18:07.421939 13725 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.451978 13930 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.472960 10056 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:07.457872 13993 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:07.482206 10056 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:07.487251 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.458596 13690 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.474414 13691 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:07.499884 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 160410us
W20260709 13:18:07.500641 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94764.9us
I20260709 13:18:07.423091 13967 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:07.480370 13993 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.460388 13196 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:18:07.513038 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.514765 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.479332 13725 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.517582 10200 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:07.541136 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.536659 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:07.541374 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46305.6us
I20260709 13:18:07.541157 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:07.541502 10201 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:07.545300 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.548352 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.546957 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1820) took 1930 ms. Trace:
I20260709 13:18:07.550041 13196 consensus_queue.cc:1243] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:07.513795 13993 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.552394 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:07.537750 13231 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.555531 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:07.564318 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:07.569818 10203 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:07.573933 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.573993 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:07.575067 10204 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.577997 13845 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:07.578770 13845 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.579383 13845 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:07.582542 13967 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.583596 13967 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.584508 13967 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:07.550387 10050 rpcz_store.cc:276] 0709 13:18:05.548821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.740651 (+191830us) service_pool.cc:224] Handling call
0709 13:18:05.741260 (+   609us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:18:05.880731 (+139471us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:05.880862 (+   131us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:05.882315 (+  1453us) log.cc:844] Serialized 78 byte log entry
0709 13:18:06.304751 (+422436us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:06.304929 (+   178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.305145 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.347008 (+1041863us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:07.467513 (+120505us) raft_consensus.cc:1685] finished
0709 13:18:07.467544 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.479013 (+ 11469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43100,"raft.queue_time_us":42172,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":29990,"spinlock_wait_cycles":20743936,"thread_start_us":319109,"threads_started":1,"wal-append.queue_time_us":224806}
I20260709 13:18:07.573395 13838 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:07.587872 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.544183 13344 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.152s
I20260709 13:18:07.592002 13983 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:07.595525 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1830) took 1716 ms. Trace:
I20260709 13:18:07.570982 13930 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:07.540606 13690 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.599407 13942 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.602612 13725 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.604110 13905 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.604164 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.631366 10204 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:07.587906 13981 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.616s	user 0.000s	sys 0.003s
I20260709 13:18:07.547681 13826 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:18:07.622668 10049 rpcz_store.cc:276] 0709 13:18:05.878797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.783161 (+904364us) service_pool.cc:224] Handling call
0709 13:18:06.846037 (+ 62876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.961704 (+115667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.961776 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:07.018487 (+ 56711us) log.cc:844] Serialized 26 byte log entry
0709 13:18:07.286475 (+267988us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:07.286537 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.286803 (+   266us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.595225 (+308422us) raft_consensus.cc:1685] finished
0709 13:18:07.595247 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.595437 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6870,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":5320,"thread_start_us":251415,"threads_started":1,"wal-append.queue_time_us":459643}
I20260709 13:18:07.625247 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.628974 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:07.624706 13231 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:07.646479 13690 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.646770 13959 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.647284 13959 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.625985 13983 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:07.648576 13983 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.641489 13437 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.649755 10203 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
W20260709 13:18:07.641963 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69011.1us
I20260709 13:18:07.650172 13983 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:07.642447 13826 consensus_queue.cc:1243] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 11.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:07.637976 13967 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:07.651523 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108919us
W20260709 13:18:07.651937 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 106185us
I20260709 13:18:07.652794 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.652674 13967 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.654521 13924 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:07.655619 13826 raft_consensus.cc:2764] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:07.655447 13995 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:07.574643 13691 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.572849 14000 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:18:07.642648 10202 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:07.619676 13838 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.646142 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 174725us
I20260709 13:18:07.654779 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:07.671352 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.672276 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:07.676245 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:07.681255 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:07.676268 13838 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:07.685223 13943 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.602s	user 0.005s	sys 0.007s
I20260709 13:18:07.685436 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:07.686318 10129 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:07.687309 13826 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.689826 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:07.690202 13924 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:07.691206 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 630) took 1539 ms. Trace:
I20260709 13:18:07.691641 10894 rpcz_store.cc:276] 0709 13:18:06.151512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.151684 (+   172us) service_pool.cc:224] Handling call
0709 13:18:06.152102 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.884817 (+732715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.690831 (+806014us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b5400953b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:07.691086 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1626144000}
I20260709 13:18:07.695225 13437 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:07.696257 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.696615 10130 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:07.698343 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:07.699373 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:07.701445 13344 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:07.702893 13786 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:07.704661 13344 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:07.708060 13877 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:07.708894 13877 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:07.709050 13943 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.710114 13877 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.710546 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.712210 13790 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:07.713655 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.714155 13961 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:07.714831 13961 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:07.720705 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:07.695405 13991 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:07.737426 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:07.738926 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 199484us
I20260709 13:18:07.719683 13790 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.731622 13231 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:07.772619 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:07.773924 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73416.1us
I20260709 13:18:07.774911 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.778126 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.782368 10200 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:07.747051 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.783244 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 128620us
I20260709 13:18:07.784109 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:07.784797 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:07.786824 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73302.5us
I20260709 13:18:07.713660 13838 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.742064 13991 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.788790 10894 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:07.729354 13196 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:18:07.744923 13826 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.803361 13843 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:07.803568 13826 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.802891 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.798695 13691 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.803928 13843 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.788455 13932 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.625s	user 0.014s	sys 0.002s
I20260709 13:18:07.803618 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.817699 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.822237 13843 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:07.824494 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.824517 10900 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:18:07.826092 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93138.8us
I20260709 13:18:07.828941 14000 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:07.829445 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:07.796926 13991 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:07.832216 13932 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.840075 13630 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:07.845324 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:07.821123 13810 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:07.841013 13367 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:18:07.860780 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.864837 13932 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:07.865298 13800 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.889475 13800 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.889977 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.891316 13785 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:07.866020 13663 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:07.893508 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:07.894009 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.894418 10203 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:07.892659 13800 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.869412 13845 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.869830 13843 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.872954 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.896731 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.899789 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.874163  9954 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 5 to 3, term changed from 8 to 6, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from VOTER to NON_VOTER. New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:18:07.877975 13752 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:07.881001 13810 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:07.883097 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62245.4us
I20260709 13:18:07.841744 13196 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:18:07.841070 13630 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:07.897481 13843 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.912416 13845 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:07.908017 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79263.3us
I20260709 13:18:07.877846 13838 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:07.909274 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:07.907374 14009 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:18:07.910326 13196 consensus_queue.cc:1243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:18:07.918007 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 136847us
W20260709 13:18:07.931416 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 128607us
W20260709 13:18:07.908037 14003 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:18:07.959156 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.933678 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:07.907446 13999 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.328s	user 0.004s	sys 0.000s
I20260709 13:18:07.931777 13231 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:07.974249 13231 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.975875 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1814) took 2293 ms. Trace:
I20260709 13:18:07.976001 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:07.976548 10047 rpcz_store.cc:276] 0709 13:18:05.544680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.544883 (+   203us) service_pool.cc:224] Handling call
0709 13:18:05.545535 (+   652us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:07.472299 (+1926764us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b1800383bc0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa277ffcbfb 00007fa27800764b
0709 13:18:07.493968 (+ 21669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.494003 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:07.553910 (+ 59907us) log.cc:844] Serialized 78 byte log entry
0709 13:18:07.720842 (+166932us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:07.783046 (+ 62204us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.786083 (+  3037us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.796763 (+ 10680us) log.cc:844] Serialized 14 byte log entry
0709 13:18:07.797120 (+   357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.822101 (+ 24981us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.831928 (+  9827us) raft_consensus.cc:1685] finished
0709 13:18:07.831950 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:07.837891 (+  5941us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17911,"prepare.queue_time_us":311701,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":100047,"raft.run_cpu_time_us":611,"raft.run_wall_time_us":26219,"spinlock_wait_cycles":43238656,"thread_start_us":264784,"threads_started":3,"wal-append.queue_time_us":214468}
W20260709 13:18:07.980528 13972 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.353s	user 0.012s	sys 0.012s
W20260709 13:18:07.972193 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.972631 13967 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:07.972924 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:07.969434 13845 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:07.997063  9954 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 4 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:07.997889 13961 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:07.989809 13967 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:07.999795 13967 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:08.003377 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:07.967779 14016 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:18:08.010316 13725 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:08.012715 13997 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.432s	user 0.001s	sys 0.000s
I20260709 13:18:08.012713 13725 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:08.014648 13725 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:07.980608 13826 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.021132 13725 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:08.019420 13961 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.007450 13845 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:07.931493 13989 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.034646 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49048.6us
I20260709 13:18:07.913736 13993 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:08.037663 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 208365us
W20260709 13:18:08.023041 14003 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:18:08.060184 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66406.9us
I20260709 13:18:08.035518 13845 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:08.080816 14005 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:18:08.081770 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:08.078604  9954 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 3 to 4, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 4 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:08.090764 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 176572us
I20260709 13:18:08.096772 10051 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:18:08.085063 13845 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.161531 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:08.149631 13992 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:18:08.052815 13989 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.164759 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1816) took 2552 ms. Trace:
W20260709 13:18:08.169381 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.168215 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.164757 13923 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.024s	sys 0.034s
W20260709 13:18:08.176023 13923 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.025s	sys 0.034s
W20260709 13:18:08.176208 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94676.9us
I20260709 13:18:08.178257 13686 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:08.105465 14021 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:08.170747 10053 rpcz_store.cc:276] 0709 13:18:05.547344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.691159 (+143815us) service_pool.cc:224] Handling call
0709 13:18:05.707892 (+ 16733us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:05.716222 (+  8330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:05.716268 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:05.716869 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:18:07.354676 (+1637807us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b180036c9a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:18:07.536895 (+182219us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:07.536983 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:07.544414 (+  7431us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:07.895511 (+351097us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:08.099379 (+203868us) raft_consensus.cc:1685] finished
0709 13:18:08.099416 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.099637 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16588,"raft.queue_time_us":103201,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":20335,"spinlock_wait_cycles":94755456,"thread_start_us":127261,"threads_started":1,"wal-append.queue_time_us":1756551}
I20260709 13:18:08.095275 13959 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.164898 13574 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
W20260709 13:18:08.181174 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.204624 10891 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:08.155786 13752 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:18:08.222543 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:08.228734 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:08.229683 10205 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.217672  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 4 to 7, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 7 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.184294 14021 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:08.240072 10201 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:18:08.240350 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:08.241014 10204 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.200145 13823 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:08.250582 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54410.5us
I20260709 13:18:08.224931 13752 consensus_queue.cc:1243] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:08.264693 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:08.269431 10202 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:08.270829 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:08.242241 13961 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:08.242838 13993 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.243155 14021 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:08.285257 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136639us
W20260709 13:18:08.289546 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109524us
I20260709 13:18:08.239086 14019 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:18:08.253328 13573 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.294062 13573 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.324226 14021 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.336824  9954 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.339311 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:08.342123 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.295513 13692 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:08.353727 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45710.2us
I20260709 13:18:08.355934 10204 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:08.358870 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94855.5us
I20260709 13:18:08.360666 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:08.358203 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:08.295725 13826 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:08.371973  9954 catalog_manager.cc:5697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:08.382776 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76403.4us
I20260709 13:18:08.357542 14021 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.390035 13800 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:08.391073 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:08.366178 13367 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:08.370393 14016 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:08.402565 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.386113 13991 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.001s
I20260709 13:18:08.407147 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:08.412189 10205 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:08.366658 13574 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:18:08.415602 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:08.363292 13967 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.404716 14001 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:08.418084 14016 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:08.444998  9954 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 2 to 6, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 6 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.447613 13983 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:08.449556 14031 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:08.413048 13367 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.454276 13367 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.457975 10197 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
W20260709 13:18:08.459916 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61072.5us
I20260709 13:18:08.461308 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:08.420802 13991 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.467453 14021 raft_consensus.cc:2764] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:08.468923 13989 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.469566 13991 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.473158 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:08.450865 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:08.480981 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.427907 13877 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:08.500120 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.514503 10128 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:08.515466 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:08.528116 10126 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:18:08.510257 14029 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.077s	user 0.000s	sys 0.004s
W20260709 13:18:08.539314 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56097us
I20260709 13:18:08.493762 14001 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:08.501717 13573 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.519629 13800 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.523212 13877 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:08.572280 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:08.538898 13983 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:08.563336 14004 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:08.600279 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.602218 10122 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:08.604524 10122 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:08.606642 13877 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.571544 13573 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.617995 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 167359us
W20260709 13:18:08.629803 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 108225us
I20260709 13:18:08.601727 13785 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.641513 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74743.9us
W20260709 13:18:08.657922 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.628624 13991 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:08.581736 13800 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.671140 10054 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 11. Ops: []
I20260709 13:18:08.633456 13692 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:18:08.641538 13924 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
W20260709 13:18:08.675424 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.533s (SENT) [suppressed 25 similar messages]
I20260709 13:18:08.636502 13573 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.659338 13686 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:18:08.695762 13692 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:08.696399 13692 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:08.697472 13692 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.700651 10202 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:08.595638 14016 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.703673 10198 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:08.685854 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.714653 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:08.726058 10127 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:08.662209 13785 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.699012 13692 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:08.732662 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.737360 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:08.738627 10204 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:08.744613 10131 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:08.745358 10203 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:18:08.747074 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.756094 14043 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:08.763275 13875 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
W20260709 13:18:08.792712 13725 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.757s	user 0.014s	sys 0.000s
I20260709 13:18:08.794216 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.795938 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:08.798722 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 408) took 1244 ms. Trace:
I20260709 13:18:08.800073 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:08.801088 10127 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:08.799257 10198 rpcz_store.cc:276] 0709 13:18:07.554619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.554774 (+   155us) service_pool.cc:224] Handling call
0709 13:18:08.798477 (+1243703us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003106d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:08.798620 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2596252416}
I20260709 13:18:08.803884 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.804654 10131 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:08.805652 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:08.806437 10131 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:08.728122 13967 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.810232 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:08.810720 13785 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:08.811617 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109688us
I20260709 13:18:08.812664 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:08.813798 13800 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.814533 13800 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:08.812708 13573 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:08.818819 13845 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.000s
I20260709 13:18:08.742873 14001 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:08.814774 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:08.819682 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:08.828408 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:08.835491 10131 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:08.814975 13800 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:08.840678 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:08.843271 10895 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:18:08.845142 10096 leader_election.cc:336] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.987s (SENT)
I20260709 13:18:08.846017 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:08.845619 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:08.849750 10891 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:08.826020 13725 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.848073 10863 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.107s (SENT)
I20260709 13:18:08.826381 14001 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:08.858325 13725 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.846233 13800 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:08.861554 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:08.866545 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.863534 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:08.872653 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57984.7us
I20260709 13:18:08.827931 13845 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:08.874048 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 634) took 1245 ms. Trace:
I20260709 13:18:08.880721 10900 rpcz_store.cc:276] 0709 13:18:07.620870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.621107 (+   237us) service_pool.cc:224] Handling call
0709 13:18:08.865864 (+1244757us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b540063ef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:08.866082 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428100864}
I20260709 13:18:08.889810 10055 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:08.890229 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:08.891448 13790 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:08.891770 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:08.892825 10055 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:08.895915 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:08.896975 10050 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:08.902242 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:08.903625 13875 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:08.904235 13875 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.904656 13875 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:08.905398 14026 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.639s	user 0.000s	sys 0.004s
W20260709 13:18:08.906648 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70454.2us
W20260709 13:18:08.907454 10096 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.016s (SENT)
I20260709 13:18:08.907928 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:08.910388 13725 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:08.914475 13790 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:08.915428 13790 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:08.864696 14021 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:08.917793 13845 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:08.874920 13231 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:08.865067 14001 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:08.921154 13921 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:08.921707 13921 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:08.921746 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:08.922559 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.884737 13785 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:08.924547 10123 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:08.957255 10054 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:18:08.958729 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 116514us
W20260709 13:18:08.980510 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.985749 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 112690us
I20260709 13:18:08.997298 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:08.998409 10895 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:09.000344 10897 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
W20260709 13:18:09.001868 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.000573 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:08.984534 13984 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.260s	user 0.020s	sys 0.024s
I20260709 13:18:08.974902 13932 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:08.977901 13967 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:09.020210 13984 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.021s	sys 0.024s
W20260709 13:18:09.033978 14052 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:18:09.059244 10050 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:09.071799 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 210380us
I20260709 13:18:09.072654 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:09.019526 13932 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.082013 10130 raft_consensus.cc:1217] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-10.6]   Dedup: 6.4->[8.5-10.6]
W20260709 13:18:09.080952 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61198.3us
I20260709 13:18:09.085014 10130 pending_rounds.cc:85] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:18:09.091130 10130 pending_rounds.cc:107] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:18:09.091548 10130 raft_consensus.cc:2889] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:09.083921 14054 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:18:09.084344 13932 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:09.111871 14001 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.068377 13967 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.097460 13790 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.073761 14062 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:18:09.138850 13942 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:09.150209 14062 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:09.153527 13810 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.139887 13993 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:09.154500 13993 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:09.155387 13785 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.156095 13785 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.155654 13993 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.156512 14001 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.157466 13993 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:09.159380 14058 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:18:09.141660 14004 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:09.162120 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 385) took 3651 ms (client timeout 2347 ms). Trace:
I20260709 13:18:09.161990 14004 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.146946 13921 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.162461 10200 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:09.163311 13921 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.166981 13967 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.131527 14016 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:18:09.168883 14004 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.168870 14016 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:09.169934 14016 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:09.171005 14016 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.171532 13838 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.172451 14016 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:09.172319 13838 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.175464 13838 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:09.137562 13196 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:09.176215 13196 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.176632 13196 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.189074 13959 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:09.167006 10123 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:18:09.132722 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 110831us
I20260709 13:18:09.202491 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:09.150751 14062 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:09.162590 10197 rpcz_store.cc:269] 0709 13:18:05.510556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.510698 (+   142us) service_pool.cc:224] Handling call
0709 13:18:09.161790 (+3651092us) spinlock_profiling.cc:243] Waited 2.87 s on lock 0x7b54001ef0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:09.162002 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6601710848}
I20260709 13:18:09.220399 10895 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:09.222617 13776 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:18:09.227828 14062 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.227739 13196 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.232620 13196 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:09.234822 13800 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.237990 13776 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:09.237660 13932 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.238451 13776 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.238881 13776 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.243257 13231 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:09.245709 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:09.247720 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:09.250316 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1783) took 1042 ms. Trace:
I20260709 13:18:09.259405 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:09.234921 14057 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:18:09.258144 10200 rpcz_store.cc:276] 0709 13:18:08.207749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.207878 (+   129us) service_pool.cc:224] Handling call
0709 13:18:09.161999 (+954121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.163212 (+  1213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.249881 (+ 86669us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54001ef0d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:09.250160 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2193304448}
W20260709 13:18:09.263815 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 294791us
I20260709 13:18:09.270452 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:09.271333 10891 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:09.271502 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.272835 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.274446 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.270469 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.277534 10203 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:09.270764 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.278668 10199 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:18:09.280967 10202 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:09.267243 10123 pending_rounds.cc:85] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 3
I20260709 13:18:09.282829 10123 pending_rounds.cc:107] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:18:09.283360 10123 raft_consensus.cc:2889] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:09.276144 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.273031 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:09.294077 10205 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:09.237691 13800 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.305307 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.304266 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:09.307353 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74137.1us
I20260709 13:18:09.304580 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.341611 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.341902 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:09.343905 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:09.335055 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 249244us
I20260709 13:18:09.259442 13875 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.342904 10053 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:09.349390 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.296993 13961 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.296890 14018 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:09.352365 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:09.354651 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.364347 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:09.367933 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.366395 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.370007 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.375396 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.380427 10054 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:09.375531 10048 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:09.381832 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.381635 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.359827 14018 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:09.417192 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.378057 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.421711 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.423728 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.424801 10055 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:09.425868 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.420253 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.418524 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:09.434720 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:09.436976 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:09.437278 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 139044us
W20260709 13:18:09.422842 14038 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.850s	user 0.004s	sys 0.001s
I20260709 13:18:09.439801 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:09.420347 14018 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:09.449497 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.450289 10196 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.451673 14084 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:09.452533 14084 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.453068 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:09.453110 14084 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:09.456151 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.456302 10049 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:09.457999 10048 raft_consensus.cc:2410] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:09.368353 13961 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.425277 13932 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.464157 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:09.375288 13785 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.466174 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.429677 14075 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:09.437041 14078 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:09.450599 13921 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:18:09.528041 13776 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.539831 13231 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:09.540417 13231 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:09.541424 13776 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:09.542896 14079 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:18:09.544500 10196 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:09.545612 10202 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:18:09.546618 13800 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.547538 13800 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:09.550354 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 80723.5us
I20260709 13:18:09.488116 14075 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:09.554534 14075 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.486397 14078 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:09.519848 14022 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.350s	user 0.002s	sys 0.004s
I20260709 13:18:09.568444 10891 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:09.581879 13875 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:09.541414 13231 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.626402 10056 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:09.633069 10051 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:09.645066 10049 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:09.649890 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.647711 10196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:09.656771 10056 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:09.635330 13231 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:09.663574 13923 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.624248 14078 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.706918 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.710670 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.708850 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:09.714392 13942 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.714596 14073 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.715236 14073 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.715163 13942 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.714984 10020 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:18:09.711485 10056 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:09.701671 14064 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
I20260709 13:18:09.731405 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.735497 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.728559 13932 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:18:09.745132 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.746095 10205 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:09.751854 14086 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
W20260709 13:18:09.746138 14082 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260709 13:18:09.755991 14081 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.304s	user 0.002s	sys 0.000s
I20260709 13:18:09.757372 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:09.717742 14073 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.759889 13785 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.766580 10205 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:09.769444 10051 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:09.773183 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:09.786124 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:09.787261 10199 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:09.717773 13942 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:09.790867 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:09.801981 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:09.815163 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.815907 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:09.766124 13961 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:09.702832 13923 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:09.801424 14067 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:09.809639 14073 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:09.826355 13991 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:09.884145 14073 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:09.896142 14073 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:09.938122 10199 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:09.870153 13785 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:09.940560 10205 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:09.978334 10196 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:09.885907 14067 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:09.919739 13686 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.202103 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.145687 13686 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.142381 13961 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.187613 14018 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.721s	user 0.016s	sys 0.000s
W20260709 13:18:10.245894 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.33837e+06us
I20260709 13:18:09.980113 13785 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:10.120241 14097 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.419s	user 0.001s	sys 0.000s
I20260709 13:18:10.222756 13686 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:10.179644 14103 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:10.265830 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.266969 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.273943 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.233822 14084 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.002s
I20260709 13:18:10.118366 13942 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:10.119200 13932 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.302842 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.061s
W20260709 13:18:10.303467 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.015s	sys 0.061s
W20260709 13:18:10.303946 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 529584us
I20260709 13:18:10.301584 14084 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.312608 14084 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:10.309122 13932 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:10.313654 14076 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.933s	user 0.007s	sys 0.010s
I20260709 13:18:10.316147 13932 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:10.270273 14078 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:10.320024 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 106991us
I20260709 13:18:10.304679 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:10.322160 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:10.285518 13984 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.262254 14018 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.272101 13961 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.319648 14095 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.290918 14103 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.346827 13932 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:10.368880 10048 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:10.352066 13725 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.348836 13196 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:10.379235 10891 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:10.327106 13942 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:10.396677 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.381649 13826 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.381413 13923 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:10.296119 14067 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.352558 13574 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:18:10.376133 14021 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:10.357780 13984 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.471315 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:10.378806 13991 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:10.483330 10891 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:10.485853 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 203700us
I20260709 13:18:10.365957 14078 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.395673 13989 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:10.436465 13993 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:10.505722 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1759) took 1937 ms. Trace:
I20260709 13:18:10.525986 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:10.372159 14095 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.430867 13932 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.461515 13725 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.479206 13826 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.500072 14021 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:10.506260 13991 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.783566 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1749) took 1947 ms. Trace:
I20260709 13:18:10.847007 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:10.847008 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:10.712213 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.539572 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1758) took 1982 ms. Trace:
I20260709 13:18:10.877820 13993 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:10.549795 10048 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:10.840806 10122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.008s	sys 0.027s
W20260709 13:18:10.767503 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.891737 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.916842 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:10.522243 10096 connection.cc:256] RPC call timeout handler was delayed by -1.8863149769997563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2391s for a response
W20260709 13:18:10.934684 10096 connection.cc:256] RPC call timeout handler was delayed by -1.6240661159999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 13:18:10.935228 10096 connection.cc:256] RPC call timeout handler was delayed by -1.5938149769997381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2716s for a response
W20260709 13:18:10.935634 10096 connection.cc:256] RPC call timeout handler was delayed by -1.5729149729997971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2544s for a response
W20260709 13:18:10.935999 10096 connection.cc:256] RPC call timeout handler was delayed by -1.5452149769998869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.277s for a response
W20260709 13:18:10.936372 10096 connection.cc:256] RPC call timeout handler was delayed by -1.4782661159999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2005s for a response
W20260709 13:18:10.936759 10096 connection.cc:256] RPC call timeout handler was delayed by -1.4138149769999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2916s for a response
W20260709 13:18:10.937122 10096 connection.cc:256] RPC call timeout handler was delayed by -1.3936661159998494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:18:10.970410 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.013s	sys 0.026s
W20260709 13:18:10.971079 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.013s	sys 0.026s
W20260709 13:18:10.971679 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.028s
W20260709 13:18:10.972147 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.011s	sys 0.028s
I20260709 13:18:10.841506 10131 rpcz_store.cc:276] 0709 13:18:08.568803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.840997 (+272194us) service_pool.cc:224] Handling call
0709 13:18:08.855302 (+ 14305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:09.056750 (+201448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.056797 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:09.081437 (+ 24640us) log.cc:844] Serialized 52 byte log entry
0709 13:18:09.130140 (+ 48703us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:09.130207 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.130304 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.312516 (+1182212us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:10.494730 (+182214us) raft_consensus.cc:1685] finished
0709 13:18:10.494768 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.505617 (+ 10849us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10262,"raft.queue_time_us":899103,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"spinlock_wait_cycles":202636672,"thread_start_us":231010,"threads_started":2,"wal-append.queue_time_us":260830}
I20260709 13:18:10.875097 10126 rpcz_store.cc:276] 0709 13:18:08.557294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.811799 (+254505us) service_pool.cc:224] Handling call
0709 13:18:08.812324 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.838466 (+ 26142us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:08.848192 (+  9726us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.149709 (+301517us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b180000f2e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:10.232138 (+1082429us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.244507 (+ 12369us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.244754 (+   247us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:10.516416 (+271662us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.523316 (+  6900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.523451 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.539205 (+ 15754us) raft_consensus.cc:1685] finished
0709 13:18:10.539242 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.539472 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1303810,"prepare.queue_time_us":378424,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":2487,"raft.queue_time_us":1246416,"raft.run_cpu_time_us":567,"raft.run_wall_time_us":46277,"spinlock_wait_cycles":6385536,"thread_start_us":455734,"threads_started":2,"wal-append.queue_time_us":601868}
W20260709 13:18:11.009697 13231 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.351s	user 0.007s	sys 0.007s
W20260709 13:18:10.691473 14110 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.169s	user 0.001s	sys 0.001s
W20260709 13:18:10.782828 14115 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:18:10.574847 14066 raft_consensus.cc:1064] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:18:10.783331 14113 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:18:10.591363 14068 raft_consensus.cc:1064] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:18:10.625200 14064 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:18:10.842631 10125 rpcz_store.cc:276] 0709 13:18:08.478538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.478690 (+   152us) service_pool.cc:224] Handling call
0709 13:18:09.181786 (+703096us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:09.182025 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.397446 (+1215421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.397498 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:10.425524 (+ 28026us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:10.425575 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.425661 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.425828 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1228772,"raft.queue_time_us":1255445,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":8961,"spinlock_wait_cycles":4462336}
I20260709 13:18:10.696333 13983 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:11.016985 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1886) took 1182 ms. Trace:
W20260709 13:18:11.033874 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.614614 13785 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.045710 10052 rpcz_store.cc:276] 0709 13:18:09.834295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.201498 (+367203us) service_pool.cc:224] Handling call
0709 13:18:10.210665 (+  9167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.210980 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:10.258826 (+ 47846us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.016506 (+757680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.016586 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:11.016645 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.016729 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.016902 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46913,"prepare.queue_time_us":50463,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"thread_start_us":712358,"threads_started":1}
I20260709 13:18:10.932722 14019 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.077220 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:11.080864 10052 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:10.515055 14116 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.534s
I20260709 13:18:11.024292 14111 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:11.034643 14104 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.677s	user 0.001s	sys 0.001s
I20260709 13:18:11.083428 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:10.568866 13989 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.038355 13845 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.094305 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.116889 10126 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8.
W20260709 13:18:11.031769 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.013s	sys 0.013s
I20260709 13:18:11.051571 14018 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.102293 10125 raft_consensus.cc:2410] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:18:11.096133 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47590.2us
W20260709 13:18:11.110562 10124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.007s	sys 0.027s
W20260709 13:18:11.130875 10124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.007s	sys 0.027s
I20260709 13:18:11.045889 13574 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:11.122061 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.014s	sys 0.014s
W20260709 13:18:11.099205 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.004s	sys 0.024s
W20260709 13:18:11.138137 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.005s	sys 0.024s
I20260709 13:18:11.084144 13961 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:11.125830 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.014s	sys 0.026s
I20260709 13:18:11.086884 13993 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:11.149712 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.014s	sys 0.026s
I20260709 13:18:11.095162 14130 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.162437 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:11.119316 14115 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:18:11.172755 10019 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.516s (SENT)
I20260709 13:18:11.166640 10131 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:11.173542 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:11.127182 10051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.020s	sys 0.015s
I20260709 13:18:11.179490 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.075394 13983 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:11.169174 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 166565us
I20260709 13:18:11.120447 14111 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:11.185225 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:11.216414 10051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.024s	sys 0.015s
I20260709 13:18:11.222348 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.225354 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 395) took 1413 ms. Trace:
I20260709 13:18:11.226058 10056 rpcz_store.cc:276] 0709 13:18:09.812379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.812589 (+   210us) service_pool.cc:224] Handling call
0709 13:18:09.813031 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.813280 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.813306 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.826103 (+ 12797us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.185784 (+359681us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:10.185848 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.185960 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.224986 (+1039026us) raft_consensus.cc:1685] finished
0709 13:18:11.225027 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.225251 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":311282,"threads_started":1,"wal-append.queue_time_us":545946}
I20260709 13:18:11.161862 14118 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.270603 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1761) took 2665 ms. Trace:
W20260709 13:18:11.290009 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117280us
I20260709 13:18:11.242213 10125 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:11.242992 10096 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.391s (SENT)
I20260709 13:18:11.233884 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:11.295449 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:11.295881 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:11.296546 10203 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:11.243912 13826 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.244421 14095 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.247047 14128 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.249511 13989 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:11.250175 10021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.016s	sys 0.011s
W20260709 13:18:11.325215 10021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.016s	sys 0.011s
I20260709 13:18:11.251260 13991 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.162000 14103 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.128453 13845 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.255724 13984 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.256269 14134 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:11.257843 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.260464 13800 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.260641 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1767) took 2653 ms. Trace:
I20260709 13:18:11.261358 13877 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.262420 14121 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:18:11.247200 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1764) took 2640 ms. Trace:
I20260709 13:18:11.265471 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2898) took 1451 ms. Trace:
I20260709 13:18:11.268759 14133 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.107990 14019 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.383843 10205 rpcz_store.cc:276] 0709 13:18:09.814088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.815011 (+   923us) service_pool.cc:224] Handling call
0709 13:18:09.939285 (+124274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.953386 (+ 14101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.265182 (+1311796us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54003106d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:11.265375 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284668800}
I20260709 13:18:11.276993 10124 rpcz_store.cc:276] 0709 13:18:08.605110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.868778 (+263668us) service_pool.cc:224] Handling call
0709 13:18:08.895195 (+ 26417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.365539 (+1470344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.365603 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.379969 (+ 14366us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.131547 (+751578us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:11.131622 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.131756 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.270325 (+138569us) raft_consensus.cc:1685] finished
0709 13:18:11.270350 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.270520 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1400670,"raft.queue_time_us":1445203,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":10956,"thread_start_us":643809,"threads_started":1,"wal-append.queue_time_us":830088}
I20260709 13:18:11.295513 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:11.295495 14118 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.311790 14078 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.312353 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:11.313282 13725 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.309026 14111 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.323053 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:11.155906 13574 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.423954 10203 raft_consensus.cc:2368] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11.
I20260709 13:18:11.425339 13574 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:11.349030 10095 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items
W20260709 13:18:11.351848 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 304696us
I20260709 13:18:11.335778 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:11.311816 14089 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:18:11.269266 14068 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.365711 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:11.161804 13942 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.376341 10131 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:11.389191 10048 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.005s
I20260709 13:18:11.320940 14137 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.348278 14116 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.394791 10019 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.542s (SENT)
I20260709 13:18:11.394196 14133 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.396876 13800 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.396781 14138 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.181344 14130 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.397995 13845 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.399350 10126 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:11.398942 14019 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.402942 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1763) took 2797 ms. Trace:
I20260709 13:18:11.394428 13877 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.409129 13983 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.354282 14128 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.534935 14128 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:11.355257 14114 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:11.536016 14114 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.356069 14143 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:11.372044 10128 rpcz_store.cc:276] 0709 13:18:08.607453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.175948 (+568495us) service_pool.cc:224] Handling call
0709 13:18:09.195202 (+ 19254us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:10.461564 (+1266362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.461606 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:10.501380 (+ 39774us) log.cc:844] Serialized 52 byte log entry
0709 13:18:11.056835 (+555455us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:11.056900 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.057025 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.260301 (+203276us) raft_consensus.cc:1685] finished
0709 13:18:11.260331 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.260559 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1148067,"raft.queue_time_us":1186372,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":30223,"thread_start_us":644823,"threads_started":2,"wal-append.queue_time_us":730111}
I20260709 13:18:11.416594 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2901) took 1592 ms. Trace:
I20260709 13:18:11.381665 10122 rpcz_store.cc:276] 0709 13:18:08.606354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.161124 (+554770us) service_pool.cc:224] Handling call
0709 13:18:09.161599 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.282290 (+1120691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.282363 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.289964 (+  7601us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.887937 (+597973us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:11.016574 (+128637us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.034989 (+ 18415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.035085 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.246817 (+211732us) raft_consensus.cc:1685] finished
0709 13:18:11.246851 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.247056 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1214474,"prepare.queue_time_us":128634,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":1324062,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":35643,"thread_start_us":547486,"threads_started":1,"wal-append.queue_time_us":795674}
I20260709 13:18:11.431591 10048 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:11.432565 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 96899.3us
W20260709 13:18:11.459195 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.504390 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:11.509649 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:11.456524 14073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.011s	sys 0.022s
I20260709 13:18:11.429900 13932 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.528923 10900 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:11.568265 10125 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:11.530658 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.569509 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1895) took 1057 ms. Trace:
I20260709 13:18:11.524703 10130 rpcz_store.cc:276] 0709 13:18:08.605983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.081157 (+475174us) service_pool.cc:224] Handling call
0709 13:18:09.081723 (+   566us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:10.278283 (+1196560us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:10.278333 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:10.286936 (+  8603us) log.cc:844] Serialized 52 byte log entry
0709 13:18:10.972699 (+685763us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:10.972769 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.972885 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.402629 (+429744us) raft_consensus.cc:1685] finished
0709 13:18:11.402652 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.402860 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1213958,"raft.queue_time_us":1203900,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":20335,"thread_start_us":635229,"threads_started":1,"wal-append.queue_time_us":1084236}
I20260709 13:18:11.562436 10894 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:11.570549 10048 rpcz_store.cc:276] 0709 13:18:10.504532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.504838 (+   306us) service_pool.cc:224] Handling call
0709 13:18:11.561645 (+1056807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.540841 10897 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:18:11.561877 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.456869 14129 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:11.457700 14134 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:11.539700 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 296757us
W20260709 13:18:11.580185 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.565063 10197 rpcz_store.cc:276] 0709 13:18:09.823764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.824079 (+   315us) service_pool.cc:224] Handling call
0709 13:18:10.203001 (+378922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.212010 (+  9009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.212051 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.317468 (+105417us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:10.318258 (+   790us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.697578 (+379320us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:10.697655 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.697769 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.416163 (+718394us) raft_consensus.cc:1685] finished
0709 13:18:11.416186 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.416506 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11485,"raft.queue_time_us":1535,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":18179,"spinlock_wait_cycles":29608704,"thread_start_us":378421,"threads_started":1,"wal-append.queue_time_us":1095122}
W20260709 13:18:11.580756 10123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.001s	sys 0.012s
W20260709 13:18:11.594729 10123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.001s	sys 0.013s
W20260709 13:18:11.581379 10049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.005s	sys 0.009s
W20260709 13:18:11.563771 14073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.221s	user 0.012s	sys 0.022s
W20260709 13:18:11.595402 10049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.980s	user 0.006s	sys 0.009s
I20260709 13:18:11.539050 14114 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.597513 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1896) took 1077 ms. Trace:
I20260709 13:18:11.507324 14137 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.605271 14137 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.606478 14130 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.584828 14151 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.426178 10095 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 350)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
I20260709 13:18:11.608728 10049 rpcz_store.cc:276] 0709 13:18:10.520506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.520868 (+   362us) service_pool.cc:224] Handling call
0709 13:18:10.547271 (+ 26403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.547546 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:10.615079 (+ 67533us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.597061 (+981982us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.597145 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:11.597199 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.597275 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.597440 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52963,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":111,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":981050,"threads_started":1}
I20260709 13:18:11.601001 14133 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.569291 14143 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.616021 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:11.586726 14134 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.562245 13877 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:11.618528 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 223781us
I20260709 13:18:11.568214 14078 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.577874 13776 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.519354 13800 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:11.581692 14083 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.004s	sys 0.010s
W20260709 13:18:11.636026 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.716s (SENT)
I20260709 13:18:11.670729 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:11.637876 10048 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:11.568075 13942 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:11.641495 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:11.657193 10126 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:11.519852 14116 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:11.665582 10052 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.584s	user 0.014s	sys 0.002s
I20260709 13:18:11.672484 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:11.678797 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1898) took 1102 ms. Trace:
I20260709 13:18:11.679455 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:11.680565 10048 raft_consensus.cc:2410] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:18:11.681418 10052 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:11.679591 10053 rpcz_store.cc:276] 0709 13:18:10.577820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.578599 (+   779us) service_pool.cc:224] Handling call
0709 13:18:10.612718 (+ 34119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.613056 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:11.341319 (+728263us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.678243 (+336924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.678336 (+    93us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:11.678396 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.678496 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.678704 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":403756,"thread_start_us":633871,"threads_started":2}
I20260709 13:18:11.548368 14138 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.660176 14134 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:11.675316 14083 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.004s	sys 0.010s
I20260709 13:18:11.688588 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 399) took 1013 ms. Trace:
I20260709 13:18:11.696439 10052 rpcz_store.cc:276] 0709 13:18:10.669650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.065986 (+396336us) service_pool.cc:224] Handling call
0709 13:18:11.682619 (+616633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.698485 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 397) took 1085 ms. Trace:
I20260709 13:18:11.698995 10052 rpcz_store.cc:276] 0709 13:18:10.612965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.697418 (+1084453us) service_pool.cc:224] Handling call
0709 13:18:11.697817 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.698038 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.698075 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.698135 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:11.698183 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.698266 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.698418 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:11.700474 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 241891us
I20260709 13:18:11.639436 14137 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:18:11.722784 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 350) took 3032 ms. Trace:
W20260709 13:18:11.731499 10863 connection.cc:256] RPC call timeout handler was delayed by -2.0930327209998723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:18:11.723263 10895 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:18:11.686648 14129 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:11.731594 10095 rpcz_store.cc:276] 0709 13:18:08.689999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.704366 (+3014367us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:11.730464 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 398) took 1069 ms. Trace:
I20260709 13:18:11.735821 10052 rpcz_store.cc:276] 0709 13:18:10.661117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.711606 (+1050489us) service_pool.cc:224] Handling call
0709 13:18:11.729477 (+ 17871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.729800 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.729975 (+   175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.730046 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:11.730095 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.730180 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.730367 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.639807 14151 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:11.732355 10863 connection.cc:256] RPC call timeout handler was delayed by -1.9676989169997796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31770000000000004s for a response
W20260709 13:18:11.745278 10863 connection.cc:256] RPC call timeout handler was delayed by -1.7776901390000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:18:11.745831 10863 connection.cc:256] RPC call timeout handler was delayed by -1.6941186059998472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
W20260709 13:18:11.746233 10863 connection.cc:256] RPC call timeout handler was delayed by -1.5186503719999109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 13:18:11.746856 10863 connection.cc:256] RPC call timeout handler was delayed by -1.0803503719998844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
I20260709 13:18:11.674778 13776 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.671718 14078 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:11.766680 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 130722us
I20260709 13:18:11.764516 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:11.725981 14072 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.769264 10052 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:11.726039 13942 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:11.734733 10095 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45470 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
I20260709 13:18:11.773911 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1900) took 1106 ms. Trace:
I20260709 13:18:11.774744 10049 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:11.776368 10052 rpcz_store.cc:276] 0709 13:18:10.668135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.737330 (+1069195us) service_pool.cc:224] Handling call
0709 13:18:11.769046 (+ 31716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.773571 (+  4525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.773843 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.778921 14111 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:11.783823 14157 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:18:11.786041 14078 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.777595 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
I20260709 13:18:11.793349 14118 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.796049 14062 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:11.798446 10096 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.770s (SENT)
I20260709 13:18:11.799003 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:11.791445 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
I20260709 13:18:11.771531 14095 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:11.804898 10125 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:11.802706 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
W20260709 13:18:11.809242 14108 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.439s	user 0.005s	sys 0.012s
I20260709 13:18:11.809376 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
I20260709 13:18:11.814611 13826 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.815248 13826 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.818799 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 425) took 2352 ms. Trace:
I20260709 13:18:11.819276 10202 rpcz_store.cc:276] 0709 13:18:09.466919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.467317 (+   398us) service_pool.cc:224] Handling call
0709 13:18:09.545255 (+ 77938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.546184 (+   929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.818521 (+2272337us) spinlock_profiling.cc:243] Waited 64.6 ms on lock 0x7b54001ff750. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:11.818693 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148552448}
I20260709 13:18:11.807139 14118 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:11.817837 13826 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:11.813578 14109 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.322s	user 0.001s	sys 0.000s
W20260709 13:18:11.824654 14154 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:18:11.825924 14156 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:18:11.826056 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 639) took 2053 ms (client timeout 1616 ms). Trace:
I20260709 13:18:11.822674 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
I20260709 13:18:11.780210 13942 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:11.780369 14072 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:11.826517 10891 rpcz_store.cc:269] 0709 13:18:09.772527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.772764 (+   237us) service_pool.cc:224] Handling call
0709 13:18:11.825818 (+2053054us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b5400041854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:11.825970 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1391260928}
W20260709 13:18:11.897562 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.855933 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76420.7us
W20260709 13:18:11.859484 10863 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.533s (SENT)
W20260709 13:18:11.953282 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93826.4us
I20260709 13:18:11.855705 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1768) took 3247 ms. Trace:
I20260709 13:18:11.930411 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:11.955255 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.020s (SENT)
I20260709 13:18:11.955482 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:18:11.959066 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.959755 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1872) took 2340 ms. Trace:
I20260709 13:18:11.954749 10123 rpcz_store.cc:276] 0709 13:18:08.607784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.219356 (+611572us) service_pool.cc:224] Handling call
0709 13:18:09.266884 (+ 47528us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:10.618256 (+1351372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.618308 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:10.854461 (+236153us) log.cc:844] Serialized 52 byte log entry
0709 13:18:11.595362 (+740901us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:11.595418 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.595516 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.827196 (+231680us) raft_consensus.cc:1685] finished
0709 13:18:11.827218 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.855585 (+ 28367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1094521,"raft.queue_time_us":1107192,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9667,"thread_start_us":833876,"threads_started":2,"wal-append.queue_time_us":679113}
W20260709 13:18:11.964962 10096 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.916s (SENT)
I20260709 13:18:11.962904 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.966645 10122 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:11.859949 14143 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.862370 13800 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:11.970376 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2918) took 1294 ms. Trace:
I20260709 13:18:11.969066 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:11.971171 10122 rpcz_store.cc:276] 0709 13:18:10.673340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.962567 (+1289227us) service_pool.cc:224] Handling call
0709 13:18:11.967197 (+  4630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.970393 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.959610 14078 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:11.972687 10123 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:11.860985 14116 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:11.961625 10054 rpcz_store.cc:276] 0709 13:18:09.618906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.631889 (+ 12983us) service_pool.cc:224] Handling call
0709 13:18:09.652337 (+ 20448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.652792 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:10.016139 (+363347us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.337441 (+321302us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.347489 (+ 10048us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.267647 (+920158us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18001c1e60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:11.824021 (+556374us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b18001c1e60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:18:11.958961 (+134940us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.959268 (+   307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.959357 (+    89us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:11.959432 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.959516 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.959676 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388334,"prepare.queue_time_us":73082,"prepare.run_cpu_time_us":241,"prepare.run_wall_time_us":18068,"raft.queue_time_us":1904808,"raft.run_cpu_time_us":563,"raft.run_wall_time_us":98666,"spinlock_wait_cycles":49715584,"thread_start_us":939640,"threads_started":4,"wal-append.queue_time_us":353201}
I20260709 13:18:11.980969 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.983450 10122 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:18:11.984335 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.828897 14095 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:11.987257 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:11.987994 10122 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:11.989907 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.990054 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:11.973899 13826 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:11.991233 10123 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:11.993923 14055 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 2.565s	user 0.016s	sys 0.028s
I20260709 13:18:11.995422 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1869) took 3778 ms. Trace:
I20260709 13:18:11.991796 10122 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:11.996932 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.995867 10047 rpcz_store.cc:276] 0709 13:18:08.217542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.563679 (+346137us) service_pool.cc:224] Handling call
0709 13:18:08.575390 (+ 11711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.583083 (+  7693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.583209 (+   126us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:08.617233 (+ 34024us) log.cc:844] Serialized 26 byte log entry
0709 13:18:08.863206 (+245973us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:08.863285 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.863406 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.881921 (+3018515us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:11.995102 (+113181us) raft_consensus.cc:1685] finished
0709 13:18:11.995157 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.995354 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9948,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":21537,"spinlock_wait_cycles":82852096,"thread_start_us":225861,"threads_started":1,"wal-append.queue_time_us":722115}
I20260709 13:18:11.998835 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.001495 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2921) took 1219 ms. Trace:
I20260709 13:18:12.003328 10122 rpcz_store.cc:276] 0709 13:18:10.830954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.996662 (+1165708us) service_pool.cc:224] Handling call
0709 13:18:12.001429 (+  4767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.002524 14068 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.978075 14078 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:12.009559 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51312.4us
I20260709 13:18:12.000988 13855 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.013509 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.017302 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:11.999715 13691 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:12.019912 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:12.004683 14130 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.029559 13823 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:12.034185 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 16) took 2389 ms. Trace:
I20260709 13:18:12.002494 14071 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:12.038436 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.039439 10131 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:12.040598 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.007841 13983 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.043082 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.006810 14068 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.010327 14019 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.034804 10196 rpcz_store.cc:276] 0709 13:18:09.644915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.645111 (+   196us) service_pool.cc:224] Handling call
0709 13:18:12.032451 (+2387340us) spinlock_profiling.cc:243] Waited 9.5 ms on lock 0x7b4c001ac6c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2762d0a20 00007fa2762e1a29 00007fa2762a7644 00007fa2762cef2d 00007fa2762ceb93 00007fa277ffec92 00007fa277ffe84a 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691
0709 13:18:12.033914 (+  1463us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:12.034028 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":771921536}
I20260709 13:18:12.011015 14129 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.002674 14063 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:12.044337 10122 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:12.057757 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.060714 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:12.062258 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.064343 10131 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:12.065651 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.066566 10122 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:12.049703 13800 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.082504 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.043638 13661 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:18:12.084004 10122 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:12.025496 13855 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.106694 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:12.070976 14148 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:18:12.097252 13800 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.043723 13691 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:12.101024 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.067435 13826 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:12.102388 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.114748 10122 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:12.115955 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:12.116596 10122 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:12.044255 14130 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:12.119185 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.065850 14118 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:12.049547 13983 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:12.106530 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44353.9us
I20260709 13:18:12.125438 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.051860 14019 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.127918 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:12.130586 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75630.3us
I20260709 13:18:12.123705 14118 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:12.109931 14167 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:18:12.117951 13959 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.108521 13942 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.055178 14129 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:12.112797 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.119270 13826 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.167519 13826 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:12.172724 13983 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.173566 13983 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:12.141058 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.181607 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75013.3us
I20260709 13:18:12.144083 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:12.143965 13691 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.142446 14173 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:12.143760 13959 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:12.145653 14075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.363s	user 0.011s	sys 0.018s
I20260709 13:18:12.150111 13800 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.154850 10123 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:12.187379 14071 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:12.192787 14071 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.193377 14118 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:12.165336 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.194111 14075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.464s	user 0.011s	sys 0.018s
I20260709 13:18:12.179289 10122 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:12.195581 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.196319 10122 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:12.196571 13959 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.197400 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:12.193262 14071 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:12.198717 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.179919 14019 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.193213 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:12.187050 14063 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:12.201364 14129 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.144285 14072 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.160867 14171 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:12.224800 14171 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:12.225914 14171 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.227483 14171 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:12.228089 13855 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.183880 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.205403 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:18:12.239012 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50772.7us
I20260709 13:18:12.231237 10123 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:12.240921 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:12.246953 14066 raft_consensus.cc:1068] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Unable to promote non-voter db052bef7536453fa2d7f5a00e314a18: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }.
  Pending config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.249431 13752 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.265748 13752 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.249729 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:12.267416 14068 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.267627 10053 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11. Re-sending same reply.
I20260709 13:18:12.268421 10201 raft_consensus.cc:2410] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:18:12.268714 14130 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:12.269652 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2880) took 3011 ms (client timeout 1562 ms). Trace:
W20260709 13:18:12.270295 10201 rpcz_store.cc:269] 0709 13:18:09.258673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.258828 (+   155us) service_pool.cc:224] Handling call
0709 13:18:12.269390 (+3010562us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:12.269572 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6914039168}
I20260709 13:18:12.251839 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.254427 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1904) took 1236 ms. Trace:
I20260709 13:18:12.272020 10124 raft_consensus.cc:2410] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:18:12.242239 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58493.2us
I20260709 13:18:12.272785 10053 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:12.273257 10053 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:12.273391 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.272207 10049 rpcz_store.cc:276] 0709 13:18:11.017949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.253513 (+1235564us) service_pool.cc:224] Handling call
0709 13:18:12.253898 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.254082 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.254108 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.254152 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:12.254199 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.254257 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.254363 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.260408 13826 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.261600 14019 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:12.223882 10094 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1793)
I20260709 13:18:12.277974 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:12.280227 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.281064 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2923) took 1263 ms. Trace:
I20260709 13:18:12.281662 10124 rpcz_store.cc:276] 0709 13:18:11.017257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.251529 (+1234272us) service_pool.cc:224] Handling call
0709 13:18:12.280972 (+ 29443us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:12.284484 14107 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.935s	user 0.008s	sys 0.013s
W20260709 13:18:12.263384 14134 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.009s
I20260709 13:18:12.286397 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.276572 13437 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:12.289915 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.298580 10123 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:12.299505 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2934) took 1034 ms. Trace:
I20260709 13:18:12.300691 10123 rpcz_store.cc:276] 0709 13:18:11.265020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.192889 (+927869us) service_pool.cc:224] Handling call
0709 13:18:12.299431 (+106542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.284991 14068 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:12.285281 14130 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:12.285079 14066 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:12.303483 10124 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:12.276757 14127 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:18:12.298545 14063 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:18:12.307482 14078 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.309607 10049 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:12.313892 13752 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.301153 14019 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:12.290678 14134 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.308821 13437 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:12.302515 14071 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.334112 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.339767 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.345105 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2924) took 1309 ms. Trace:
I20260709 13:18:12.346377 10128 rpcz_store.cc:276] 0709 13:18:11.035593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.339424 (+1303831us) service_pool.cc:224] Handling call
0709 13:18:12.345021 (+  5597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.294281 10096 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 363)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 365)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1796)
W20260709 13:18:12.352854 10096 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45470 dropped due to backpressure: service queue is full with 50 items [suppressed 8 similar messages]
I20260709 13:18:12.331912 14063 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:18:12.353691 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:12.356248 10201 raft_consensus.cc:606] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:12.356581 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.355036 14103 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:12.353328 10096 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45470 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1796)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
I20260709 13:18:12.358176 14103 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:12.359294 14103 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.360136 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.360576 14103 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:12.360949 10130 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:12.361622 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2941) took 1094 ms. Trace:
I20260709 13:18:12.361975 10130 rpcz_store.cc:276] 0709 13:18:11.267080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.359803 (+1092723us) service_pool.cc:224] Handling call
0709 13:18:12.361543 (+  1740us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.359562 10096 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 368)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 369)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 367)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1796)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
I20260709 13:18:12.367426 14134 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.369038 10049 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:12.371152 13437 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.377712 10053 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:12.379776 10053 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:12.383710 14021 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.386224 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2938) took 1119 ms. Trace:
I20260709 13:18:12.386709 10125 rpcz_store.cc:276] 0709 13:18:11.266275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.322549 (+ 56274us) service_pool.cc:224] Handling call
0709 13:18:12.385936 (+1063387us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540023b9d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:12.386081 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319971584}
I20260709 13:18:12.388873 13826 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.391554 14071 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:12.370153 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:12.396204 10053 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:12.396899 10053 pending_rounds.cc:85] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:18:12.397334 10053 pending_rounds.cc:107] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:12.397713 10053 raft_consensus.cc:2889] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:12.398389 10049 raft_consensus.cc:2368] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11.
I20260709 13:18:12.388015 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.399322 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.399457 10125 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:12.400712 10053 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:12.408321 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:12.411077 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:12.411739 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.411921 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.410444 10201 raft_consensus.cc:606] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:12.413029 10053 raft_consensus.cc:2410] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:18:12.414889 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45470 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:12.419677 10049 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:12.420430 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2907) took 1127 ms. Trace:
I20260709 13:18:12.422097 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:12.422915 10049 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:12.421690 10204 rpcz_store.cc:276] 0709 13:18:11.293524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.299965 (+  6441us) service_pool.cc:224] Handling call
0709 13:18:11.309378 (+  9413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.426679 (+117301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.426712 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.426765 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:11.426812 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.420040 (+993228us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002de1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:12.420097 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.420320 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244807040}
I20260709 13:18:12.426394 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.426806 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:12.427620 10053 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 6: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:12.443184 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.445456 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:12.450670 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1899) took 1846 ms. Trace:
I20260709 13:18:12.450938 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.452775 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.448698 13826 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:12.470798 10053 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:12.464742 10204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:12.471385 10204 raft_consensus.cc:606] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:12.471776 10204 raft_consensus.cc:614] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:12.465644 14137 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.466354 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1897) took 1913 ms. Trace:
I20260709 13:18:12.468956 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:12.468214 10051 rpcz_store.cc:276] 0709 13:18:10.609164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.296412 (+687248us) service_pool.cc:224] Handling call
0709 13:18:11.296740 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.296973 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:12.417505 (+1120532us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.450130 (+ 32625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.450215 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:12.450284 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.450383 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.450575 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1120158,"prepare.queue_time_us":1154944,"raft.queue_time_us":128,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":374,"spinlock_wait_cycles":5637376,"thread_start_us":32116,"threads_started":1}
I20260709 13:18:12.472515 10055 rpcz_store.cc:276] 0709 13:18:10.552711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.553078 (+   367us) service_pool.cc:224] Handling call
0709 13:18:10.592401 (+ 39323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.592745 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:11.011442 (+418697us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.417077 (+1405635us) spinlock_profiling.cc:243] Waited 978 us on lock 0x7b18000bbd20. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:18:12.417341 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.465272 (+ 47931us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b48000030d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:18:12.465385 (+   113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.465451 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:12.465503 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.466086 (+   583us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.466276 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1248947,"prepare.queue_time_us":1238010,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":48238,"raft.run_cpu_time_us":4252,"raft.run_wall_time_us":1206204,"spinlock_wait_cycles":31063296,"thread_start_us":246678,"threads_started":2,"wal-append.queue_time_us":391665,"wal-append.run_cpu_time_us":1452,"wal-append.run_wall_time_us":1002907}
I20260709 13:18:12.477097 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.479646 10894 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:12.486123 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.486907 10125 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:12.486176 14071 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:12.488034 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.488677 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:12.489169 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2930) took 1328 ms. Trace:
I20260709 13:18:12.489503 10125 rpcz_store.cc:276] 0709 13:18:11.160754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.487741 (+1326987us) service_pool.cc:224] Handling call
0709 13:18:12.489109 (+  1368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.487713 14071 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:12.492041 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:12.496768 10169 connection.cc:256] RPC call timeout handler was delayed by -1.4352445609997631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 13:18:12.497295 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:12.494364 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:12.496884 10204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:12.499100 10204 raft_consensus.cc:606] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:12.499495 10204 raft_consensus.cc:614] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:12.497469 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.503647 10053 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:18:12.504848 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.506029 10125 raft_consensus.cc:2410] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:12.506632 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.508529 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.509433 10053 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:18:12.510680 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.512473 10053 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:12.514359 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.513696 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.516512 10053 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:12.522018 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:12.524331 10049 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:12.525610 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:12.527057 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1901) took 1834 ms. Trace:
I20260709 13:18:12.526168 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:12.528530 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.528841 10894 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:12.502200 14091 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:12.531610 10049 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:12.533829 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:12.529935 13826 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:12.855959 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:12.534997 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1894) took 2058 ms. Trace:
I20260709 13:18:12.535300 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:13.281682 10125 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:12.582424 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:13.284070 10128 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:13.284752 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2959) took 1242 ms. Trace:
I20260709 13:18:13.285085 10128 rpcz_store.cc:276] 0709 13:18:12.041965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.581911 (+539946us) service_pool.cc:224] Handling call
0709 13:18:13.284657 (+702746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:13.286376 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.595206 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.605731 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1760) took 4023 ms. Trace:
I20260709 13:18:12.679016 10095 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
I20260709 13:18:12.613217  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:12.676486 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1903) took 1744 ms. Trace:
I20260709 13:18:13.863947 10047 rpcz_store.cc:276] 0709 13:18:10.932003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.021757 (+1089754us) service_pool.cc:224] Handling call
0709 13:18:12.037215 (+ 15458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.037476 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:12.526885 (+489409us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.676065 (+149180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.676143 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:12.676193 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.676275 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.676412 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":431218,"prepare.queue_time_us":449120,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":1806,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":93,"thread_start_us":187815,"threads_started":2,"wal-append.queue_time_us":75884,"wal-append.run_cpu_time_us":8907,"wal-append.run_wall_time_us":1250010}
I20260709 13:18:12.970602 10050 rpcz_store.cc:276] 0709 13:18:10.477198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.478063 (+   865us) service_pool.cc:224] Handling call
0709 13:18:10.478547 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.502041 (+ 23494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.502092 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:10.502162 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:10.502216 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.534670 (+2032454us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b54000da054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:12.534746 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.534916 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53255936}
I20260709 13:18:12.659091 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.084681 10051 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
W20260709 13:18:14.085479 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1842) took 1682 ms (client timeout 2227 ms). Trace:
W20260709 13:18:14.085985 10051 rpcz_store.cc:269] 0709 13:18:12.402802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.658770 (+255968us) service_pool.cc:224] Handling call
0709 13:18:14.085359 (+1426589us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:12.676667 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.090691 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.782011 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1902) took 2018 ms. Trace:
I20260709 13:18:12.710831 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.093801 10055 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:14.092747 10054 rpcz_store.cc:276] 0709 13:18:10.763451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.004268 (+1240817us) service_pool.cc:224] Handling call
0709 13:18:12.015354 (+ 11086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.015578 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:12.475855 (+460277us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.781468 (+305613us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.781564 (+    96us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:12.781626 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.781720 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.781915 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447604,"prepare.queue_time_us":470805,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":4232,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":120,"thread_start_us":305107,"threads_started":1,"wal-append.queue_time_us":117024,"wal-append.run_cpu_time_us":42024,"wal-append.run_wall_time_us":1439800}
W20260709 13:18:14.094529 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1916) took 1798 ms (client timeout 2034 ms). Trace:
W20260709 13:18:14.095072 10055 rpcz_store.cc:269] 0709 13:18:12.296162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.503068 (+206906us) service_pool.cc:224] Handling call
0709 13:18:14.094446 (+1591378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.095571 13961 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.633477 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.129989 10056 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:18:14.130614 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1852) took 1720 ms (client timeout 2218 ms). Trace:
W20260709 13:18:14.131032 10056 rpcz_store.cc:269] 0709 13:18:12.410690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.633083 (+222393us) service_pool.cc:224] Handling call
0709 13:18:14.130523 (+1497440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.132215 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.133107 10056 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:14.133564 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1840) took 1732 ms. Trace:
I20260709 13:18:14.133891 10056 rpcz_store.cc:276] 0709 13:18:12.401674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.131919 (+1730245us) service_pool.cc:224] Handling call
0709 13:18:14.133504 (+  1585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:13.182911 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 396) took 2571 ms. Trace:
I20260709 13:18:14.135735 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1908) took 1842 ms. Trace:
I20260709 13:18:14.134604 10048 rpcz_store.cc:276] 0709 13:18:10.611478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.688022 (+1076544us) service_pool.cc:224] Handling call
0709 13:18:11.688403 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.730960 (+ 42557us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:12.449341 (+718381us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.788036 (+338695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.788115 (+    79us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:12.788176 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.182548 (+394372us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b54001b7e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:13.182614 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.182822 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":712906,"prepare.queue_time_us":727166,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":748,"raft.queue_time_us":895,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":97305856,"thread_start_us":353596,"threads_started":2,"wal-append.queue_time_us":143634,"wal-append.run_cpu_time_us":43299,"wal-append.run_wall_time_us":1158042}
I20260709 13:18:14.136260 10056 rpcz_store.cc:276] 0709 13:18:12.293569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.134632 (+1841063us) service_pool.cc:224] Handling call
0709 13:18:14.134963 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.135345 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.135373 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.135423 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:14.135467 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.135527 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.135665 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:13.084388 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1762) took 4478 ms. Trace:
I20260709 13:18:14.155238 10129 rpcz_store.cc:276] 0709 13:18:08.605561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.894822 (+289261us) service_pool.cc:224] Handling call
0709 13:18:08.913929 (+ 19107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.823765 (+909836us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:10.260408 (+436643us) log.cc:844] Serialized 14 byte log entry
0709 13:18:13.037975 (+2777567us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b180044faa0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:18:13.038218 (+   243us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:13.038838 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.054048 (+ 15210us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:13.083585 (+ 29537us) log.cc:844] Serialized 14 byte log entry
0709 13:18:13.083868 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.083961 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.084110 (+   149us) raft_consensus.cc:1685] finished
0709 13:18:13.084126 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.084294 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1381645,"prepare.queue_time_us":693690,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":134,"raft.queue_time_us":916992,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":24883,"spinlock_wait_cycles":14559616,"thread_start_us":685062,"threads_started":4,"wal-append.queue_time_us":1128856,"wal-append.run_cpu_time_us":17799,"wal-append.run_wall_time_us":2022435}
I20260709 13:18:14.158236 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.693801 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.161707 10123 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:18:14.162376 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2958) took 2149 ms (client timeout 1810 ms). Trace:
I20260709 13:18:12.824070 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2952) took 1015 ms. Trace:
I20260709 13:18:14.164040 10131 rpcz_store.cc:276] 0709 13:18:11.808893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.040293 (+231400us) service_pool.cc:224] Handling call
0709 13:18:12.823745 (+783452us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b540015ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:12.823975 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52213120}
I20260709 13:18:14.165364 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.166025 10131 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:18:14.166563 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2967) took 1503 ms (client timeout 1658 ms). Trace:
I20260709 13:18:12.880539 10094 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2968)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
I20260709 13:18:13.108667 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:12.772724 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.173388 10124 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:14.162792 10123 rpcz_store.cc:269] 0709 13:18:12.012889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.693367 (+680478us) service_pool.cc:224] Handling call
0709 13:18:14.162274 (+1468907us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.173972 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 363) took 1880 ms (client timeout 1533 ms). Trace:
W20260709 13:18:13.236269 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2917) took 2563 ms (client timeout 1873 ms). Trace:
W20260709 13:18:14.175830 10126 rpcz_store.cc:269] 0709 13:18:10.672935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.133068 (+460133us) service_pool.cc:224] Handling call
0709 13:18:13.235972 (+2102904us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54001c6e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:13.236140 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422502784}
I20260709 13:18:14.177165 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:14.178049 10126 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:14.178491 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2964) took 1992 ms (client timeout 2145 ms). Trace:
I20260709 13:18:12.575227 10169 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:12.548527 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.813040 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 433) took 1469 ms. Trace:
I20260709 13:18:12.572021 10864 leader_election.cc:400] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11.
I20260709 13:18:14.186450 10864 leader_election.cc:403] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.185283 10199 rpcz_store.cc:276] 0709 13:18:11.343344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.343490 (+   146us) service_pool.cc:224] Handling call
0709 13:18:11.343922 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.125470 (+781548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.125504 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.129812 (+  4308us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.288501 (+158689us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:12.288545 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.812468 (+523923us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400354654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:12.812536 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.812605 (+    69us) raft_consensus.cc:1685] finished
0709 13:18:12.812622 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.812950 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23174,"raft.queue_time_us":20218,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3825,"spinlock_wait_cycles":1750405376,"thread_start_us":143458,"threads_started":1,"wal-append.queue_time_us":191503,"wal-append.run_cpu_time_us":15170,"wal-append.run_wall_time_us":1720464}
W20260709 13:18:12.576452 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.190178 10865 connection.cc:256] RPC call timeout handler was delayed by -1.1942987509999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 13:18:12.648000 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.210753 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.189680  9949 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 9, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 9 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:12.856065 10891 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:14.213313 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:14.213795 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 644) took 1680 ms (client timeout 1786 ms). Trace:
I20260709 13:18:14.214130 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:12.554913 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:14.216732 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 645) took 1662 ms (client timeout 2034 ms). Trace:
W20260709 13:18:14.217366 10894 rpcz_store.cc:269] 0709 13:18:12.554187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.554444 (+   257us) service_pool.cc:224] Handling call
0709 13:18:14.216630 (+1662186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:12.584872 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 458) took 1167 ms. Trace:
I20260709 13:18:12.573732 10898 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:14.235042 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 647) took 1665 ms. Trace:
I20260709 13:18:12.573124 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:12.836282 13574 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:14.239413 13574 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:13.227511 13691 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:14.243330 13691 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.246404 13691 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.840368 13776 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:12.856612 13826 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:12.649943 13855 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:14.260630 13855 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:14.261747 13855 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.837181 13875 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:14.263984 13875 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:14.265291 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:12.542274 14021 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:14.266749 10203 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:12.517460 14088 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:14.268563 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:12.708556 14095 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.479377 14137 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:12.723541 14138 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:12.652660 14151 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:12.629447 14159 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.851s	user 0.002s	sys 0.000s
I20260709 13:18:13.098112 14070 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:13.249109 14131 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:14.285282 14131 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:14.285969 10863 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.182s (SENT)
I20260709 13:18:13.052443 14072 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.289219 14072 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:12.789440 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.880677 14179 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.335s	user 0.002s	sys 0.001s
I20260709 13:18:13.138340 14170 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:14.294870 14072 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:14.295318 14170 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.296916 14131 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:14.297935 14170 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:13.102174 13823 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:13.287089 10128 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:14.299535 14137 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:14.299896 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2963) took 2165 ms (client timeout 2083 ms). Trace:
W20260709 13:18:14.300443 10128 rpcz_store.cc:269] 0709 13:18:12.134580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.285970 (+1151390us) service_pool.cc:224] Handling call
0709 13:18:14.299784 (+1013814us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.300832 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080016c300 after lost signal to thread 8928
I20260709 13:18:14.301749 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.302228 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000a58a0 after lost signal to thread 9988
I20260709 13:18:14.302408 10128 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:13.798339 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:14.303452 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000d9d80 after lost signal to thread 10049
W20260709 13:18:14.303599 10053 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1910) took 2009 ms (client timeout 2240 ms). Trace:
W20260709 13:18:14.303996 10053 rpcz_store.cc:269] 0709 13:18:12.294177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.519214 (+225037us) service_pool.cc:224] Handling call
0709 13:18:14.303527 (+1784313us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.304369 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080010bba0 after lost signal to thread 10080
I20260709 13:18:14.305073 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:14.305898 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08003ab0a0 after lost signal to thread 10155
W20260709 13:18:14.306202 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800308f80 after lost signal to thread 10156
W20260709 13:18:14.306473 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08003025e0 after lost signal to thread 10169
I20260709 13:18:13.807446 14195 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:14.306730 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800339580 after lost signal to thread 10171
I20260709 13:18:14.307456 14195 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.310019 14195 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:13.859961 10095 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items [suppressed 3 similar messages]
I20260709 13:18:14.313185 14209 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.310636 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1799)
I20260709 13:18:13.868563 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.314081 14209 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:14.316643 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002ab720 after lost signal to thread 10199
I20260709 13:18:12.856961 10049 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:14.317304 14209 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:14.096220 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.102366 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:14.102909 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.106050 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.108971 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:14.112341 10152 debug-util.cc:398] Leaking SignalData structure 0x7b0800358f60 after lost signal to thread 8928
I20260709 13:18:13.287848 10127 rpcz_store.cc:276] 0709 13:18:08.582724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.840883 (+258159us) service_pool.cc:224] Handling call
0709 13:18:08.855920 (+ 15037us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.763501 (+907581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.763539 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.764171 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.118046 (+353875us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:10.624446 (+506400us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.624897 (+   451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.812878 (+1187981us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:12.603927 (+791049us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b18002e0900. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419
0709 13:18:12.605297 (+  1370us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:12.605356 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.605410 (+    54us) raft_consensus.cc:1685] finished
0709 13:18:12.605423 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.605631 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1257462,"prepare.queue_time_us":410852,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":3451,"raft.queue_time_us":900188,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":65677,"spinlock_wait_cycles":171340288,"thread_start_us":708630,"threads_started":2,"wal-append.queue_time_us":620337,"wal-append.run_cpu_time_us":6262,"wal-append.run_wall_time_us":2427144}
I20260709 13:18:14.123274 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:14.144372 10168 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.915s (SENT) [suppressed 42 similar messages]
I20260709 13:18:12.529863 10052 rpcz_store.cc:276] 0709 13:18:10.693231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.803480 (+1110249us) service_pool.cc:224] Handling call
0709 13:18:11.805382 (+  1902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.805644 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:12.462164 (+656520us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.523517 (+ 61353us) log.cc:844] Serialized 14 byte log entry
0709 13:18:12.524859 (+  1342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.524918 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:12.524978 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.525057 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.526980 (+  1923us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":644644,"prepare.queue_time_us":662975,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":2017,"raft.queue_time_us":222,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":18296,"spinlock_wait_cycles":26532992,"thread_start_us":52284,"threads_started":2,"wal-append.queue_time_us":72610,"wal-append.run_cpu_time_us":15611,"wal-append.run_wall_time_us":1238464}
W20260709 13:18:14.153959 10125 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.872s	user 0.018s	sys 0.023s
I20260709 13:18:14.174767 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.174507 10124 rpcz_store.cc:269] 0709 13:18:12.293203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.772300 (+479097us) service_pool.cc:224] Handling call
0709 13:18:14.173882 (+1401582us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.179152 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1911) took 1884 ms. Trace:
W20260709 13:18:14.317598 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002eaa80 after lost signal to thread 10229
W20260709 13:18:14.319790 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800380de0 after lost signal to thread 10230
W20260709 13:18:14.166945 10131 rpcz_store.cc:269] 0709 13:18:12.663371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.165044 (+1501673us) service_pool.cc:224] Handling call
0709 13:18:14.166492 (+  1448us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.320353 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800305600 after lost signal to thread 10864
W20260709 13:18:14.320734 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002d6ae0 after lost signal to thread 10865
W20260709 13:18:14.320982 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080016bd60 after lost signal to thread 10866
I20260709 13:18:14.321224 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.321686 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002e85a0 after lost signal to thread 10891
I20260709 13:18:14.186842 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1912) took 1892 ms. Trace:
W20260709 13:18:14.178821 10126 rpcz_store.cc:269] 0709 13:18:12.186219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.176857 (+1990638us) service_pool.cc:224] Handling call
0709 13:18:14.178432 (+  1575us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.190912 10865 connection.cc:256] RPC call timeout handler was delayed by -1.1897987509998984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 13:18:14.322216 10125 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
W20260709 13:18:14.322934 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 367) took 1971 ms (client timeout 1819 ms). Trace:
I20260709 13:18:14.323154 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.324383 10126 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:14.324987 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2973) took 1646 ms (client timeout 1913 ms). Trace:
I20260709 13:18:14.325157 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:14.193403 10129 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:14.173611 10094 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2972)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 370)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2969)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1800)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1799)
W20260709 13:18:14.322279 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 370) took 1926 ms (client timeout 2182 ms). Trace:
I20260709 13:18:14.191977  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:18:14.219882 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.64799e+06us
I20260709 13:18:14.209111 14199 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.326840 10054 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:14.227383 10895 rpcz_store.cc:276] 0709 13:18:11.417879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.418065 (+   186us) service_pool.cc:224] Handling call
0709 13:18:11.430113 (+ 12048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.737095 (+306982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.584474 (+847379us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001e7154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:12.584780 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647141120}
I20260709 13:18:14.229142 14202 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:14.225019 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.223240 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.226353 14201 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11.
I20260709 13:18:14.237358 10900 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:14.238287 10899 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:18:14.235500 10898 rpcz_store.cc:276] 0709 13:18:12.569863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.572639 (+  2776us) service_pool.cc:224] Handling call
0709 13:18:12.573230 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.234641 (+1661411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.234948 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68096}
I20260709 13:18:14.240554 13574 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.256924 13826 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:14.258378 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.264967 13875 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.266013 10199 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:12.575945 14071 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:14.268636 14088 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:14.512079 14088 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:12.536890 14091 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.516814 14091 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:14.272899 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.262957 13855 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:14.520521 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.522457 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.276497 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.524803 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.276897  9949 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.531255  9949 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.283710 14070 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:14.293903 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.546241 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 218664us
W20260709 13:18:14.305146  9954 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 196) took 49145 ms (client timeout 15000 ms). Trace:
W20260709 13:18:13.974385 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.096220 13961 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.321802 10050 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:14.551025 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1921) took 2150 ms. Trace:
I20260709 13:18:14.551653 10050 rpcz_store.cc:276] 0709 13:18:12.400428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.101954 (+1701526us) service_pool.cc:224] Handling call
0709 13:18:14.550920 (+448966us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.326143 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 203010us
W20260709 13:18:14.214491 10891 rpcz_store.cc:269] 0709 13:18:12.533339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.533495 (+   156us) service_pool.cc:224] Handling call
0709 13:18:14.213687 (+1680192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.220948 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.554026 10894 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:14.226267 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 1.65098e+06us
I20260709 13:18:14.554785 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:14.328042 10123 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:18:14.559176 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2968) took 1895 ms (client timeout 1658 ms). Trace:
W20260709 13:18:14.559836 10123 rpcz_store.cc:269] 0709 13:18:12.663708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.174495 (+1510787us) service_pool.cc:224] Handling call
0709 13:18:14.559041 (+384546us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.561316 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.562337 10123 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:14.326675 10131 rpcz_store.cc:269] 0709 13:18:12.396344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.320876 (+1924532us) service_pool.cc:224] Handling call
0709 13:18:14.322201 (+  1325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.319547 10048 rpcz_store.cc:276] 0709 13:18:12.294494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.137206 (+1842712us) service_pool.cc:224] Handling call
0709 13:18:14.137667 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.146098 (+  8431us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:14.146665 (+   567us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.168302 (+ 21637us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.169022 (+   720us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.169258 (+   236us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.169303 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.169380 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.178881 (+  9501us) raft_consensus.cc:1685] finished
0709 13:18:14.178904 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.179048 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33204,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":5867,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":261,"spinlock_wait_cycles":62336,"thread_start_us":19579,"threads_started":1,"wal-append.queue_time_us":21142}
W20260709 13:18:14.325553 10126 rpcz_store.cc:269] 0709 13:18:12.678309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.322743 (+1644434us) service_pool.cc:224] Handling call
0709 13:18:14.324919 (+  2176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.326272 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 362) took 2033 ms (client timeout 2020 ms). Trace:
W20260709 13:18:14.330689 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1ee0 after lost signal to thread 9988
I20260709 13:18:14.562764 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:14.564617 10129 rpcz_store.cc:269] 0709 13:18:12.292884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.157918 (+1865034us) service_pool.cc:224] Handling call
0709 13:18:14.326155 (+168237us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.565933 10152 debug-util.cc:398] Leaking SignalData structure 0x7b0800349ca0 after lost signal to thread 10080
W20260709 13:18:14.567408 10152 debug-util.cc:398] Leaking SignalData structure 0x7b080034e820 after lost signal to thread 10155
W20260709 13:18:14.317791 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1848) took 1910 ms (client timeout 2320 ms). Trace:
I20260709 13:18:14.325378 10055 raft_consensus.cc:2368] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
I20260709 13:18:14.315166 10047 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:18:14.568693 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 1909) took 2274 ms (client timeout 2850 ms). Trace:
W20260709 13:18:14.569175 10047 rpcz_store.cc:269] 0709 13:18:12.293879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.868128 (+1574249us) service_pool.cc:224] Handling call
0709 13:18:14.568620 (+700492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.320139 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.323397 10125 rpcz_store.cc:269] 0709 13:18:12.351620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.521229 (+169609us) service_pool.cc:224] Handling call
0709 13:18:14.322839 (+1801610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.336472 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:14.302899 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2972) took 1624 ms (client timeout 1686 ms). Trace:
I20260709 13:18:14.322257 10056 rpcz_store.cc:276] 0709 13:18:12.294815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.137564 (+1842749us) service_pool.cc:224] Handling call
0709 13:18:14.137888 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.156723 (+ 18835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.156763 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.157326 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.171046 (+ 13720us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.171096 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.171199 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.186518 (+ 15319us) raft_consensus.cc:1685] finished
0709 13:18:14.186547 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.186764 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17894,"raft.queue_time_us":18173,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"thread_start_us":13290,"threads_started":1,"wal-append.queue_time_us":13666}
W20260709 13:18:14.352537 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08001468c0 after lost signal to thread 10894
I20260709 13:18:14.327648 10051 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:18:14.353042 10019 connection.cc:256] RPC call timeout handler was delayed by -1.3487449469998865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
I20260709 13:18:14.355744 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1850) took 1947 ms. Trace:
I20260709 13:18:14.373961 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:14.372088 10204 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 13:18:14.374447 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.574424 10204 pending_rounds.cc:85] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 9
I20260709 13:18:14.574898 10204 pending_rounds.cc:107] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:18:14.570235 10127 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11. Re-sending same reply.
W20260709 13:18:14.423233 10900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 646) took 1854 ms (client timeout 1652 ms). Trace:
W20260709 13:18:14.576118 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 364) took 2282 ms (client timeout 2182 ms). Trace:
I20260709 13:18:14.358846 14209 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:14.426561 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:14.358882 14021 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:14.581600 14021 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.570091 14208 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:14.582844 14208 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.584676 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:14.433238 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.433358 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.434244 10094 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1808)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1802)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1807)
I20260709 13:18:14.354717 14170 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:14.623540 14170 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.623613 14070 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:14.454103 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 309746us
I20260709 13:18:14.625662 14208 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:14.626580 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.627336 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.466696 14210 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.467363 10892 raft_consensus.cc:2410] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:14.469580 14131 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:14.629026 14210 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.630056 14131 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.631784 14210 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:14.632658 14131 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:14.472235 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 100998us
I20260709 13:18:14.435209 13574 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:14.567899 14206 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:14.478153 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 153044us
I20260709 13:18:14.455107 10095 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:45986 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1806)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1805)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1802)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1803)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1807)
I20260709 13:18:14.478909 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:14.637185 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:14.567797 14103 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:14.472337 13875 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:14.492568 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1914) took 2197 ms. Trace:
I20260709 13:18:14.529165 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.646540 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.282347 10196 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:18:14.536813 13826 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.544028 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.575359 10204 raft_consensus.cc:2889] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.359484 14137 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.576072 10900 rpcz_store.cc:269] 0709 13:18:12.569409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.572654 (+  3245us) service_pool.cc:224] Handling call
0709 13:18:14.423084 (+1850430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88448}
W20260709 13:18:14.651894 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89260.4us
W20260709 13:18:14.572439 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0da0 after lost signal to thread 10895
W20260709 13:18:14.654650 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800346140 after lost signal to thread 10898
W20260709 13:18:14.571206 10128 rpcz_store.cc:269] 0709 13:18:12.678001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.301404 (+1623403us) service_pool.cc:224] Handling call
0709 13:18:14.302844 (+  1440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.572248 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:14.574690 10124 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:14.573498 10019 connection.cc:256] RPC call timeout handler was delayed by -1.3352449469998646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
I20260709 13:18:14.573655 10054 rpcz_store.cc:276] 0709 13:18:12.408766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.095016 (+1686250us) service_pool.cc:224] Handling call
0709 13:18:14.355635 (+260619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.576583 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.581192  9949 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.299538 13823 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.586253 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:14.563095 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2966) took 1900 ms (client timeout 2186 ms). Trace:
W20260709 13:18:14.567780 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08003cb460 after lost signal to thread 10156
W20260709 13:18:14.674432 10152 debug-util.cc:398] Leaking SignalData structure 0x7b080036d820 after lost signal to thread 10229
W20260709 13:18:14.674782 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08001696a0 after lost signal to thread 10230
I20260709 13:18:14.675061 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:14.660297 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.671307 10123 rpcz_store.cc:269] 0709 13:18:12.662942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.560865 (+1897923us) service_pool.cc:224] Handling call
0709 13:18:14.563003 (+  2138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.678325 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.678624 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.568145 10049 rpcz_store.cc:269] 0709 13:18:12.407008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.855570 (+448562us) service_pool.cc:224] Handling call
0709 13:18:14.317698 (+1462128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.680994 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1925) took 1080 ms. Trace:
W20260709 13:18:14.576723 10127 rpcz_store.cc:269] 0709 13:18:12.293520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.319750 (+2026230us) service_pool.cc:224] Handling call
0709 13:18:14.576026 (+256276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.563364 14202 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:14.587435 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.617420 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.585604 10205 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:14.616674 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1919) took 2286 ms. Trace:
I20260709 13:18:14.619555 14214 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:14.621695 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.682845 10129 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:14.687038 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:14.688074 10129 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:14.616180 14209 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.622643 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:14.646181 13776 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:14.635314 10020 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Timed out: RequestConsensusVote RPC to 127.8.183.67:40505 timed out after 1.562s (SENT)
W20260709 13:18:14.695418 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60172.3us
W20260709 13:18:14.637506 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 176226us
I20260709 13:18:14.698019 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:14.359802 14199 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.569936 14207 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.646229 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.653530 14210 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:14.643782 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1922) took 1044 ms. Trace:
I20260709 13:18:14.711580 10048 rpcz_store.cc:276] 0709 13:18:13.599981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.588297 (+988316us) service_pool.cc:224] Handling call
0709 13:18:14.643210 (+ 54913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.643411 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.643437 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.643481 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.643529 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.643584 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.643709 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.646090 10053 rpcz_store.cc:276] 0709 13:18:12.295510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.306193 (+2010683us) service_pool.cc:224] Handling call
0709 13:18:14.306543 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.306946 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.306976 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.343832 (+ 36856us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.480769 (+136937us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:14.480825 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.480921 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:14.492266 (+ 11345us) raft_consensus.cc:1685] finished
0709 13:18:14.492292 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.492484 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":204,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":155,"thread_start_us":61108,"threads_started":1,"wal-append.queue_time_us":131460}
I20260709 13:18:14.712386 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.572373 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1843) took 2169 ms. Trace:
I20260709 13:18:14.713789 10055 rpcz_store.cc:276] 0709 13:18:12.403178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.095896 (+1692718us) service_pool.cc:224] Handling call
0709 13:18:14.572307 (+476411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.655009 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:14.670269 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1924) took 1069 ms. Trace:
I20260709 13:18:14.671466 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:14.571066 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.713544 10127 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:14.716961 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62078.5us
I20260709 13:18:14.586319 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.719404 10125 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:14.683658 10050 rpcz_store.cc:276] 0709 13:18:12.330340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.575355 (+2245015us) service_pool.cc:224] Handling call
0709 13:18:14.616004 (+ 40649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.616275 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.616302 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.616347 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.616393 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.616457 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.616593 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.684329 10131 raft_consensus.cc:2410] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:18:14.721050 10054 rpcz_store.cc:276] 0709 13:18:13.600755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.669319 (+1068564us) service_pool.cc:224] Handling call
0709 13:18:14.669692 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.669887 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.669913 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.669957 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:14.669998 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.670053 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.670191 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.655239 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08001618e0 after lost signal to thread 10900
I20260709 13:18:14.723443 10126 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:18:14.685209 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2969) took 2021 ms (client timeout 2047 ms). Trace:
W20260709 13:18:14.724298 10124 rpcz_store.cc:269] 0709 13:18:12.664021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.327649 (+1663628us) service_pool.cc:224] Handling call
0709 13:18:14.685139 (+357490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.676318 10198 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
W20260709 13:18:14.723014 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4060 after lost signal to thread 10924
W20260709 13:18:14.726058 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080015a6c0 after lost signal to thread 10925
W20260709 13:18:14.726384 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800378a80 after lost signal to thread 13574
W20260709 13:18:14.726649 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800256c60 after lost signal to thread 13776
W20260709 13:18:14.726882 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0f00 after lost signal to thread 13826
W20260709 13:18:14.727097 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080035a880 after lost signal to thread 13855
W20260709 13:18:14.727357 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080012b0c0 after lost signal to thread 13875
W20260709 13:18:14.727602 10077 debug-util.cc:398] Leaking SignalData structure 0x7b080038b1c0 after lost signal to thread 14021
W20260709 13:18:14.689576 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08000a77a0 after lost signal to thread 10924
W20260709 13:18:14.728122 10152 debug-util.cc:398] Leaking SignalData structure 0x7b080024eac0 after lost signal to thread 10925
I20260709 13:18:14.681471 10049 rpcz_store.cc:276] 0709 13:18:13.601074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.680033 (+1078959us) service_pool.cc:224] Handling call
0709 13:18:14.680379 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.680594 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.680623 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.680670 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:14.680724 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.680781 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.680918 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.690515 10019 connection.cc:256] RPC call timeout handler was delayed by -1.2993969399999514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:18:14.731896 10019 connection.cc:256] RPC call timeout handler was delayed by -1.2785449470000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 13:18:14.732321 10019 connection.cc:256] RPC call timeout handler was delayed by -1.1867449469998519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 13:18:14.732613 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0765999689999717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:18:14.732895 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0641984950000278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
W20260709 13:18:14.733183 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0482969399997728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 13:18:14.733466 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0401511760001085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:18:14.733739 10019 connection.cc:256] RPC call timeout handler was delayed by -1.039069961999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:18:14.734015 10019 connection.cc:256] RPC call timeout handler was delayed by -1.03472111699989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
I20260709 13:18:14.691818 10199 raft_consensus.cc:2410] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:18:14.624148 14170 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:14.692060 10130 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:14.737262 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2971) took 2059 ms (client timeout 2523 ms). Trace:
W20260709 13:18:14.737907 10131 rpcz_store.cc:269] 0709 13:18:12.677668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.576148 (+1898480us) service_pool.cc:224] Handling call
0709 13:18:14.737154 (+161006us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:14.692986 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120806us
I20260709 13:18:14.652004 14072 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:14.641062 14219 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:18:14.700016 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.642511 10056 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:14.670104 14137 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:14.716015 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1926) took 1114 ms. Trace:
W20260709 13:18:14.743104 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.652323 14103 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.739006 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1920) took 2400 ms. Trace:
W20260709 13:18:14.547636  9954 rpcz_store.cc:269] 0709 13:17:25.159623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:25.167366 (+  7743us) service_pool.cc:224] Handling call
0709 13:18:14.304987 (+49137621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:09.110081 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:09.110781 (+   700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:09.110850 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:18:09.110869 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:09.111859 (+   990us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:09.111954 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:09.111986 (+    32us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:18:09.112795 (+   809us) tablet.cc:701] Row locks acquired
0709 13:18:09.112850 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:18:09.113098 (+   248us) write_op.cc:270] Start()
0709 13:18:09.113284 (+   186us) write_op.cc:276] Timestamp: P: 1783603089113015 usec, L: 0
0709 13:18:09.113315 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:18:09.114232 (+   917us) log.cc:844] Serialized 2906 byte log entry
0709 13:18:09.661055 (+546823us) op_driver.cc:464] REPLICATION: finished
0709 13:18:10.337485 (+676430us) write_op.cc:301] APPLY: starting
0709 13:18:10.353495 (+ 16010us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:11.880304 (+1526809us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:11.880538 (+   234us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:12.245702 (+365164us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:12.246101 (+   399us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:18:12.246161 (+    60us) write_op.cc:312] APPLY: finished
0709 13:18:12.246620 (+   459us) log.cc:844] Serialized 73 byte log entry
0709 13:18:12.330604 (+ 83984us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:12.335863 (+  5259us) write_op.cc:454] Released schema lock
0709 13:18:12.343969 (+  8106us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:12.414740 (+ 70771us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:12.420735 (+  5995us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44088,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":622615,"apply.run_cpu_time_us":57815,"apply.run_wall_time_us":2074365,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27118,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6213,"cfile_init":2,"lbm_read_time_us":255999,"lbm_reads_10-100_ms":2,"lbm_reads_gt_100_ms":1,"lbm_reads_lt_1ms":9,"num_ops":7,"prepare.queue_time_us":293366,"prepare.run_cpu_time_us":35839,"prepare.run_wall_time_us":185137,"raft.queue_time_us":61373,"raft.run_cpu_time_us":37527,"raft.run_wall_time_us":482449,"replication_time_us":511245,"thread_start_us":631746,"threads_started":5,"wal-append.queue_time_us":424516,"wal-append.run_cpu_time_us":18835,"wal-append.run_wall_time_us":2622216}]]}
I20260709 13:18:14.693990 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.744163 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08003c33e0 after lost signal to thread 14071
W20260709 13:18:14.797745 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08003467e0 after lost signal to thread 14088
I20260709 13:18:14.786042 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:14.800423 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800061d80 after lost signal to thread 14091
W20260709 13:18:14.801214 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1c00 after lost signal to thread 14095
W20260709 13:18:14.801594 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800334b40 after lost signal to thread 14137
I20260709 13:18:14.786993 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:14.801935 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000fd6c0 after lost signal to thread 14138
W20260709 13:18:14.802268 10077 debug-util.cc:398] Leaking SignalData structure 0x7b0800204a80 after lost signal to thread 14151
W20260709 13:18:14.802657 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5800 after lost signal to thread 14159
W20260709 13:18:14.705873 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.805835 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.789624 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1849) took 2381 ms. Trace:
I20260709 13:18:14.807801 10056 rpcz_store.cc:276] 0709 13:18:12.408194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.578764 (+2170570us) service_pool.cc:224] Handling call
0709 13:18:14.637181 (+ 58417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.789172 (+151991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.789521 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.711417 14210 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:14.715962 14214 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:14.749701 14021 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.636629 14206 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.816780 14206 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:14.817534 14206 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:14.816233  9949 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 3 to 5, term changed from 6 to 11, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to <none>, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:14.817979 14206 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.822049 14206 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:14.793213 10047 rpcz_store.cc:276] 0709 13:18:12.338136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.576461 (+2238325us) service_pool.cc:224] Handling call
0709 13:18:14.620031 (+ 43570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.620307 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:14.651297 (+ 30990us) log.cc:844] Serialized 14 byte log entry
0709 13:18:14.738520 (+ 87223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.738596 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:14.738661 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.738748 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.738905 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30776,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":121,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":41021,"thread_start_us":86645,"threads_started":1,"wal-append.queue_time_us":98224}
I20260709 13:18:14.736869 13855 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:14.823804 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.801482 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.824671 10056 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:14.825552 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.825754 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.826407 10048 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:14.826728 10056 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:18:14.803690 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:14.823889 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.789594 10128 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:14.828510 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.828655 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:14.829429 10128 raft_consensus.cc:2376] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6. Re-sending same reply.
I20260709 13:18:14.830753 10128 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:14.831645 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.832683 10048 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:18:14.809072 14088 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:14.811290 10077 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3100 after lost signal to thread 14179
I20260709 13:18:14.837898 10053 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:14.838706 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1928) took 1236 ms. Trace:
I20260709 13:18:14.713210 10892 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 8
I20260709 13:18:14.838864 14088 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:14.797806 10019 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.511s (SENT)
I20260709 13:18:14.790053 14095 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.818885  9949 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:14.821098 13823 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:14.810537 14208 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:14.794552 10131 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:18:14.797220 10124 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:18:14.810168 13961 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:14.829546 10047 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:14.832800 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.834136 14072 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.835949 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:14.892452 10054 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:14.835248 14021 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:14.800729 10127 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:14.792977 10055 rpcz_store.cc:276] 0709 13:18:13.601462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.714723 (+1113261us) service_pool.cc:224] Handling call
0709 13:18:14.715213 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.715500 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.715523 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.715574 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:14.715638 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.715732 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.715912 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.839095 10053 rpcz_store.cc:276] 0709 13:18:13.602212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.837346 (+1235134us) service_pool.cc:224] Handling call
0709 13:18:14.837700 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.838426 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.838641 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.839365 10897 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:14.840891 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1772) took 6192 ms. Trace:
I20260709 13:18:14.841895 10048 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:14.847649 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:14.863504 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:14.866135 10126 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2970) took 2201 ms (client timeout 2453 ms). Trace:
W20260709 13:18:14.869076 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2953) took 3054 ms (client timeout 1939 ms). Trace:
W20260709 13:18:14.902684 10126 rpcz_store.cc:269] 0709 13:18:12.677239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.564328 (+1887089us) service_pool.cc:224] Handling call
0709 13:18:14.865916 (+301588us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b540023b4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:14.866061 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88663936}
W20260709 13:18:14.902906 10122 rpcz_store.cc:269] 0709 13:18:11.822219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.108279 (+1286060us) service_pool.cc:224] Handling call
0709 13:18:14.868716 (+1760437us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54002eccd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:14.868947 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310754688}
I20260709 13:18:14.873261 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1931) took 1107 ms. Trace:
W20260709 13:18:14.875593 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 238472us
W20260709 13:18:14.876215 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 178285us
I20260709 13:18:14.878885 10131 pending_rounds.cc:85] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:18:14.903956 10049 rpcz_store.cc:276] 0709 13:18:13.765968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.871846 (+1105878us) service_pool.cc:224] Handling call
0709 13:18:14.872378 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.872685 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.872729 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.872795 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.872872 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.872959 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.873153 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.904987 10131 pending_rounds.cc:107] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:18:14.905377 10131 raft_consensus.cc:2889] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.879038 10124 pending_rounds.cc:85] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
I20260709 13:18:14.905974 10124 pending_rounds.cc:107] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:18:14.906306 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1936) took 1138 ms. Trace:
I20260709 13:18:14.839337 10892 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 12.9
W20260709 13:18:14.897773 14220 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:18:14.897414 14231 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:14.814325 14210 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:14.901960 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104193us
W20260709 13:18:14.905885 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 2946) took 3212 ms (client timeout 2224 ms). Trace:
I20260709 13:18:14.906502 10124 raft_consensus.cc:2889] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.893182 10056 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:14.886895 14208 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.900866 10128 rpcz_store.cc:276] 0709 13:18:08.648670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.830210 (+6181540us) service_pool.cc:224] Handling call
0709 13:18:14.830564 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.840568 (+ 10004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.840776 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.906896 10049 rpcz_store.cc:276] 0709 13:18:13.767583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.905389 (+1137806us) service_pool.cc:224] Handling call
0709 13:18:14.905746 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.905974 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.905997 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.906044 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.906094 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.906145 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.906249 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.910118 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1934) took 1143 ms. Trace:
I20260709 13:18:14.909052 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1929) took 1306 ms. Trace:
I20260709 13:18:14.909041 10050 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:14.901078 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1933) took 1134 ms. Trace:
I20260709 13:18:14.911785 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:14.898748 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:14.714893 14207 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:14.967561 10130 rpcz_store.cc:269] 0709 13:18:11.693672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.594703 (+901031us) service_pool.cc:224] Handling call
0709 13:18:14.905626 (+2310923us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002ed1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:14.905795 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3228962560}
W20260709 13:18:14.964939 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60538.3us
W20260709 13:18:14.922309 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.916128 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1771) took 6267 ms. Trace:
W20260709 13:18:14.965426 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66766.3us
I20260709 13:18:14.967079 10054 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:18:14.971081 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59429.1us
I20260709 13:18:14.973814 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 342) took 6298 ms. Trace:
W20260709 13:18:14.969064 10019 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.620s (SENT)
I20260709 13:18:14.969417 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1930) took 1366 ms. Trace:
I20260709 13:18:14.966562 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.973824 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1935) took 1206 ms. Trace:
I20260709 13:18:14.909543 10892 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.968542 10055 rpcz_store.cc:276] 0709 13:18:13.766666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.900147 (+1133481us) service_pool.cc:224] Handling call
0709 13:18:14.900507 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.900706 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.900749 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.900794 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:14.900839 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.900890 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.901016 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.914436 13961 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:14.975654 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:14.983796 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.975219 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:15.034765 10128 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:15.035107 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.006062 10129 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:14.878285 14223 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:14.984861 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1770) took 6336 ms. Trace:
I20260709 13:18:15.039778 10124 rpcz_store.cc:276] 0709 13:18:08.648084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.748822 (+6100738us) service_pool.cc:224] Handling call
0709 13:18:14.791740 (+ 42918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.984426 (+192686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.984765 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.000335 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.982995 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.041771 10047 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:18:15.042502 10047 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:18:15.043272 10122 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:15.027649 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.973816 10131 rpcz_store.cc:276] 0709 13:18:08.648380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.745787 (+6097407us) service_pool.cc:224] Handling call
0709 13:18:14.790850 (+ 45063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.915791 (+124941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.916053 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.900470 14072 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:15.034876 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60228.9us
I20260709 13:18:15.045038 10096 leader_election.cc:400] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
I20260709 13:18:15.045430 10096 leader_election.cc:403] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.970103 10053 rpcz_store.cc:276] 0709 13:18:13.766976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.900320 (+1133344us) service_pool.cc:224] Handling call
0709 13:18:14.909531 (+  9211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.909755 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.909783 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.909827 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:14.909877 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.909938 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.910047 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.036782 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:14.976025 10123 raft_consensus.cc:2376] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11. Re-sending same reply.
I20260709 13:18:14.886536 14214 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:14.908806 14224 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:18:15.043370 10047 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:18:15.058851 10047 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.974737 10127 rpcz_store.cc:276] 0709 13:18:08.675276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.899929 (+6224653us) service_pool.cc:224] Handling call
0709 13:18:14.973138 (+ 73209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.973451 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.973479 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.973523 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:14.973582 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.973640 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:14.973742 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.014417 14227 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:15.021896 10050 rpcz_store.cc:276] 0709 13:18:13.602960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.868072 (+1265112us) service_pool.cc:224] Handling call
0709 13:18:14.908785 (+ 40713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.968566 (+ 59781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.969332 (+   766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.023582 10048 rpcz_store.cc:276] 0709 13:18:13.602573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.841325 (+1238752us) service_pool.cc:224] Handling call
0709 13:18:14.841691 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.908698 (+ 67007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.908967 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.023173 10054 rpcz_store.cc:276] 0709 13:18:13.767295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.900915 (+1133620us) service_pool.cc:224] Handling call
0709 13:18:14.966680 (+ 65765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.973526 (+  6846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.973756 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.047231 10129 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
W20260709 13:18:15.023504 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 305251us
I20260709 13:18:15.080539 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.081552 10131 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:15.084463 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:14.980079 14216 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:18:15.022890 10126 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:15.024451 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:14.970315 14208 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.069176 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1923) took 1468 ms. Trace:
I20260709 13:18:15.078495 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.026485 13826 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.085903 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:14.877043 14095 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:15.102725 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 119905us
I20260709 13:18:15.104640 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.087301 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:15.088058 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.088472 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.048133 10130 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:15.091912 10051 rpcz_store.cc:276] 0709 13:18:13.600387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.630793 (+1030406us) service_pool.cc:224] Handling call
0709 13:18:14.631458 (+   665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.813126 (+181668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.813161 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:14.813405 (+   244us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:14.813462 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.068647 (+255185us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540010aed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:15.068730 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.068995 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417169664}
I20260709 13:18:15.092236 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:15.003018 14206 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:15.086316 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:15.071841 14225 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:18:15.099709 10900 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:14.974977 14228 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.123s
I20260709 13:18:15.091794 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.099706 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.120760 10127 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:15.122017 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.122715 10127 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:15.106523 10124 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:15.108417 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.115715 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.115869 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:15.091428 14197 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:15.079629 14227 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.115438  9949 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 6, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 6 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:15.179987 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 204408us
I20260709 13:18:15.132915 14021 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.133277 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1769) took 6525 ms. Trace:
W20260709 13:18:15.184939 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140079us
I20260709 13:18:15.185499 10125 rpcz_store.cc:276] 0709 13:18:08.608124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.745557 (+6137433us) service_pool.cc:224] Handling call
0709 13:18:14.792073 (+ 46516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.798515 (+  6442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.798550 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.801351 (+  2801us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.874741 (+ 73390us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:14.874793 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.874890 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.132986 (+258096us) raft_consensus.cc:1685] finished
0709 13:18:15.133020 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.133196 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":83890,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":143,"thread_start_us":55053,"threads_started":1,"wal-append.queue_time_us":55631}
I20260709 13:18:15.133136 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.186214 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.135022 10198 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:15.135747 10126 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:15.136329 10128 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:15.120760 10122 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:15.116680 13826 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:15.118489 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.213560 10130 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:15.141594 14095 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:15.147852 14170 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:15.149659 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.150627 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.090804 13691 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:18:15.157862 14238 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:15.158272 14204 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:15.183473 14227 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:15.092475 14209 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.159987 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:15.230271 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.163277 14091 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.000s
I20260709 13:18:15.168294 14199 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.171732 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.173771 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1917) took 2877 ms. Trace:
I20260709 13:18:15.183346 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.091239 14226 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:15.215796 14021 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:15.184398 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 325907us
I20260709 13:18:15.121022 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:15.190791 10096 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.791s (SENT)
I20260709 13:18:15.241882 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.134229 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.243129 10127 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:15.138739 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:15.204929 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.236527 10052 rpcz_store.cc:276] 0709 13:18:12.296450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.323883 (+2027433us) service_pool.cc:224] Handling call
0709 13:18:14.324338 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.324760 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.324793 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.403899 (+ 79106us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b2c000014c8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:14.404452 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:18:14.589232 (+184780us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:14.589303 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:14.589444 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.173337 (+583893us) raft_consensus.cc:1685] finished
0709 13:18:15.173369 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.173670 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":97,"spinlock_wait_cycles":43699072,"thread_start_us":153033,"threads_started":1,"wal-append.queue_time_us":313526}
I20260709 13:18:15.204253  9949 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 14 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:15.205653 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.205929 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.206806 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.207594 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:15.107856 13961 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:15.208402 13875 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.000s
I20260709 13:18:15.208395 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:15.174666 13776 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.212769 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:15.214854 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.216725 10129 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:15.253268 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 343) took 6565 ms. Trace:
I20260709 13:18:15.253677 10129 rpcz_store.cc:276] 0709 13:18:08.687665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.216092 (+6528427us) service_pool.cc:224] Handling call
0709 13:18:15.216477 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.252923 (+ 36446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.253187 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.170835 14207 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:15.158877 14236 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11.
I20260709 13:18:15.208312 14206 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.227212 10125 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:15.259779 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47110us
I20260709 13:18:15.260227 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 344) took 6572 ms. Trace:
I20260709 13:18:15.258451 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 347) took 6569 ms. Trace:
W20260709 13:18:15.231046 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.231616 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 257849us
I20260709 13:18:15.225100 14214 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:15.273597 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.274089 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83355.1us
W20260709 13:18:15.274857 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.830s (SENT)
I20260709 13:18:15.249570 13961 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:15.183979 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.244002 10124 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:15.241544 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:15.250168 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 281095us
I20260709 13:18:15.251171 10126 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:15.269220  9949 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.258621 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.284365 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.285687 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.282917  9949 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 1 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.288295 14170 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:15.288760  9949 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 7, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 7 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.289844 14236 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:15.267102 14199 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.260641 10125 rpcz_store.cc:276] 0709 13:18:08.688122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.217619 (+6529497us) service_pool.cc:224] Handling call
0709 13:18:15.226964 (+  9345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.259931 (+ 32967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.260165 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:15.297448 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56012.6us
I20260709 13:18:15.261117 10127 rpcz_store.cc:276] 0709 13:18:08.689139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.249283 (+6560144us) service_pool.cc:224] Handling call
0709 13:18:15.253874 (+  4591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.258069 (+  4195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.258100 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.258155 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:15.258207 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.258269 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.258380 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:15.241139 13574 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 13:18:15.267499 10122 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11. Re-sending same reply.
I20260709 13:18:15.267932 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:15.266032 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.256922 13776 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.240479 14226 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.235745 14237 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.262974 10129 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:15.233515 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 346) took 6544 ms. Trace:
I20260709 13:18:15.310798 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 348) took 6621 ms. Trace:
I20260709 13:18:15.311340 10129 rpcz_store.cc:276] 0709 13:18:08.689425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.254548 (+6565123us) service_pool.cc:224] Handling call
0709 13:18:15.254852 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.310454 (+ 55602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.310725 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.310662 10130 rpcz_store.cc:276] 0709 13:18:08.688793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.232438 (+6543645us) service_pool.cc:224] Handling call
0709 13:18:15.232838 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.233092 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.233119 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.233178 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:15.233222 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.233289 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.233439 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.312207 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 352) took 6621 ms. Trace:
I20260709 13:18:15.275350 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.303959 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.312772 10125 rpcz_store.cc:276] 0709 13:18:08.690560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.297087 (+6606527us) service_pool.cc:224] Handling call
0709 13:18:15.311462 (+ 14375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.311708 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.311748 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.311814 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:15.311898 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.311971 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.312123 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.307595 10865 leader_election.cc:400] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11.
I20260709 13:18:15.275902 14151 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:15.239380 14091 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:15.315676 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.265254 14239 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:15.317035 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49171.8us
I20260709 13:18:15.324508 10123 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:15.325310 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1776) took 6526 ms. Trace:
I20260709 13:18:15.325644 10123 rpcz_store.cc:276] 0709 13:18:08.799279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.323776 (+6524497us) service_pool.cc:224] Handling call
0709 13:18:15.324126 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.324982 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.325238 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.320693 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.327557 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.328150  9949 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.331647 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.291532 13961 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:15.332515 14240 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:18:15.304231 14170 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:15.334745 14170 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:15.335243 14170 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:15.341884 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.343031 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.345723 10125 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:15.346486 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1775) took 6558 ms. Trace:
I20260709 13:18:15.346794 10125 rpcz_store.cc:276] 0709 13:18:08.788328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.314024 (+6525696us) service_pool.cc:224] Handling call
0709 13:18:15.345517 (+ 31493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.346224 (+   707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.346426 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.348217 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.350090 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.304405 14021 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:15.328205 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.353721 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.357860 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.308041 14206 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.359900 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.357807 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.360659 10205 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:15.358815 10891 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:15.364923 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:15.366696 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91883.7us
W20260709 13:18:15.367529 10096 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.843s (SENT)
I20260709 13:18:15.367935 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.368489 10127 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:15.370512 10200 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:15.366271 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.310456 14226 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:15.321442 14151 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:15.375777 14236 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.330155  9949 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index -1 to 4, term changed from 4 to 8, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 8 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:18:15.409668 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:15.447402 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1777) took 6647 ms. Trace:
I20260709 13:18:15.447697 10056 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:15.382344 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:15.444360 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87849.4us
I20260709 13:18:15.447979 10123 rpcz_store.cc:276] 0709 13:18:08.799692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.358067 (+6558375us) service_pool.cc:224] Handling call
0709 13:18:15.405116 (+ 47049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.446854 (+ 41738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.446892 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.446951 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:15.447000 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.447138 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.447318 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590144}
I20260709 13:18:15.449209 10200 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
W20260709 13:18:15.449416 10019 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.643s (SENT)
I20260709 13:18:15.449956 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:15.451242 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:15.451444 10019 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.658s (SENT)
I20260709 13:18:15.452013 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:15.391043 14091 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:15.403869 14243 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:18:15.457692 13776 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:15.459306 14208 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:15.459992 14170 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:15.461262 10056 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:15.393747 14072 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:15.463608 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.445156 10126 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
W20260709 13:18:15.465142 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.005s (SENT)
W20260709 13:18:15.466145 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98322us
I20260709 13:18:15.464112 13826 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:15.464495 14209 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:15.468186  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:15.442653 14151 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:15.670337 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.679351 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:15.504364 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.558856 10900 raft_consensus.cc:2368] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:15.742084 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.606235 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1773) took 6849 ms. Trace:
W20260709 13:18:15.743498 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 278387us
W20260709 13:18:15.744522 10096 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.064s (SENT)
I20260709 13:18:15.508124 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:15.745276 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83091.8us
I20260709 13:18:15.665537 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.674274 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:15.681749 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.686601 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.682565 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 361) took 6928 ms. Trace:
I20260709 13:18:15.743101 10129 rpcz_store.cc:276] 0709 13:18:08.757286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.312220 (+6554934us) service_pool.cc:224] Handling call
0709 13:18:15.312627 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.337521 (+ 24894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.337557 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.338149 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.546652 (+208503us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:15.546728 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.546849 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.605852 (+ 59003us) raft_consensus.cc:1685] finished
0709 13:18:15.605885 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.606116 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71289,"raft.queue_time_us":30994,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4392,"thread_start_us":207932,"threads_started":1,"wal-append.queue_time_us":259356}
I20260709 13:18:15.675496 10124 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 11. Ops: []
I20260709 13:18:15.598944 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.666946 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 642) took 3164 ms. Trace:
W20260709 13:18:15.503752 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52353.4us
W20260709 13:18:15.700075 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49806.6us
I20260709 13:18:15.642874 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1932) took 1876 ms. Trace:
I20260709 13:18:15.668735 10128 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:15.672605 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:15.465778 14237 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.794586 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1779) took 6789 ms. Trace:
I20260709 13:18:15.796473 10897 rpcz_store.cc:276] 0709 13:18:12.502225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.502382 (+   157us) service_pool.cc:224] Handling call
0709 13:18:12.517282 (+ 14900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.900592 (+2383310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.666596 (+766004us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54004aef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:15.666835 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5339729152}
I20260709 13:18:15.794679 10127 rpcz_store.cc:276] 0709 13:18:08.753996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.303964 (+6549968us) service_pool.cc:224] Handling call
0709 13:18:15.307178 (+  3214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.369188 (+ 62010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.682280 (+313092us) spinlock_profiling.cc:243] Waited 60.8 ms on lock 0x7b540014c654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:15.682489 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139787648}
I20260709 13:18:15.793630 10047 rpcz_store.cc:276] 0709 13:18:13.766379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.892933 (+1126554us) service_pool.cc:224] Handling call
0709 13:18:14.964653 (+ 71720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.059688 (+ 95035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.631108 (+571420us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b5400136d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:15.642768 (+ 11660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165881216}
W20260709 13:18:15.508344 14138 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.024s	sys 0.020s
I20260709 13:18:15.816814 10199 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:15.817420 10197 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:15.817996 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.818315 10125 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:15.806391 10899 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:18:15.807318 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 299289us
I20260709 13:18:15.802790 10128 rpcz_store.cc:276] 0709 13:18:09.005611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.668157 (+6662546us) service_pool.cc:224] Handling call
0709 13:18:15.668526 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.794208 (+125682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.794507 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.815289 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:15.766184 10126 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:18:15.820611 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1780) took 5384 ms. Trace:
I20260709 13:18:15.820670 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.510380 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.822629 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:15.807816 10019 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.658s (SENT)
I20260709 13:18:15.808908 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1781) took 5372 ms. Trace:
W20260709 13:18:15.810979 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61730.9us
I20260709 13:18:15.813781 14206 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.827227 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:15.827559 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.826124 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:15.818464 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.823498 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.565670  9949 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:15.823868 10129 rpcz_store.cc:276] 0709 13:18:10.436245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.794140 (+5357895us) service_pool.cc:224] Handling call
0709 13:18:15.808318 (+ 14178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.808525 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.808549 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.808597 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:15.808651 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.808709 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.808843 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:15.680536 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 471) took 1042 ms. Trace:
I20260709 13:18:15.752278 13855 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.653522 14137 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:15.817945 14138 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.024s	sys 0.065s
I20260709 13:18:15.793623 14209 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:15.558661 14210 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.003s
W20260709 13:18:15.825794 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.847844 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:15.752445 14236 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:15.838083 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.848603 10895 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:15.850476 14200 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.019s	sys 0.029s
I20260709 13:18:15.845927 10200 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:15.752077 14235 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.751330 14199 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:15.852336 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.856068 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:15.857404 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1774) took 7099 ms. Trace:
I20260709 13:18:15.857087 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:15.861970 10047 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.821017 10126 rpcz_store.cc:276] 0709 13:18:10.435880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.765253 (+5329373us) service_pool.cc:224] Handling call
0709 13:18:15.765817 (+   564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.820278 (+ 54461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.820542 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:15.850943 14200 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.019s	sys 0.029s
I20260709 13:18:15.540966 14091 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.303s
I20260709 13:18:15.512809 14231 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.855914 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.846191 14244 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:15.906100 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1784) took 5388 ms. Trace:
I20260709 13:18:15.863749 14235 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:15.856151 13823 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:15.863862 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.866443 14170 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:15.868654 14072 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:15.868798 14208 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:15.589011 14219 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.258s
I20260709 13:18:15.847438 13826 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.871464 14262 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:15.828715 14214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.009s
W20260709 13:18:15.873119 14227 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:18:15.838224 13855 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:15.831598 10019 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.673s (SENT)
I20260709 13:18:15.947743 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:15.949031 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117480us
I20260709 13:18:15.859133 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:15.951393 10203 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:15.674901 13776 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.662667 14103 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.502965 14071 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:15.750985 14239 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.904531 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:15.972939 10200 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:15.906715 10125 rpcz_store.cc:276] 0709 13:18:10.470478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.850831 (+5380353us) service_pool.cc:224] Handling call
0709 13:18:15.858410 (+  7579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.858625 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.858642 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.858677 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:15.858761 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.858820 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.858923 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:15.908546 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52744.2us
I20260709 13:18:15.975333 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:15.861933 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 117441us
I20260709 13:18:15.908931 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:15.979575 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.099s (SENT)
I20260709 13:18:15.980098 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:15.877068 10895 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 8 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:15.842346 14021 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:15.857748 14210 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:15.917376 14204 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:15.918040 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.983201 10205 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
W20260709 13:18:15.986440 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74104.9us
I20260709 13:18:15.987322 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1786) took 5469 ms. Trace:
I20260709 13:18:15.987933 10125 rpcz_store.cc:276] 0709 13:18:10.505162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.973848 (+5468686us) service_pool.cc:224] Handling call
0709 13:18:15.974197 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.974404 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.974432 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.974475 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:15.974520 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.974578 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.974706 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:15.990818 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106853us
I20260709 13:18:15.993435 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:15.972425 14131 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:15.998358 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:15.931362 14252 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:18:15.931499 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:15.561149 13961 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.003304 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:16.004242 13961 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.004612 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:16.005267 10052 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:15.963380 14208 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:15.961999 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 103007us
I20260709 13:18:15.841333 14209 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:15.876988 14201 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:15.925885 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.008s	sys 0.009s
I20260709 13:18:15.905455 14199 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:15.971241 14256 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:15.971499 13855 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:15.914418 14236 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:15.927237 14232 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:15.999094 14200 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:15.978929 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:16.001652 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:16.067668 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:16.068930 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90073.1us
W20260709 13:18:15.950599 10019 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.810s (SENT)
I20260709 13:18:16.070531 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:15.997227 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1787) took 5487 ms. Trace:
I20260709 13:18:16.071417 10125 rpcz_store.cc:276] 0709 13:18:10.510313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.989026 (+5478713us) service_pool.cc:224] Handling call
0709 13:18:15.996597 (+  7571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.996824 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.996850 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.996899 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:15.996953 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.997021 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.997155 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.029157 14209 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:15.963543 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.075451 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:16.019091 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:16.027074 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.027905 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:16.009909 14131 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.029142 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:16.012126 14170 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:16.035192 14269 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:18:15.983907 14021 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:15.986748 13826 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:15.961941 14227 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.073515 10125 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:16.011430 10096 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.544s (SENT)
I20260709 13:18:16.086905 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:16.088891 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77507.4us
I20260709 13:18:16.090378 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1790) took 5550 ms. Trace:
I20260709 13:18:16.078539 10054 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:18:16.078092 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 127509us
I20260709 13:18:16.078742 10056 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:16.078977 10895 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:16.077957 10201 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:16.078490 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:16.072759 14226 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.673s	user 0.012s	sys 0.001s
I20260709 13:18:15.941064 14070 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:18:16.047958 14264 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:16.084439 14131 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:16.080747 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:16.108614 14221 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.277s	user 0.003s	sys 0.004s
I20260709 13:18:16.090939 10125 rpcz_store.cc:276] 0709 13:18:10.540551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.072752 (+5532201us) service_pool.cc:224] Handling call
0709 13:18:16.073253 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.090017 (+ 16764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.090305 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.072861 14236 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:16.092976 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:16.094432 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 349) took 7404 ms. Trace:
I20260709 13:18:16.048957 14201 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:16.101224 14070 consensus_queue.cc:1243] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:16.106823 14264 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:16.091686 10019 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.816s (SENT)
I20260709 13:18:16.128165 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.128767 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:16.110348 10047 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:16.075040 14208 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:16.134159 10019 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.871s (SENT)
I20260709 13:18:16.134807 10895 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:16.134939 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.111927 10200 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:16.116816 10201 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:16.119266 10131 rpcz_store.cc:276] 0709 13:18:08.689710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.262439 (+6572729us) service_pool.cc:224] Handling call
0709 13:18:15.280954 (+ 18515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.281231 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:15.446144 (+164913us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.093997 (+647853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.094062 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:16.094148 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.094232 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.094360 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":164822,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":50,"raft.queue_time_us":23747,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":148,"thread_start_us":807201,"threads_started":3,"wal-append.queue_time_us":365855}
I20260709 13:18:16.110245 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:15.991645 14239 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.122334 10122 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:16.100250 14209 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.084867 14232 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:16.085719 14021 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.085429 13826 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.842763 10892 rpcz_store.cc:276] 0709 13:18:14.638534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.641274 (+  2740us) service_pool.cc:224] Handling call
0709 13:18:14.651508 (+ 10234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.079278 (+427770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.087459 (+  8181us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.211754 (+124295us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.280430 (+ 68676us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:15.299789 (+ 19359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.679849 (+380060us) spinlock_profiling.cc:243] Waited 42.4 ms on lock 0x7b540064e1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:15.679945 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.680041 (+    96us) raft_consensus.cc:1685] finished
0709 13:18:15.680062 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.680420 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97567488,"thread_start_us":57748,"threads_started":1,"wal-append.queue_time_us":123670}
I20260709 13:18:16.136263 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:16.162783 14231 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:16.123900 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:16.132647 14264 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.172235 10054 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:18:16.180071 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 181801us
W20260709 13:18:16.147209 10019 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.873s (SENT)
I20260709 13:18:16.125519 14219 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:16.135174 14273 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:16.176591 10056 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.121893 14137 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:16.177445 10125 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:16.181300 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.181047 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:16.095081 14226 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.181883 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:16.190937 14235 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:16.193329 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:16.259821 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66586.4us
I20260709 13:18:16.260784 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.194139 13574 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.188041 14269 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:16.196069 14203 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:16.201244 14227 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:16.186645 14254 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 13:18:16.292459 10123 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:18:16.185170 14273 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.208688 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1783) took 5771 ms. Trace:
I20260709 13:18:16.209473 14131 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.172607 14251 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:16.211099 14237 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:16.212169 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65055.6us
I20260709 13:18:15.860281 10130 rpcz_store.cc:276] 0709 13:18:08.757651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.312594 (+6554943us) service_pool.cc:224] Handling call
0709 13:18:15.312880 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.341638 (+ 28758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.341688 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.358569 (+ 16881us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.832348 (+473779us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:15.856783 (+ 24435us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.857019 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.857122 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:15.857165 (+    43us) raft_consensus.cc:1685] finished
0709 13:18:15.857177 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.857313 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153821,"prepare.queue_time_us":25060,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":49200,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":11450,"spinlock_wait_cycles":23579392,"thread_start_us":427611,"threads_started":1,"wal-append.queue_time_us":430554}
I20260709 13:18:16.220501 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:16.223666 10050 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:16.225060 10129 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:16.229561 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:16.273183 13875 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:16.231643 14201 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.233167 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:16.232901 14232 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.234129 14210 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:16.240919 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 351) took 7550 ms. Trace:
I20260709 13:18:16.274318 14203 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:16.243959 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2927) took 1381 ms. Trace:
I20260709 13:18:16.244699 10202 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:18:16.246515 14199 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:16.127734 14240 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:18:16.247459 14219 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:16.330590 14151 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.001s
I20260709 13:18:16.269567 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 656) took 1120 ms. Trace:
I20260709 13:18:16.345350 10900 rpcz_store.cc:276] 0709 13:18:15.149034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.149262 (+   228us) service_pool.cc:224] Handling call
0709 13:18:16.269256 (+1119994us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:16.269480 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779743104}
I20260709 13:18:16.293808 10127 rpcz_store.cc:276] 0709 13:18:10.436880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.798899 (+5362019us) service_pool.cc:224] Handling call
0709 13:18:15.808485 (+  9586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.844173 (+ 35688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.844205 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.844259 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:15.844317 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.208347 (+364030us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b540014a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:16.208403 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.208592 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81564928}
I20260709 13:18:16.295955 10128 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:16.297158 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:16.298142 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.271932 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:16.349869 10056 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:16.176312 14091 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:16.320845 10051 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:18:16.321609 10019 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.883s (SENT)
I20260709 13:18:16.356880 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.357393 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1791) took 4211 ms. Trace:
I20260709 13:18:16.357932 10128 rpcz_store.cc:276] 0709 13:18:12.181781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.289571 (+4107790us) service_pool.cc:224] Handling call
0709 13:18:16.295664 (+  6093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.357087 (+ 61423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.357327 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.271485 13574 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:16.367420 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45870.8us
W20260709 13:18:16.369361 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.923s (SENT)
I20260709 13:18:16.329249 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.272792 14269 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:16.331983 10124 rpcz_store.cc:276] 0709 13:18:08.690287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.280104 (+6589817us) service_pool.cc:224] Handling call
0709 13:18:15.675218 (+395114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.794586 (+119368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.240569 (+445983us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54001c6e50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:16.240813 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879503616}
I20260709 13:18:16.241264 14239 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.306790 14273 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:16.335645 10052 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:18:16.335546 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65362.4us
I20260709 13:18:16.336143 10199 rpcz_store.cc:276] 0709 13:18:14.862965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.863162 (+   197us) service_pool.cc:224] Handling call
0709 13:18:16.243660 (+1380498us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54003106d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:16.243877 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101925632}
I20260709 13:18:16.309928 14235 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.263577 14072 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:18:16.328020 14272 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.239s	user 0.000s	sys 0.002s
I20260709 13:18:16.170603 13826 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:16.274382 14228 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:18:16.351956 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1785) took 5855 ms. Trace:
I20260709 13:18:16.437728 10126 rpcz_store.cc:276] 0709 13:18:10.497033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.863406 (+5366373us) service_pool.cc:224] Handling call
0709 13:18:15.879264 (+ 15858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.071958 (+192694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.071984 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.072018 (+    34us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.072055 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.351625 (+279570us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:16.351696 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.351873 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442632576}
I20260709 13:18:16.332499 14264 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.440563 10126 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:16.441507 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 366) took 4090 ms. Trace:
I20260709 13:18:16.441992 10126 rpcz_store.cc:276] 0709 13:18:12.351289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.439537 (+4088248us) service_pool.cc:224] Handling call
0709 13:18:16.440008 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.441115 (+  1107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.441418 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.327880 13875 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:16.325203 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2944) took 4642 ms. Trace:
I20260709 13:18:16.446373 10129 rpcz_store.cc:276] 0709 13:18:11.682525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.219037 (+4536512us) service_pool.cc:224] Handling call
0709 13:18:16.224680 (+  5643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.324807 (+100127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.325113 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.448808 10129 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:16.449501 10129 pending_rounds.cc:85] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:18:16.449859 10129 pending_rounds.cc:107] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:16.450212 10129 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:16.371980 10019 leader_election.cc:336] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.928s (SENT)
I20260709 13:18:16.451063 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:16.374769 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 45691.1us
I20260709 13:18:16.376526 14131 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.419796 10124 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:16.429414 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.373512 13574 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.454497 13875 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:16.378814 14269 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:16.452823 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.430675 14235 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:16.482367 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1797) took 4184 ms. Trace:
I20260709 13:18:16.354081 14151 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.381036 14279 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:16.542244 10897 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:16.542510 14274 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:18:16.543833 14284 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:16.544374 14284 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:16.545753 14151 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:16.505532 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 152943us
I20260709 13:18:16.512635 14091 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:16.373381 14207 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:16.519050 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 89773.2us
I20260709 13:18:16.571998 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:16.480988 14231 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:16.575790 14208 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:16.576483 14208 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:16.523949 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 71295.2us
I20260709 13:18:16.578413 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.525940 10124 rpcz_store.cc:276] 0709 13:18:12.302206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.379500 (+4077294us) service_pool.cc:224] Handling call
0709 13:18:16.419579 (+ 40079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.480760 (+ 61181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.482247 (+  1487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.524085 14131 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:16.532663 14264 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.440158 14223 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
W20260709 13:18:16.483907 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 112049us
W20260709 13:18:16.582223 10019 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.991s (SENT)
I20260709 13:18:16.494788 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 371) took 4033 ms. Trace:
I20260709 13:18:16.583338 10129 rpcz_store.cc:276] 0709 13:18:12.461644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.447749 (+3986105us) service_pool.cc:224] Handling call
0709 13:18:16.448243 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.494294 (+ 46051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.494672 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.584445 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:16.585119 10129 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:16.534773 14235 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:16.582842 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.588130 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:16.597921 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1807) took 2160 ms. Trace:
I20260709 13:18:16.598474 10129 rpcz_store.cc:276] 0709 13:18:14.426320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.584456 (+2158136us) service_pool.cc:224] Handling call
0709 13:18:16.584843 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.586712 (+  1869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.587014 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.535195 14276 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:16.607879 10019 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.047s (SENT)
I20260709 13:18:16.608424 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.377120 14219 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.587929 14206 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:16.614780 10019 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.083s (SENT)
I20260709 13:18:16.545374 14284 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:16.616477 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 349962us
I20260709 13:18:16.617075 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.623486 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1803) took 2263 ms. Trace:
I20260709 13:18:16.541261 14021 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.612354 14207 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:16.643911 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1802) took 2291 ms. Trace:
I20260709 13:18:16.616070 14131 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:16.611340 14279 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:16.577666 14208 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.637229 14103 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:16.604530 14285 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:18:16.616585 14226 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:16.655488 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.614533 14228 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:18:16.657846 14281 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:16.649449 14284 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:16.645105 13875 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.624397 14091 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.710994 14235 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.716976 14235 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:16.723361 14286 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
I20260709 13:18:16.728487 14269 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:16.731249 14269 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:16.625936 10019 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.145s (SENT)
I20260709 13:18:16.769573 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:16.772897 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 146923us
W20260709 13:18:16.775511 10019 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.177s (SENT)
I20260709 13:18:16.776283 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.788568 10054 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:16.605970 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:16.806656 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 200651us
I20260709 13:18:16.644397 10126 rpcz_store.cc:276] 0709 13:18:14.352127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.448240 (+2096113us) service_pool.cc:224] Handling call
0709 13:18:16.448696 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.449056 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:16.496642 (+ 47586us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.638957 (+142315us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.643539 (+  4582us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.643601 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.643650 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.643709 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.643826 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5521,"prepare.queue_time_us":50178,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":92,"raft.queue_time_us":6000,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":3993,"spinlock_wait_cycles":14540672,"thread_start_us":141581,"threads_started":1,"wal-append.queue_time_us":196187}
I20260709 13:18:16.820497 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1804) took 2446 ms. Trace:
I20260709 13:18:16.638634 10122 rpcz_store.cc:276] 0709 13:18:14.359800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.517019 (+2157219us) service_pool.cc:224] Handling call
0709 13:18:16.538934 (+ 21915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.539243 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:16.579805 (+ 40562us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.621592 (+ 41787us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.621691 (+    99us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.621794 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.621898 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.623366 (+  1468us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40912,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":3886,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":4044,"thread_start_us":40573,"threads_started":1,"wal-append.queue_time_us":104980}
I20260709 13:18:16.823787 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1795) took 4540 ms. Trace:
I20260709 13:18:16.824306 10127 rpcz_store.cc:276] 0709 13:18:12.283392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.358982 (+4075590us) service_pool.cc:224] Handling call
0709 13:18:16.368682 (+  9700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.368935 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:16.590081 (+221146us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.822674 (+232593us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.822990 (+   316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.823060 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:16.823184 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.823294 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.823548 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63463,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3293,"thread_start_us":403529,"threads_started":2,"wal-append.queue_time_us":122220}
I20260709 13:18:16.826303 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1817) took 1801 ms. Trace:
I20260709 13:18:16.826877 10122 rpcz_store.cc:276] 0709 13:18:15.024442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.824334 (+1799892us) service_pool.cc:224] Handling call
0709 13:18:16.824985 (+   651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.825429 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.825463 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.825518 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.825562 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.825620 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.826077 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.828070 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1792) took 4645 ms. Trace:
I20260709 13:18:16.829771 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1819) took 1784 ms. Trace:
I20260709 13:18:16.828516 10130 rpcz_store.cc:276] 0709 13:18:12.182550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.322440 (+4139890us) service_pool.cc:224] Handling call
0709 13:18:16.322961 (+   521us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:16.323523 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.323568 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:16.330989 (+  7421us) log.cc:844] Serialized 52 byte log entry
0709 13:18:16.502478 (+171489us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.526697 (+ 24219us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.527068 (+   371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.527191 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.722761 (+195570us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:16.827333 (+104572us) raft_consensus.cc:1685] finished
0709 13:18:16.827497 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.827870 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2006,"prepare.queue_time_us":28793,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":3410,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":3295,"spinlock_wait_cycles":39868800,"thread_start_us":123998,"threads_started":1,"wal-append.queue_time_us":256005}
I20260709 13:18:16.830189 10122 rpcz_store.cc:276] 0709 13:18:15.045676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.828369 (+1782693us) service_pool.cc:224] Handling call
0709 13:18:16.828875 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.829363 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.829389 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.829449 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.829508 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.829577 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.829706 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.821439 10124 rpcz_store.cc:276] 0709 13:18:14.373754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.579568 (+2205814us) service_pool.cc:224] Handling call
0709 13:18:16.581842 (+  2274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.582098 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:16.604431 (+ 22333us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.721804 (+117373us) spinlock_profiling.cc:243] Waited 9.38 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:16.818801 (+ 96997us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.819232 (+   431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.819340 (+   108us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.819568 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.819754 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.820153 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27773,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"spinlock_wait_cycles":21574912,"thread_start_us":212439,"threads_started":1,"wal-append.queue_time_us":91542}
I20260709 13:18:16.832638 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1823) took 1740 ms. Trace:
I20260709 13:18:16.833052 10130 rpcz_store.cc:276] 0709 13:18:15.092311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.831041 (+1738730us) service_pool.cc:224] Handling call
0709 13:18:16.831858 (+   817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.832292 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.832316 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.832374 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.832416 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.832467 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.832588 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.849148 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.854889 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 659) took 1112 ms. Trace:
I20260709 13:18:16.855592 10898 rpcz_store.cc:276] 0709 13:18:15.742592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.873396 (+130804us) service_pool.cc:224] Handling call
0709 13:18:15.873946 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.874465 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.874510 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.876290 (+  1780us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.088969 (+212679us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:16.089015 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.089132 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.717909 (+628777us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b2400021548. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274d9d9c4 00007fa274d9bacb 00007fa274f56dc9 00007fa274f56a55 00007fa274f55fb0 00007fa277bcc488 00007fa277bccbe8 00007fa277ba4398 00007fa277b9f76a 00007fa277b9ebb6
0709 13:18:16.854124 (+136215us) raft_consensus.cc:1685] finished
0709 13:18:16.854171 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.854603 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":805,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":24581888,"thread_start_us":212092,"threads_started":1,"wal-append.queue_time_us":212452}
I20260709 13:18:16.860059 14103 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:16.861481 14103 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:16.862591 14239 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:16.882890 14203 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:16.854236 10019 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.186s (SENT)
I20260709 13:18:16.859814 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:16.893333 13875 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.893953 13574 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.919727 14276 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.920681 14276 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.929383 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 2928) took 5809 ms. Trace:
I20260709 13:18:16.864233 14239 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:18:16.938998 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:16.944867 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 477) took 1281 ms. Trace:
I20260709 13:18:16.945921 14199 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:16.946924 14199 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:16.947881 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 718199us
I20260709 13:18:16.953299 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.954113 13691 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.954226 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.955370 14208 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:16.955574 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:16.956786 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 43970.3us
I20260709 13:18:16.958909 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.961373 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:16.963235 14021 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:16.944984 13574 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:16.965704 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 667528us
I20260709 13:18:16.966434 13574 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:16.945638 10897 rpcz_store.cc:276] 0709 13:18:15.663372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.854982 (+191610us) service_pool.cc:224] Handling call
0709 13:18:15.855454 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.542887 (+687433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.944163 (+401276us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:16.944611 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1578740480}
I20260709 13:18:16.967343 13574 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:16.969274 13574 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:16.971904 14072 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:16.972744 14072 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:16.946833 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.972054 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1824) took 1300 ms. Trace:
I20260709 13:18:16.975924 10130 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:16.976786 10124 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:16.979372 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:16.981758 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1778) took 8168 ms. Trace:
I20260709 13:18:16.974745 10122 rpcz_store.cc:276] 0709 13:18:15.671674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.831232 (+1159558us) service_pool.cc:224] Handling call
0709 13:18:16.909417 (+ 78185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.971475 (+ 62058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.971529 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.971616 (+    87us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:16.971698 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.971786 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.971972 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.982349 10123 rpcz_store.cc:276] 0709 13:18:08.813254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.449860 (+6636606us) service_pool.cc:224] Handling call
0709 13:18:15.502697 (+ 52837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.293206 (+790509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.981073 (+687867us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54001f3454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:16.981499 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815392512}
I20260709 13:18:16.992476 13823 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:16.993291 13823 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:16.993877 13823 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:16.998277 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.954912 13691 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:17.001322 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:16.930552 14273 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.049456 14273 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.050827 10129 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 12. Ops: []
W20260709 13:18:17.045408 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.055821 14072 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.057430 14070 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:17.058180 14072 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:17.045750 14276 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.055264 10169 leader_election.cc:400] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:17.062501 10169 leader_election.cc:403] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:17.062834 14199 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.032524 13691 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:17.096917 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48083.5us
I20260709 13:18:17.097648 14070 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:17.098291 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:17.099282 14276 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:17.103372 14137 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:18:17.105618 14137 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:17.106215 14137 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:17.111330 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1816) took 2145 ms. Trace:
W20260709 13:18:17.103904 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 324838us
W20260709 13:18:17.116873 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49846.1us
I20260709 13:18:17.111828 10126 rpcz_store.cc:276] 0709 13:18:14.965578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.817749 (+1852171us) service_pool.cc:224] Handling call
0709 13:18:16.818575 (+   826us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.819164 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.819198 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.820330 (+  1132us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.910503 (+ 90173us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:17.033729 (+123226us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.063877 (+ 30148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.064013 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.107820 (+ 43807us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b180046b8e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:17.110710 (+  2890us) raft_consensus.cc:1685] finished
0709 13:18:17.110859 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.111058 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72183,"prepare.queue_time_us":127877,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":39556,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":14650,"spinlock_wait_cycles":11796352,"thread_start_us":89506,"threads_started":1,"wal-append.queue_time_us":213578}
I20260709 13:18:17.103418 14204 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:17.103933 14203 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.116786 14196 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:18:17.133419 10019 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.186s (SENT)
I20260709 13:18:17.133047 14204 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.149051 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:17.150291 14196 consensus_queue.cc:1243] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:18:17.151930 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 53809.5us
I20260709 13:18:17.155222 14227 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.156116 14137 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.167666 14071 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:17.170001 14235 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:17.174279 14202 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:17.174582 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 379) took 1054 ms. Trace:
I20260709 13:18:17.175069 13776 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:17.176355 14021 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.185055 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:17.134531 13875 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:17.226449 13875 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.188668 14204 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.192883 10128 rpcz_store.cc:276] 0709 13:18:16.120519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.924002 (+803483us) service_pool.cc:224] Handling call
0709 13:18:16.924653 (+   651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.933106 (+  8453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.933145 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.934126 (+   981us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.039592 (+105466us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:17.063611 (+ 24019us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.093225 (+ 29614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.093345 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.174157 (+ 80812us) raft_consensus.cc:1685] finished
0709 13:18:17.174182 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.174379 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36923,"prepare.queue_time_us":23660,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":3366,"raft.queue_time_us":39720,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":219,"spinlock_wait_cycles":27688832,"thread_start_us":103704,"threads_started":1,"wal-append.queue_time_us":205151}
I20260709 13:18:17.200429 14137 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.141782 14070 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.200801 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.201485 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.207085 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.207769 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.208079 14264 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:18:17.209791 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.194967 14202 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:17.212244 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 82950.3us
I20260709 13:18:17.213022 14208 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.212261 14273 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:17.214998 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.214998 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.215729 14071 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:17.218525 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1815) took 2306 ms. Trace:
I20260709 13:18:17.224141  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:17.223918 14235 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:17.225229 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:16.930078 10131 rpcz_store.cc:276] 0709 13:18:11.119432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.161211 (+5041779us) service_pool.cc:224] Handling call
0709 13:18:16.161657 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.327729 (+166072us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:16.345002 (+ 17273us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.645717 (+300715us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.646053 (+   336us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.822407 (+176354us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.822765 (+   358us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:16.823207 (+   442us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.823537 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.823671 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.928639 (+104968us) raft_consensus.cc:1685] finished
0709 13:18:16.928686 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.929056 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33655,"prepare.queue_time_us":98490,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":150,"raft.queue_time_us":95294,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":26518,"spinlock_wait_cycles":107278080,"thread_start_us":320342,"threads_started":3,"wal-append.queue_time_us":138599}
I20260709 13:18:17.247053 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 1969) took 1049 ms. Trace:
W20260709 13:18:17.254886 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70050.7us
I20260709 13:18:17.245378 10129 rpcz_store.cc:276] 0709 13:18:14.912379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.599897 (+1687518us) service_pool.cc:224] Handling call
0709 13:18:16.600411 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.052004 (+451593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.218090 (+166086us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400389ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:17.218428 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034674432}
I20260709 13:18:17.247547 14071 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:17.262934 10200 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:17.248049 14235 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:17.254509 10047 rpcz_store.cc:276] 0709 13:18:16.190451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.192082 (+  1631us) service_pool.cc:224] Handling call
0709 13:18:16.192532 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.234904 (+ 42372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.234939 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.234986 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.235029 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.199012 (+963983us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b54002355d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:17.229787 (+ 30775us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.238864 (+  9077us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96894720}
I20260709 13:18:17.248154 14273 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:17.276296 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.278190 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.282938 10205 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:17.295097 10205 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.295712 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.297488 10205 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:17.270653 14227 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.293651 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1818) took 2258 ms. Trace:
I20260709 13:18:17.299906 13823 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.300967 14227 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.300081 10127 rpcz_store.cc:276] 0709 13:18:15.034902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.826099 (+1791197us) service_pool.cc:224] Handling call
0709 13:18:16.826630 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.869288 (+ 42658us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:17.135883 (+266595us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.283974 (+148091us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.293029 (+  9055us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.293245 (+   216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.293298 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:17.293363 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.293437 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.293578 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74856,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":63,"raft.queue_time_us":7244,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":330,"thread_start_us":251238,"threads_started":2}
I20260709 13:18:17.294979 10199 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:17.299907 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:17.280236 14071 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.309691 14232 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:17.302597 13823 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:17.270525 14310 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.133s	user 0.001s	sys 0.004s
I20260709 13:18:17.249706 14021 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:17.320540 10019 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.199s (SENT)
W20260709 13:18:17.340525 13826 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.859s	user 0.006s	sys 0.020s
I20260709 13:18:17.342224 14202 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.342381 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.347369 13875 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.349453 10130 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:17.350850 10131 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:18:17.342213 14282 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:17.341177 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:17.355369 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.360148 14227 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.504371 14227 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.505754 14203 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:17.388006 10198 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.507593 14202 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:17.381038 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 447) took 2169 ms. Trace:
I20260709 13:18:17.508704 10202 rpcz_store.cc:276] 0709 13:18:15.210936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.254743 (+ 43807us) service_pool.cc:224] Handling call
0709 13:18:16.244429 (+989686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.336405 (+ 91976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.379824 (+1043419us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b54003106d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:17.380084 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2244640000}
I20260709 13:18:17.328205 14239 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.511298 14239 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.511637 13826 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.388671 13961 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.513476 13961 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.383692 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:17.390962 14232 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:17.515754 14137 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:17.393211 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93498.9us
I20260709 13:18:17.393885 14070 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:17.393966 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1809) took 1819 ms. Trace:
I20260709 13:18:17.395846 10203 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:17.391067 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.504410 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:17.396502 14287 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:17.371100 14273 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.376008 14302 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:17.406941 14309 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.063s	user 0.000s	sys 0.002s
I20260709 13:18:17.409400 14235 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.518937 10196 rpcz_store.cc:276] 0709 13:18:15.574622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.004336 (+429714us) service_pool.cc:224] Handling call
0709 13:18:17.393505 (+1389169us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002184d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:17.393735 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361048960}
I20260709 13:18:17.417596 10129 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.436993 10199 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:17.439810 14088 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:17.440729 14201 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:17.446936 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:17.449144 14071 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.329480 10205 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:17.498572 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 330147us
W20260709 13:18:17.537604 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 196207us
W20260709 13:18:17.553333 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 212279us
I20260709 13:18:17.558899 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:17.521095 14070 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.533161 14302 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:17.562961 14217 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:18:17.563717 14088 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:17.564241 14088 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:17.564320 14021 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.517225 14232 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:17.565547 14088 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:17.567708 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 300201us
I20260709 13:18:17.570014 14240 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:17.570454 13961 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.571225 14091 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:17.571489 13826 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.571755 14091 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:17.534612 14227 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.581051 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:17.572659 14226 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.502700 14204 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.600597 10898 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:17.634188 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.573408 14137 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:17.577651 14088 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.573783 14070 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:17.597304 13961 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.651715 10200 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:17.640328 13826 raft_consensus.cc:2764] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:18:17.600463 14208 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:17.657450 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.601243 14199 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.699326 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.576797 13691 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.573159 14242 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:18:17.573076 14091 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.757136 14091 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:17.701786 14239 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.660830 14235 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:17.705122 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48906.8us
I20260709 13:18:17.762477 13823 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:17.710543 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 136434us
I20260709 13:18:17.713934 13691 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.714695 10198 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:17.717192 14287 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:17.720615 14199 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:17.729580 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 170758us
I20260709 13:18:17.729842 14273 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:17.741658 14269 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:17.746666 14203 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:17.750110 14208 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:17.755470 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:17.764528 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:17.601512 14021 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.767836 14287 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:17.793715 14095 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:17.794304 14095 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.794679 14095 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.795235 14021 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:17.790866 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 147958us
I20260709 13:18:17.800920 13875 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:17.804783 14199 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.773663 14203 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:17.798669 14021 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.828099 14204 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:17.829941 14203 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.831540 14203 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:17.835732 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47296.6us
I20260709 13:18:17.778825 13961 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.778865 14269 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.867763 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:17.870491 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.847656 14242 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:17.862205 14021 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.866976 14070 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.882658 14021 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.876504 14199 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.869593 14208 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:18:17.884944 14199 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.885558 14208 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:17.885674 14021 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.886049 14208 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:17.886857 10198 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:17.888202 14287 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.884181 14269 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:17.884157 14070 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:17.891018 14208 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.893139 14208 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:17.895457 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:17.896224 10125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.029s	sys 0.061s
W20260709 13:18:17.896602 10125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.029s	sys 0.061s
I20260709 13:18:17.897452 14323 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:17.897998 14323 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:17.899675 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.900403 14287 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:17.900063 14323 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.901599 14323 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:17.912151 14227 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.912910 14227 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:17.913174 14302 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:17.915871 14227 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.918202 13961 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:17.919175 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 280757us
I20260709 13:18:17.919803 10055 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:17.921891 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:17.922107 14088 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:17.931725 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.924175 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:17.914852 14302 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:17.942813 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 374266us
I20260709 13:18:17.943282 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:17.954360 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:17.954370 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:17.955454 10897 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
W20260709 13:18:17.954032 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.956075 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:17.961612 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:17.941200 14204 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:17.966902 10197 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:17.958371 14227 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:17.972620 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:17.923091 14088 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:18:17.939757 14201 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:17.958367 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:17.978796 10019 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.240s (SENT)
I20260709 13:18:17.986435 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:17.982018 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:17.973359 14199 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:17.993237 10050 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:17.990965 10898 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:18.001114 10055 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:17.977949 14204 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:17.993891 14239 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:17.984424 14201 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:18.009232 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.992889 14021 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:18.019445 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52929.6us
I20260709 13:18:17.961134 14208 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:18:18.021413 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:17.989256 13961 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.010349 14204 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.036437 10055 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:18.039326 10019 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.453s (SENT)
I20260709 13:18:18.040457 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:18.041069 14199 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.032572 10056 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:18.012364 14137 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.017757 14239 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.052408 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:18.000459 14227 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.061053 10049 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:18.067334 14199 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.070751 14137 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.073457 14137 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.078287 13574 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:18.080156 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:18.081487 14226 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:18.082722 14226 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.083426 14226 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.085909 14226 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.078979 13574 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:18.087809 14201 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.088218 13574 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.089233 14201 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.089629 13574 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:18.092206 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52935.9us
W20260709 13:18:18.094139 10019 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.523s (SENT)
I20260709 13:18:18.094592 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:18.103981 13961 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.059275 14239 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:18.108876 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:18.150736 10048 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:18.151517 10049 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
W20260709 13:18:18.121644 14326 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 13:18:18.118211 14325 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:18:18.162132 10199 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:18:18.107038 13961 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:18.125699 14227 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.143465 14199 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.122696 14330 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.155314 14137 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:18.118422 14021 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:18.184275 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 262477us
I20260709 13:18:18.159220 14242 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.162406 14226 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.220285 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.221832 10201 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:18.224383 10197 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:18.157210 14095 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:18.227656 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 230892us
I20260709 13:18:18.232301 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:18.241303 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 320165us
I20260709 13:18:18.237574 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.236235 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.251930 10203 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:18.225208 14199 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.258185 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.258663 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:18.260579 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.231635 14226 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.227626 14242 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.280684 10898 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:18.288666 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.313745 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 219597us
I20260709 13:18:18.233951 14330 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:18.332005 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45001.3us
I20260709 13:18:18.327546 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 385) took 1480 ms. Trace:
W20260709 13:18:18.334985 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.339428 14269 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:18.340785 14269 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:18.341192 14269 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:18.342343 14333 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:18.342235 14269 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.339550 14331 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.288833 14227 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:18:18.291985 14199 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:18.424437 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 101327us
I20260709 13:18:18.294387 14151 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.242s
W20260709 13:18:18.432755 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88726.1us
I20260709 13:18:18.332816 10125 rpcz_store.cc:276] 0709 13:18:16.847624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.980034 (+132410us) service_pool.cc:224] Handling call
0709 13:18:17.042949 (+ 62915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.147429 (+104480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.147485 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.247247 (+ 99762us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.897010 (+649763us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:17.897421 (+   411us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.897643 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.897738 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.117410 (+219672us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:18.318314 (+200904us) raft_consensus.cc:1685] finished
0709 13:18:18.327096 (+  8782us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.327414 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48174,"prepare.queue_time_us":221,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":34115,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":22278,"spinlock_wait_cycles":15463040,"thread_start_us":634075,"threads_started":1,"wal-append.queue_time_us":698666}
I20260709 13:18:18.423760 14208 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:18.379465 14345 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:18.450084 14332 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.211s	user 0.002s	sys 0.000s
I20260709 13:18:18.468149 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:18.487072 14348 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.487982 14348 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.483379 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:18.462308 14088 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:18:18.488965 14232 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:18.497056 14137 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.497969 14348 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:18.497992 14137 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.499881 14269 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.501364 14137 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.520423 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:18.521586 10048 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:18.526181 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.522714 14232 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:18.531508 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.532285 14232 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.533383 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.527768 14227 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.535838 14226 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.535604 14227 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.536451 13961 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:18.537240 13961 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:18.538206 14341 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:18:18.538558 14227 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.517831 14021 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.539626 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:18.540642 10894 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:18.536520 14226 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.483940 14095 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:18.539718 14021 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.549106 10197 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:18.545439 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:18.533420 14264 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:18.518877 14302 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.006s
I20260709 13:18:18.550232 10898 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
W20260709 13:18:18.539893 14334 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.296s	user 0.002s	sys 0.000s
I20260709 13:18:18.467033 14331 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.562965 10898 pending_rounds.cc:85] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 6
I20260709 13:18:18.518275 14350 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.568393 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.569232 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.542825 14242 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.538458 13961 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.591001 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:18.598698 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115427us
I20260709 13:18:18.572113 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:18.591747 14204 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
W20260709 13:18:18.593647 14201 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.006s
I20260709 13:18:18.568451 14345 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:18.591208 14340 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.612442 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:18.594527 14227 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.568379 14226 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.627496 14227 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.619235 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.630023 10125 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:18.630231 14226 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.566973 10898 pending_rounds.cc:107] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
I20260709 13:18:18.631270 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.631309 10898 raft_consensus.cc:2917] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 13:18:18.632648 10898 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:18.567087 14021 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.615342 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.621326 14336 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:18.622443 14331 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.622475 10131 pending_rounds.cc:85] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:18:18.623037 13961 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:18.623234 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.777s (SENT)
I20260709 13:18:18.649488 10131 pending_rounds.cc:107] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:18:18.649868 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:18.649910 10131 raft_consensus.cc:2889] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:18.620769 14091 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.863s	user 0.008s	sys 0.008s
I20260709 13:18:18.619758 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:18.636348 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67373.3us
I20260709 13:18:18.639930 10124 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:18.644035 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.644758 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.645321 14349 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:18.645444 14239 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.006s
I20260709 13:18:18.691438 14349 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:18.692039 14349 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:18.691675 14239 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.618149 14345 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.694396 14345 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:18.695961 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.696775 10126 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:18.698029 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.698675 10126 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:18.699694 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.700351 10126 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:18.701318 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.702626 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.704201 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.705026 10126 raft_consensus.cc:2368] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:18:18.706075 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.707444 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.707640 14199 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.708869 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.710506 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.711351 10126 raft_consensus.cc:2393] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:18.711951 14273 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.934s	user 0.009s	sys 0.006s
I20260709 13:18:18.712754 14273 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:18.675338 14336 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:18.714469 14336 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.704234 14330 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:18.718724 10129 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:18.675470 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.676177 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.676280 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.721710 10122 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:18.722971 14227 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:18.723539 14203 raft_consensus.cc:670] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:18.650020 14095 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:18:18.688498 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68834.2us
W20260709 13:18:18.711827 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94469us
I20260709 13:18:18.689280 14232 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:18.668241 10130 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:18.732033 14239 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:18.733073 14351 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.616091 14204 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.736703 10900 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:18.739970 10123 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
W20260709 13:18:18.742842 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.751410 14232 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:18.753832 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.755856 10200 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:18.613023 14350 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:18.743016 10019 connection.cc:256] RPC call timeout handler was delayed by -3.5331939540001258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2916s for a response
W20260709 13:18:18.758735 10019 connection.cc:256] RPC call timeout handler was delayed by -3.0092056620001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3287s for a response
W20260709 13:18:18.759182 10019 connection.cc:256] RPC call timeout handler was delayed by -2.1664839370000664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 13:18:18.759522 10019 connection.cc:256] RPC call timeout handler was delayed by -2.1322839370000111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 13:18:18.759871 10019 connection.cc:256] RPC call timeout handler was delayed by -1.8348944750000555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 13:18:18.767439 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.770234 10123 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.768299 10124 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:18.750694 14287 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.777674 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.634229 14340 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.790628 10122 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:18.803532 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.806922 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:18.829238 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:18.833299 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.833631 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.834460 10126 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:18.829474 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.836115 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.836756 10198 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:18.833472 10051 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:18.828853 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:18.840499 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:18.784863 14269 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.848058 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.856204 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.800979 14095 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.861486 10047 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 13:18:18.869589 10122 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:18.875274 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.876555 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.781178 14287 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.831308 14336 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.836174 13961 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:18.879326 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:18.863442 14331 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:18.889672 10126 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:18.887095 13961 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.898698 13961 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:18.906572 10122 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:18.907883 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:18.908614 10129 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:18.894583 14264 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:18.890808 14137 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.945981 10123 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:18.873212 14095 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.933621 14232 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.966018 14137 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.796032 14351 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:18.883065 14269 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:18.893209 14336 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:18.976334 10124 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:18:19.013379 10124 pending_rounds.cc:85] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:18:19.013641 10124 pending_rounds.cc:107] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:18:18.969652 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:18.987739 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:19.013927 10124 raft_consensus.cc:2889] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:18.996724 14088 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.022261 14021 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.022043 14088 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.023177 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.023957 10898 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:18.996860 14264 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:19.026257 14264 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.028208 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.028640 14339 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:19.029003 10198 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:19.031955 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.034212 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.035422 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.036396 10047 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:18.993774 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.001303 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:18.973657 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62307.9us
I20260709 13:18:19.004788 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.040768 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:19.041433 14362 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:18:19.041909 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.042886 10198 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:18.989863 14359 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:18:18.977728 14353 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:18:19.057950 10900 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:19.049189 10051 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:19.057289 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:18.986145 14358 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:18:18.987073 14137 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.060339 10122 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:19.008162 14222 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:19.012142 14361 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:18:18.983320 14331 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.029270 14339 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.003268 14204 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.036697 14088 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.069617 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:19.046351 14302 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:18.979206 14287 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:19.076623 10096 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.755s (SENT)
I20260709 13:18:19.060878 10123 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:18.990432 13961 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.079478 10129 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:19.077268 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:19.075045 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.078068 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.080863 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:19.087661 10126 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:19.088666 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.089749 10123 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:19.086570 10127 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:19.093955 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:19.101866 10205 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:19.070921 14355 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:19.076071 14302 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:19.111799 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124220us
I20260709 13:18:19.074400 14222 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.106837 14088 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:19.125207 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.148617 10900 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:19.125617 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.128476 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.125008 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.133579 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.145913 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:19.139228 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62620.4us
W20260709 13:18:19.158473 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.756s (SENT)
I20260709 13:18:19.158972 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:19.165102 10900 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:19.114374 10122 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:19.095480 14331 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:19.136260 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58280.9us
I20260709 13:18:19.147157 14088 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.131311 14222 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.197788 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.259321 14088 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:19.145460 13875 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:19.219589 14340 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.134052 14345 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.275957 14350 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.276345 14345 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.116501 14355 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.277158 14355 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:19.120982 14302 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.134119 14227 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:19.125023 14342 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:19.126257 14204 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.291553 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.292349 10051 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:19.128937 14071 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:18:19.254700 14336 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:19.255746 14095 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:19.258442 14371 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.127s	user 0.002s	sys 0.000s
W20260709 13:18:19.258443 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100206us
I20260709 13:18:19.303925 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:19.267421 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 77229.4us
I20260709 13:18:19.306663 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.285380 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:19.287456 10124 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:18:19.310922 10124 pending_rounds.cc:85] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:18:19.311393 10124 pending_rounds.cc:107] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
W20260709 13:18:19.320096 10130 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.003s
I20260709 13:18:19.320585 10130 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:19.341156 10126 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:19.279454 14021 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.270730 13875 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:19.287334 14351 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.363418 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.349272 10124 raft_consensus.cc:2917] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 13:18:19.363639 10127 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:19.365525 10124 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:19.347293 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.353375 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.358057 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:19.370904 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:19.379065 14369 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.253s	user 0.000s	sys 0.002s
I20260709 13:18:19.378976 13875 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.384227 14227 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:19.385015 14227 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.386631 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.387601 10897 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:19.387936 14377 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:19.390220 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.390630 14227 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.392328 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:19.392446 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 242460us
I20260709 13:18:19.393551 14351 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.393724 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.393973 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:19.394551 10897 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.394282 14351 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.395633 14340 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:19.395740 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.396481 14340 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.397047 14340 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:19.397964 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.399394 10203 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:19.402478 14379 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:19.399302 14137 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:19.392477 14372 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:18:19.405633 14137 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:19.407361 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.894s (SENT) [suppressed 46 similar messages]
I20260709 13:18:19.408006 14269 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.399761 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.410120 13776 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:19.410663 13776 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.411006 13776 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.296327 14342 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:19.355540 14345 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:19.307008 14203 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.420014 14356 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.401885 10125 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:19.344805 14350 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:19.433590 14350 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:19.421792 14342 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:19.425537 14345 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.426961 14137 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.428323 14351 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.333222 14204 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.432132 10130 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:19.394933 14349 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.006s
I20260709 13:18:19.436553 10129 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:19.442762 14351 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:19.442982 14350 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.445461 14204 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.446676 10201 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:19.446805 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.450145 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.451202 10129 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:19.454033 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.453053 14349 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.457530 10894 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:19.461938 10128 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 13:18:19.438764 14345 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:19.469795 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:19.454030 14361 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:18:19.458925 14349 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.470642 10897 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
W20260709 13:18:19.478623 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 399714us
I20260709 13:18:19.485627 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:19.486085 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:19.485291 14137 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.437460 14203 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.432883 14356 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.472832 14204 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:19.482498 13961 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:19.503983 10096 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.979s (SENT)
I20260709 13:18:19.498271 14323 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:19.517251 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:19.513557 14137 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.498178 14070 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.534788 10892 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:19.481842 13776 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.541090 10054 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:18:19.541893 10054 pending_rounds.cc:85] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 5
I20260709 13:18:19.542104 10900 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:19.518764 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:19.544350 10050 raft_consensus.cc:2368] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14.
I20260709 13:18:19.518168 14339 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.547519 14323 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:19.548661 14323 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.542234 10054 pending_rounds.cc:107] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:18:19.559948 10054 raft_consensus.cc:2889] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:19.522202 10049 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:18:19.564055 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57429.4us
I20260709 13:18:19.565080 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:19.573644 14021 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:19.653883 14137 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:19.655045 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90054.2us
I20260709 13:18:19.655615 14070 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:19.656165 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:19.656366 14070 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:19.657672 14269 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:19.657543 14070 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.658905 14072 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:19.660473 13776 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.662201 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.590307 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:19.664875 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.665410 10198 raft_consensus.cc:606] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:19.665922 10198 raft_consensus.cc:614] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:19.666131 14356 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.666760 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.669306 14342 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.669884 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.670677 10051 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:19.654862 14302 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.671612 14203 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.673552 14302 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.674583 10047 raft_consensus.cc:2410] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:18:19.676234 10047 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:19.677217 14379 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:19.678037 14340 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:19.680192 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 176281us
I20260709 13:18:19.605957 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:19.681610 10056 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:18:19.682350 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 284533us
I20260709 13:18:19.681784 10054 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:19.684875 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.685218 10055 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:19.684645 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:19.692442 10056 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.707543 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.708467 10900 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:19.709054 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.711793 14088 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.716318 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:19.717375 10892 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:19.695500 14356 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:19.719794 10047 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:19.728260 14340 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.735301 10053 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:19.735690 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51869.5us
I20260709 13:18:19.745471 10056 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:19.746600 13776 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.747706 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:19.749603 14342 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:19.758494 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66791.9us
I20260709 13:18:19.697679 14072 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:19.699908 14383 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.761547 10170 leader_election.cc:400] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14.
I20260709 13:18:19.761966 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.738523 14356 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:19.760550 13776 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.769958 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:19.771399 13776 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.771598 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:19.772517 14379 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:19.772961 14387 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:19.773209 14342 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:19.773649 14342 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.774096 14342 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:19.777227 14072 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.781009 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.782469 14357 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:19.783377 14021 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:19.783931 14021 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.784330 14021 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:19.767987 14356 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:19.773017 14379 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.787984 14379 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:19.764860 14340 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:19.788903 14340 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:19.789408 14340 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:19.764561 14383 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.773669 14387 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:19.793602 14383 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.793977 14387 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.794661 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:19.795825 14355 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.003s
I20260709 13:18:19.796475 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.796762 14355 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.797420 14137 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:19.798125 10052 raft_consensus.cc:2368] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11.
I20260709 13:18:19.798422 14137 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:19.805073 14227 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.805917 10019 leader_election.cc:400] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:18:19.806311 10019 leader_election.cc:403] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:19.805835 14227 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:19.808449 14227 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.809015 14227 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:19.811638 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44325.5us
I20260709 13:18:19.813135 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:19.814546 14203 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.817129 14203 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.819973 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.820253 14379 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.821175 14137 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:19.820253 14356 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.815955 14383 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:19.827652 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.831779 14137 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:19.832443 14137 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:19.809567 14227 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.833060 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.825071 14203 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.834431 14356 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.847185 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.849027 10900 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:19.850276 13855 raft_consensus.cc:1064] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:18:19.850797 13855 raft_consensus.cc:1068] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Unable to promote non-voter db052bef7536453fa2d7f5a00e314a18: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:18:19.852062 14232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.032s	sys 0.070s
W20260709 13:18:19.852531 14232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.032s	sys 0.070s
I20260709 13:18:19.861905 14340 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.838394 14269 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:19.869326 14345 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:19.875198 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.887630 10123 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:19.881942 14287 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:19.888401 13961 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:19.895444 14227 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.833534 14137 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.899477 14287 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:19.900040 14287 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:19.900765 14331 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.901332 14287 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:19.889279 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44621.8us
I20260709 13:18:19.911047 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.915947 10124 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:18:19.890818 10900 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:19.883528 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:19.906247 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:19.918139 10128 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:19.919090 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:19.919833 10897 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:19.920181 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:19.907493 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:19.841629 14355 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:19.923223 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:19.923619 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.920802 10124 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:19.929042 10049 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:19.929050 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.939810 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:19.941980 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109048us
I20260709 13:18:19.945056 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.946048 10198 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:19.934827 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.945201 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:19.942888 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:19.936429 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:19.966158 10199 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:19.967033 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:19.964542 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:19.970734 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 57469.6us
I20260709 13:18:19.972066 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:19.926853 13961 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:19.980854 14340 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:19.981729 10897 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:19.986339 10898 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:18:19.988734 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45954.2us
I20260709 13:18:19.998198 10892 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:20.000044 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:20.001799 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:19.946992 14287 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:20.005038 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.009582 14381 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:20.010499 14357 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:19.970057 14350 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:20.011559 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.011893 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:20.049108 10201 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:20.049669 14399 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:20.012619 13691 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:20.051369 14070 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.051914 14070 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.052347 14070 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.012714 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.023293 14137 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:20.036182 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
W20260709 13:18:20.037446 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130022us
I20260709 13:18:20.009407 13961 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:20.055265 13961 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:19.977727 14349 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:20.058275 14349 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.058791 14349 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:20.065385 13691 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:20.064849 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:20.070892 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:19.989014 14095 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:19.970960 14379 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.004743 13776 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.037036 13823 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.078688 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.095561 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.062613 14070 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.089560 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.089221 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:20.076448 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109507us
I20260709 13:18:20.077507 14242 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:20.077121 14340 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:20.099104 10201 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:20.048818 14350 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:20.088142 14071 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:18:20.005975 14356 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:18:20.083132 14338 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:20.122599 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.114403 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.116015 10170 leader_election.cc:400] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11.
I20260709 13:18:20.117036 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.124915 10197 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.083316 14336 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:20.086723 14339 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:20.120451 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:20.127390 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48497.1us
W20260709 13:18:20.127811 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 138167us
I20260709 13:18:20.125839 10170 leader_election.cc:403] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:20.104308 14399 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:20.111301 14242 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.124895 10201 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:20.116153 14350 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.104931 14021 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.123303 14338 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.140360 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:20.168002 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52192.9us
I20260709 13:18:20.132592 14339 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.131501 14383 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9.
I20260709 13:18:20.208637 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:20.207384 14369 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:18:20.180828 14232 raft_consensus.cc:1064] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:18:20.215530 14232 raft_consensus.cc:1068] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Unable to promote non-voter db052bef7536453fa2d7f5a00e314a18: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:20.159161 14242 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:20.227291 10128 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:20.183393 14338 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:20.203655 14323 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.179302 14095 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:20.142514 14381 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:18:20.213394 14339 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.247759 10055 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:20.255724 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:20.259917 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128856us
I20260709 13:18:20.293902 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.295701 10197 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:20.312573 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.266696 14378 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:20.322100 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 231217us
I20260709 13:18:20.276682 14070 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:20.260218 14342 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.344225 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:20.312474 14227 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:20.352006 14398 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 13:18:20.355605 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 94644.3us
I20260709 13:18:20.356762 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:20.358300 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.359826 14021 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:20.361833 14342 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.293642 14349 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.365783 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:20.370226 10130 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.364248 14349 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.327589 13961 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:20.378690 14383 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:20.376973 14222 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:20.395221 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.400696 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.403054 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 673) took 1146 ms. Trace:
I20260709 13:18:20.404359 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.403728 10894 rpcz_store.cc:276] 0709 13:18:19.256299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.269459 (+ 13160us) service_pool.cc:224] Handling call
0709 13:18:19.269827 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.457947 (+188120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.402664 (+944717us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540064f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:20.402956 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431210880}
W20260709 13:18:20.415236 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92845.8us
I20260709 13:18:20.420967 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:20.378948 14340 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:20.309767 14399 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.421099 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.426299 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:20.427529 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:20.328222 14378 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.427238 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.375511 14227 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.414651 14021 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:20.430490 14383 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:20.432078 14340 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:20.428776 14403 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:20.473515 14404 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:18:20.423254 14349 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:18:20.482533 14227 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.485077 14378 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:20.488798 13776 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:20.490310 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 132088us
I20260709 13:18:20.495474 10130 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:20.498457 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:20.453800 14137 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:20.389547 14095 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.534487 13776 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.486570 14339 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.537431 14339 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:20.471563 14383 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:20.492151 14340 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.470814 14021 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:20.587847 10096 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.613s (SENT)
I20260709 13:18:20.548130 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:20.543084 10196 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:20.627965 10054 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:20.636765 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 192044us
W20260709 13:18:20.598989 14390 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.654s	user 0.000s	sys 0.005s
I20260709 13:18:20.648032 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2029) took 1125 ms. Trace:
W20260709 13:18:20.650588 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 202232us
I20260709 13:18:20.555572 14383 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:20.648734 10049 rpcz_store.cc:276] 0709 13:18:19.563244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.570690 (+  7446us) service_pool.cc:224] Handling call
0709 13:18:19.571242 (+   552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.929603 (+358361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.639563 (+709960us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b5400135954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:20.639846 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822345984}
I20260709 13:18:20.574150 14323 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11.
I20260709 13:18:20.600366 14088 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.568864 13823 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:20.669617 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:20.675256 10900 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.568795 14137 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:20.672544 10898 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:20.683869 10892 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:20.684829 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97021.2us
I20260709 13:18:20.696950 10197 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:18:20.698645 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:20.543795 14403 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.719792 14137 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.558378 14350 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.543100 13776 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:20.585107 14242 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:20.751890 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.812s (SENT)
I20260709 13:18:20.765202 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:20.757121 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:20.786834 10129 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:20.789824 10131 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:20.792945 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:20.794035 10131 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:20.799867 10202 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:20.801717 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49915.8us
W20260709 13:18:20.803263 10096 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.621s (SENT)
I20260709 13:18:20.803800 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:20.808905 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:20.809842 10124 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:20.820849 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:20.799295 13823 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:20.811738 14227 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:20.852401 10897 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:20.822450 14356 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:20.827288 14021 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:20.829778 14222 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:20.799435 14415 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:18:20.857080 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49268.7us
W20260709 13:18:20.860332 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 312335us
I20260709 13:18:20.796068 14424 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:18:20.867300 13961 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:20.870573 14433 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:20.871409 14433 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.874203 14433 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.876632 14350 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:20.877796 14338 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.009s
I20260709 13:18:20.878337 14338 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.880071 14340 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.881388 14338 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:20.855957 14379 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:20.887524 14379 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:20.887861 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.888083 14379 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.897315 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:20.902297 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.882951 14340 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.863205 13823 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:20.865073 14323 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:18:20.919799 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:20.932875 10897 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:20.940864 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:20.959601 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:20.941062 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:20.954545 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50915.7us
I20260709 13:18:21.180393 14071 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:21.192857 14071 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:21.194489 13875 raft_consensus.cc:993] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: : Instructing follower 01bfaf1ad56a4deea886bbc64dec466c to start an election
I20260709 13:18:21.194294 14417 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:21.001634 10096 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.679s (SENT)
I20260709 13:18:21.194015 14071 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.195706 14417 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:21.192096 14242 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.197798 13875 raft_consensus.cc:1081] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Signalling peer 01bfaf1ad56a4deea886bbc64dec466c to start an election
I20260709 13:18:21.188453 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:21.240365 10131 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:20.989338 14381 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:20.924300 13776 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.004942 10052 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:20.973874 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:20.967756 10900 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:20.962425 10049 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:20.985321 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:20.970932 10124 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:21.266386 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 69551.5us
I20260709 13:18:20.925853 14424 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.198228 10054 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:21.100795 14222 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:21.077721 14403 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:21.205153 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:21.211709 14426 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:20.979664 14345 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:20.997083 14429 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.228s
I20260709 13:18:21.237679 10196 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:21.288718 10196 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:21.245649 14381 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:20.939291 14433 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.243520 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:21.177944 14379 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:20.931602 14431 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.350s
I20260709 13:18:20.979275 14342 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:21.077996 10205 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:21.269869 10056 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:18:21.201115 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 260506us
I20260709 13:18:20.978960 13961 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.275642 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:20.976864 14410 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:20.940150 14021 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:21.279059 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:21.359285 10897 raft_consensus.cc:2368] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:21.359570 10048 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:21.279469 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:21.284461 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:21.231380 14417 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:21.286412 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 284826us
I20260709 13:18:21.272768 14071 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.239046 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:21.293991 10200 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.244657 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:20.938143 14383 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.298188 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:21.300542 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:21.369626 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69168.7us
I20260709 13:18:21.301935 14407 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:21.303470 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:21.306391 14323 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:21.306530 14070 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:21.307571 14222 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.289671 10196 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.403342 10196 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.292074 14242 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:18:21.309422 14405 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:21.316673 10049 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:21.277984 14424 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:20.982969 14434 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.307s
I20260709 13:18:21.279596 14137 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:21.348304 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:20.926101 13823 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.350153 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.354795 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.473070 10047 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.474361 10051 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:21.291612 14403 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:21.360263 10125 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:21.360368 10197 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:21.362579 10096 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.152s (SENT)
I20260709 13:18:21.308918 14338 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:21.347337 13961 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:21.409377 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.417246 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:21.400288 14337 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:21.468063 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.288458 14426 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.475912 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:21.389691 14410 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:21.484189 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.465168 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.499038 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.513868 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.524641 10197 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:21.535478 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50802.7us
I20260709 13:18:21.405274 13875 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:18:21.547708 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.505990 14418 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:18:21.398974 14021 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.495332 14337 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.499935 13961 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.576910 10056 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:21.404531 14416 consensus_queue.cc:1175] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "db052bef7536453fa2d7f5a00e314a18" responder_term: 14 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:21.598461 14356 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:21.423069 14381 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.578845 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:21.609061 14445 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:18:21.609941 14379 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.611178 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2054) took 1006 ms. Trace:
I20260709 13:18:21.611824 14021 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:21.590775 14443 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 13:18:21.611652 10053 rpcz_store.cc:276] 0709 13:18:20.605065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.609496 (+  4431us) service_pool.cc:224] Handling call
0709 13:18:20.609796 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.830163 (+220367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.830205 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.893734 (+ 63529us) log.cc:844] Serialized 26 byte log entry
0709 13:18:21.245090 (+351356us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:21.245169 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.257707 (+ 12538us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.610922 (+353215us) raft_consensus.cc:1685] finished
0709 13:18:21.610945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.611086 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":217167,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6969,"thread_start_us":488273,"threads_started":2,"wal-append.queue_time_us":418672}
I20260709 13:18:21.580302 14403 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:21.587894 14448 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:18:21.616096 14410 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:21.583185 14444 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:18:21.616163 14403 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.621912 14403 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:21.624799 14383 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.628890 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.585134 10230 heartbeater.cc:646] Failed to heartbeat to 127.8.183.126:40341 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.183.126:40341 timed out after 15.000s (SENT)
I20260709 13:18:21.630537 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.636363 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.637387 10054 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.496228 14417 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.638339 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.614003 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.639971 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.643467 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47667.7us
I20260709 13:18:21.655655 10055 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:21.657254 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.663280 10053 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.665022 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:21.669137 10055 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:21.670646 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.672205 10055 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:21.577361 14337 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:21.673935 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.582346 14428 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:21.688422 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 237512us
I20260709 13:18:21.541335 14137 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.694962 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.698688 10056 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:21.702405 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:21.702965 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.701131 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 276059us
W20260709 13:18:21.725210 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 475) took 1717 ms (client timeout 1536 ms). Trace:
W20260709 13:18:21.726545 10203 rpcz_store.cc:269] 0709 13:18:20.007965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.012288 (+  4323us) service_pool.cc:224] Handling call
0709 13:18:21.724870 (+1712582us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400354654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:21.725067 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2829105024}
W20260709 13:18:21.660804 14419 consensus_queue.cc:1175] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 12 status { last_received { term: 12 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 12 index: 6 } }
I20260709 13:18:21.603145 14357 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:18:21.741245 10894 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:21.739845 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 205205us
I20260709 13:18:21.745810 14411 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.746435 14411 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:21.747620 14411 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.749112 14411 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.752360 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.753711 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.705718 14381 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:21.762668 14242 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:21.763147 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:21.763674 14242 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.767632 14021 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:21.768033 14397 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.768558 14397 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.760594 10053 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.759799 10202 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:21.699963 14383 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:21.761826 10049 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:21.700899 14204 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:18:21.777526 10095 leader_election.cc:400] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:21.783346 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.787201 10095 leader_election.cc:403] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:21.787879 10053 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.793301 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:21.796556 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 433983us
W20260709 13:18:21.797739 10096 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.173s (SENT)
I20260709 13:18:21.798179 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:21.813242 10047 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:21.795077 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.816303 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:21.818245 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.826347 10047 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:21.828868 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.829254 10900 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:21.829460 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:21.832404 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.624886 14424 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.849804 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.815572 14021 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:21.853098 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:21.856642 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.857399 14405 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.857642 10047 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.857939 10196 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:18:21.858144 14460 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.858603 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.858760 14460 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:21.861045 14422 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:18:21.862385 14455 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:18:21.863087 14242 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:21.863335 14425 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:21.863826 14425 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:21.865101 14384 raft_consensus.cc:1064] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:18:21.870324 14466 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:21.871387 14466 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:21.876288 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78619.1us
I20260709 13:18:21.877151 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.878198 10892 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:21.878641 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.786396 14383 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.864792 14425 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.855595 14434 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.879949 14434 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.880277 14425 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:21.880956 14383 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:21.882419 14434 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.883431 10056 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:21.858371 10196 pending_rounds.cc:85] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 10
I20260709 13:18:21.885072 10196 pending_rounds.cc:107] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:21.885515 10196 raft_consensus.cc:2889] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:21.885126 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.860930  9949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 118) took 12394 ms (client timeout 15000 ms). Trace:
I20260709 13:18:21.891781 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:21.819229 14356 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:18:21.894914 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.896051 14356 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:21.896502 14356 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:21.896643 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.897521 14356 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:21.898190 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.857853 14405 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:21.899575 14222 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:21.921743 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.927234 10049 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.932808 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.898943 14356 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.870364 14137 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.935250 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.938452 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.900059 14222 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:21.943317 14222 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:21.944017 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:21.944639 10047 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8. Re-sending same reply.
I20260709 13:18:21.945669 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.946380 10047 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:21.948370 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.948658 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.949014 10055 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:21.955335 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.956840 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:21.957520 14071 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.006s
I20260709 13:18:21.959009 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:21.932677 14397 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:21.947184 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:21.953864 10200 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.001s
I20260709 13:18:21.967857 10200 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:21.978646 10204 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:21.989425 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 173217us
I20260709 13:18:21.900024 14405 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:21.990924 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.995364 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.996276 10049 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:21.997452 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.000216 10168 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:22.002141 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 125071us
W20260709 13:18:22.003263 10096 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.677s (SENT)
I20260709 13:18:22.003751 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:22.005764 10096 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.702s (SENT)
I20260709 13:18:21.998903 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:21.937883 13961 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.014431 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:22.025148 10047 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:22.004488 14350 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.007156 14071 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:22.007946 10049 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:22.030723 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.036052 10048 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.024577 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:22.032650 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:22.041508 10200 raft_consensus.cc:606] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:22.037969 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.042856 10047 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.042034 10050 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.035395 14350 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.057013 14072 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:22.057626 14072 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:22.044229 14071 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:22.061437 14071 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.038563 14469 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.069530 14469 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.070114 14469 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.030073 14405 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.077769 13776 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.078451 13776 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.107682 10198 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:18:22.090023 14350 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.091065 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:22.106709 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 2966) took 1918 ms (client timeout 1716 ms). Trace:
I20260709 13:18:22.088061 14460 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:22.124280 10049 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.103827 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.124336 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:22.122892 10205 rpcz_store.cc:269] 0709 13:18:20.188068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.208348 (+ 20280us) service_pool.cc:224] Handling call
0709 13:18:22.106397 (+1898049us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:22.106594 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128860928}
W20260709 13:18:22.126765 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102296us
I20260709 13:18:22.131736 10131 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:22.095492 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.133772 10048 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:22.101754 14469 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.140332 10126 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:22.125329 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.145226 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.146941 14137 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.155318 14137 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.157837 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.524s (SENT)
I20260709 13:18:22.158588 14137 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.162227 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.163230 10122 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:22.158409 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.166780 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:22.168008 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.169734 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.139348 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.167990 10123 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:18:22.159600 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:22.058606 14072 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.168921 10053 raft_consensus.cc:2376] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9. Re-sending same reply.
I20260709 13:18:22.173314 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:18:21.890213  9949 rpcz_store.cc:269] 0709 13:18:09.470420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.557673 (+ 87253us) service_pool.cc:224] Handling call
0709 13:18:11.648257 (+2090584us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:18:21.860639 (+10212382us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:16.574348 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:16.575022 (+   674us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:16.575064 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:18:16.575085 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:16.720212 (+145127us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:16.720363 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:16.720415 (+    52us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:18:16.722329 (+  1914us) tablet.cc:701] Row locks acquired
0709 13:18:16.722363 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:18:16.722635 (+   272us) write_op.cc:270] Start()
0709 13:18:16.722849 (+   214us) write_op.cc:276] Timestamp: P: 1783603096722548 usec, L: 0
0709 13:18:16.722889 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:18:16.733960 (+ 11071us) log.cc:844] Serialized 5564 byte log entry
0709 13:18:18.090590 (+1356630us) op_driver.cc:464] REPLICATION: finished
0709 13:18:18.335293 (+244703us) write_op.cc:301] APPLY: starting
0709 13:18:18.335517 (+   224us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:18.898053 (+562536us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:18.952675 (+ 54622us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:19.101158 (+148483us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18000861a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:19.660720 (+559562us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:19.661178 (+   458us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:18:19.661229 (+    51us) write_op.cc:312] APPLY: finished
0709 13:18:19.661740 (+   511us) log.cc:844] Serialized 129 byte log entry
0709 13:18:19.767454 (+105714us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.780698 (+ 13244us) write_op.cc:454] Released schema lock
0709 13:18:19.780780 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:19.781499 (+   719us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.781535 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":189524,"reported_tablets":22,"spinlock_wait_cycles":588919680,"child_traces":[["op",{"apply.queue_time_us":244167,"apply.run_cpu_time_us":65513,"apply.run_wall_time_us":1446221,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62535,"num_ops":14,"prepare.queue_time_us":329008,"prepare.run_cpu_time_us":56902,"prepare.run_wall_time_us":542809,"raft.queue_time_us":334399,"raft.run_cpu_time_us":12376,"raft.run_wall_time_us":163260,"replication_time_us":1367415,"spinlock_wait_cycles":27124992,"thread_start_us":993806,"threads_started":5,"wal-append.queue_time_us":735497,"wal-append.run_cpu_time_us":27510,"wal-append.run_wall_time_us":2692990}]]}
I20260709 13:18:22.174961 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.188546 10205 raft_consensus.cc:606] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:22.166685 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.137923 14379 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.194905 10049 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.306687 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.307619 10049 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:22.308694 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.315780 10049 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:22.306906 14481 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:18:22.220052 14469 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.247331 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.305028 14483 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:22.304692 14379 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.338387 14428 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.356559 14428 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.344571 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.360878 10205 raft_consensus.cc:614] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:22.354662 10053 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:22.338703 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84926.8us
W20260709 13:18:22.362888 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 205084us
I20260709 13:18:22.317328 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:22.363694 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.624s (SENT)
W20260709 13:18:22.277283 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.314595 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:22.364153 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.172052 13855 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:22.147747 14415 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:22.274420 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.366919 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.371776 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.368950 14469 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.414032 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.389976 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.389904 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.424677 10123 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:22.424989 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.391853 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.425808 10130 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:22.426100 10122 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.426784 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.424738 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.416436 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.428539 10049 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.417109 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.429344 10048 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.419945 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.194954 14458 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:22.416471 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:22.422065 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51326.7us
W20260709 13:18:22.365326 10096 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.702s (SENT)
I20260709 13:18:22.425808 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.427462 10127 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:22.431609 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.369500 14483 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.245872 14071 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.170s
I20260709 13:18:22.431692 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.450073 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.450815 10128 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:22.360621 13961 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.246970 14425 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.445577 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.445952 14415 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:22.460897 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.447827 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.462895 10048 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.187966 14337 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.436136 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70835.1us
I20260709 13:18:22.468195 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.454885 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.469199 10124 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:22.438007 14443 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:18:22.468417 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.734s (SENT)
I20260709 13:18:22.460577 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.464063 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.312146 14222 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:22.477452 10096 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.735s (SENT)
I20260709 13:18:22.413619 14416 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:18:22.481212 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.470889 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.482576 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.494382 10131 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:22.465065 14411 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.708s	user 0.016s	sys 0.000s
I20260709 13:18:22.496256 10124 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:22.373471 13855 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.487605 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.488226 14070 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:22.482482 14484 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.495867 10122 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:22.509910 10096 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.001s (SENT)
I20260709 13:18:22.453006 14458 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:22.521798 10126 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:22.524457 10122 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:18:22.492792 14137 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.473384 14488 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.534324 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.541507 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.542636 10048 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:22.467425 14425 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.549227 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.549388 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2074) took 1196 ms. Trace:
I20260709 13:18:22.532625 14201 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.550716 10052 rpcz_store.cc:276] 0709 13:18:21.352659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.352837 (+   178us) service_pool.cc:224] Handling call
0709 13:18:21.353274 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.753981 (+400707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:21.754017 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:21.754075 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:21.754122 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.549049 (+794927us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b5400198f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:22.549133 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.549310 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920945920}
I20260709 13:18:22.555441 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.556289 10052 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:22.557564 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.561388 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.562208 10048 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:22.563728 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.508663 14462 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:22.567503 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.568266 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.572249 14458 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.541231 14411 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:22.575266 14425 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:22.554652 14442 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:18:22.533762 14484 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.551577 14415 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.564363 13961 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.594781 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.610131 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.610227 10122 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:18:22.598277 14021 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.015s
W20260709 13:18:22.605086 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 95238.4us
I20260709 13:18:22.552925 14483 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:22.583058 14356 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.635s	user 0.008s	sys 0.003s
I20260709 13:18:22.608726 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.597395 10052 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:22.634927 10048 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:22.572327 13855 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:22.601956 14411 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:22.618404 14337 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:22.619925 14222 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:22.624162 10128 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:22.625609 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:22.629344 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.630s (SENT)
I20260709 13:18:22.642560 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.586761 14428 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.645640 14483 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:22.646014 14428 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:22.646418 14428 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.589460 14384 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.646792 14428 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.649470 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2975) took 1407 ms. Trace:
I20260709 13:18:22.650099 10198 rpcz_store.cc:276] 0709 13:18:21.241866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.242022 (+   156us) service_pool.cc:224] Handling call
0709 13:18:22.107280 (+865258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.119901 (+ 12621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.649102 (+529201us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540035b1d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:22.649376 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989045632}
I20260709 13:18:22.658459 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:22.583336 14201 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.617755 14379 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.614841 14469 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.547792 14137 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.585346 14406 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:22.609731 14420 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.645156 10096 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.810s (SENT)
W20260709 13:18:22.631809 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 169752us
I20260709 13:18:22.765128 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:22.766386 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 121280us
W20260709 13:18:22.767136 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.942s (SENT)
I20260709 13:18:22.767596 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.668358 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:22.669675 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:22.768860 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99289.1us
I20260709 13:18:22.677292 10051 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:22.770005 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 81814.5us
I20260709 13:18:22.634048 14021 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:22.687836 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1864) took 1327 ms. Trace:
W20260709 13:18:22.772087 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.998s (SENT)
I20260709 13:18:22.789304 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.789070 10050 rpcz_store.cc:276] 0709 13:18:21.349267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.349892 (+   625us) service_pool.cc:224] Handling call
0709 13:18:22.676579 (+1326687us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b54001b7954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:22.676852 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1295942528}
I20260709 13:18:22.772706 14469 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.790934 14420 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.791656 14469 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:22.791993 14222 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:22.790987 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.059s (SENT)
I20260709 13:18:22.793313 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:22.794607 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.065s (SENT)
I20260709 13:18:22.794796 13855 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:22.795145 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.773247 14483 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.774935 14434 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:22.796463 14434 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.798936 14434 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.799566 14434 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:22.775743 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.800521 14429 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.802037 14469 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:22.802881 14469 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.804548 14201 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.800078 14434 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.805469 14411 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.806671 14411 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:22.807497 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.894s (SENT)
I20260709 13:18:22.808037 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:22.795536 13855 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.780632 14484 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.811409 14484 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.692584 14428 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.812980 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.928s (SENT)
I20260709 13:18:22.813042 13855 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.813503 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.792564 14222 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.814186 14428 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.809396 14434 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.814317 14222 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.815160 14424 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.691680 14406 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:22.820446 14406 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.821866 14406 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:22.773005 14379 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.823534 14484 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:22.823516 14379 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.826691 14379 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:22.827340 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.278s (SENT)
I20260709 13:18:22.685876 14495 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:22.827956 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.828070 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:22.829051 10048 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:22.829293 10050 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.631757 13961 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.831636 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65456.1us
I20260709 13:18:22.831952 13961 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.836215 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:22.837005 10127 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:22.841312 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.793854 14420 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.873895 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:22.874780 10129 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:22.874626  9951 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.875998 14222 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:22.876607  9951 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.878350 14222 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:22.878686 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:22.879351 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52064.1us
I20260709 13:18:22.881266  9951 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 9, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 9 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:22.883597 14137 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:22.884109 13961 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:22.884706 13961 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:22.889405 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.874480 10125 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:22.947396 10050 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:22.949263 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:22.949331 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:22.950166 10050 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:22.951449 14483 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:22.952157 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.952765 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.953765 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64508us
I20260709 13:18:22.954236 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:22.955520 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.883015  9951 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 4 to 10, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.964725 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.981760 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:22.982612 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:22.980420 14483 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:22.986130 13855 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.987401 14469 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:22.987061 13855 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:22.989707 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.990307 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.991712 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:22.992539 10047 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:22.993183 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1916) took 1032 ms. Trace:
I20260709 13:18:22.993551 10047 rpcz_store.cc:276] 0709 13:18:21.961028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.991347 (+1030319us) service_pool.cc:224] Handling call
0709 13:18:22.993120 (+  1773us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:22.997162 13961 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:22.999032 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:23.000905 10047 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 8: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:22.901983 14021 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:23.008965 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 340717us
I20260709 13:18:23.003841 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:23.005330 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.006690 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:23.007649 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:23.010664 10050 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:23.011186 10052 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:22.977696 14201 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:23.015686 10129 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:23.017190 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:23.015472 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:23.018492 10049 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:23.010113 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:23.022133 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.025017 10050 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:23.025866 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2087) took 1668 ms. Trace:
I20260709 13:18:23.026319 10050 rpcz_store.cc:276] 0709 13:18:21.357545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.024184 (+1666639us) service_pool.cc:224] Handling call
0709 13:18:23.024650 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.025589 (+   939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.025808 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.002784 14484 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.028208 10052 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.029691 10047 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:23.052114 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2086) took 1694 ms. Trace:
I20260709 13:18:23.054337 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1919) took 1050 ms. Trace:
I20260709 13:18:23.054193 10052 rpcz_store.cc:276] 0709 13:18:21.357218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.024072 (+1666854us) service_pool.cc:224] Handling call
0709 13:18:23.027959 (+  3887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.051801 (+ 23842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.052043 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.054859 10047 rpcz_store.cc:276] 0709 13:18:22.003845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.003550 (+999705us) service_pool.cc:224] Handling call
0709 13:18:23.054266 (+ 50716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.053524 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:23.011879 14407 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.052139 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:23.053369 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.076346 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2090) took 1676 ms. Trace:
I20260709 13:18:23.082170 10052 rpcz_store.cc:276] 0709 13:18:21.400076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.055338 (+1655262us) service_pool.cc:224] Handling call
0709 13:18:23.055703 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.075875 (+ 20172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.075920 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.075977 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:23.076015 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.076095 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.076264 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.082548 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:23.018074 14356 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:23.084904 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58164.8us
I20260709 13:18:23.095012 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2089) took 1728 ms. Trace:
W20260709 13:18:23.099560 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46119.9us
I20260709 13:18:23.100306 10049 rpcz_store.cc:276] 0709 13:18:21.371040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.047273 (+1676233us) service_pool.cc:224] Handling call
0709 13:18:23.094237 (+ 46964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.094505 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.094544 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.094608 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:23.094677 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.094752 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.094923 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.024344 14383 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.028407 14137 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:23.027788 14493 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:23.014823 13961 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:18:23.031440 10050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:46802
I20260709 13:18:23.111946 10050 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:23.112385 10050 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:23.102589  9951 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 11, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:23.105352 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52085.8us
I20260709 13:18:23.116103 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.029793 14495 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.030725 14457 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:23.028216 14201 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:23.125312 14201 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.092485 14428 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.119397 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.127539 14428 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:23.104576 14489 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:18:23.076990 14434 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.028115 14379 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
W20260709 13:18:23.104153 14405 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.028s	user 0.015s	sys 0.003s
I20260709 13:18:23.131486  9951 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.134125 10049 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:23.107847 14458 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:23.092352 14021 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:23.128216 14428 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.197418 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2093) took 1715 ms. Trace:
I20260709 13:18:23.211920 10049 rpcz_store.cc:276] 0709 13:18:21.482192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.133374 (+1651182us) service_pool.cc:224] Handling call
0709 13:18:23.133920 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.197056 (+ 63136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.197333 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.118930 14462 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:23.166133 10050 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:23.227568 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.118242 14492 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.229418 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:23.233894 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.230417 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.230839 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:23.231210 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.099398 14407 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.231556 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.235365 10196 raft_consensus.cc:2410] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:18:23.133008 14356 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.242931 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.235743 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:23.237676 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:23.234980 10202 raft_consensus.cc:2376] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11. Re-sending same reply.
I20260709 13:18:23.237541  9951 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 8, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 8 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:23.127537 14457 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:23.212252 14458 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.119359 14137 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:23.119113 14383 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.211730 14434 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.242043 14492 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:23.232609 14462 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:23.211514 14021 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:23.129329 14493 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.236656 14483 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:23.134624 14495 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.256836 10050 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:23.266045  9951 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.266737 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.353701 10204 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:23.274087 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.277648 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.255918 10205 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:23.279940 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.292268 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2083) took 1936 ms. Trace:
I20260709 13:18:23.294135 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1926) took 1212 ms. Trace:
W20260709 13:18:23.300244 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1893) took 1645 ms (client timeout 1810 ms). Trace:
I20260709 13:18:23.278005 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:23.315692 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 233197us
I20260709 13:18:23.368422 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46802 (request call id 2088) took 1992 ms. Trace:
I20260709 13:18:23.368988 10050 rpcz_store.cc:276] 0709 13:18:21.366228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.027269 (+1661041us) service_pool.cc:224] Handling call
0709 13:18:23.350194 (+322925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.325385 10199 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:23.279620 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.308867 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2094) took 1826 ms. Trace:
I20260709 13:18:23.374193 10049 rpcz_store.cc:276] 0709 13:18:21.482511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.260862 (+1778351us) service_pool.cc:224] Handling call
0709 13:18:23.308240 (+ 47378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.308461 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.308487 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.308547 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:23.308592 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.308649 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.308788 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.318015 10198 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:23.368443 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:23.381321 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.382233 10126 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:23.365413 10048 rpcz_store.cc:276] 0709 13:18:22.081715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.683873 (+602158us) service_pool.cc:224] Handling call
0709 13:18:23.293914 (+610041us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b54001b8d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:23.294066 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63903104}
W20260709 13:18:23.366366 10056 rpcz_store.cc:269] 0709 13:18:21.654474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.706839 (+ 52365us) service_pool.cc:224] Handling call
0709 13:18:23.300036 (+1593197us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54001b7454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:23.300165 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299260800}
I20260709 13:18:23.364301 10051 rpcz_store.cc:276] 0709 13:18:21.356250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.595099 (+238849us) service_pool.cc:224] Handling call
0709 13:18:22.677002 (+1081903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.770589 (+ 93587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.291927 (+521338us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001b7950. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:23.292174 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2486996224}
I20260709 13:18:23.384805 10201 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:18:23.387167 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 301309us
I20260709 13:18:23.389582 10050 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:23.362001 10202 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:23.391075 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1871) took 1858 ms. Trace:
I20260709 13:18:23.391700 10050 rpcz_store.cc:276] 0709 13:18:21.532425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.388784 (+1856359us) service_pool.cc:224] Handling call
0709 13:18:23.389186 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.390806 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.391011 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.265829 14458 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:23.327270 14498 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:23.398744 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 475) took 1648 ms. Trace:
I20260709 13:18:23.430384 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.436748 10050 rpcz_store.cc:276] 0709 13:18:21.750353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.397654 (+1647301us) service_pool.cc:224] Handling call
0709 13:18:23.398091 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.398320 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.398361 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.398412 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:23.398458 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.398519 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.398675 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.438063 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.438987 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 479) took 1565 ms. Trace:
I20260709 13:18:23.439342 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.393515  9951 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 8 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:23.258069 14502 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.325620 14021 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.449995 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 474) took 1700 ms. Trace:
I20260709 13:18:23.456740 10056 rpcz_store.cc:276] 0709 13:18:21.750038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.383810 (+1633772us) service_pool.cc:224] Handling call
0709 13:18:23.449401 (+ 65591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.449613 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.449644 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.449694 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:23.449731 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.449791 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.449924 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.439504 10050 rpcz_store.cc:276] 0709 13:18:21.873492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.438119 (+1564627us) service_pool.cc:224] Handling call
0709 13:18:23.438449 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.438643 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.438675 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.438723 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:23.438756 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.438810 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.438925 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.257575 14405 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.433836 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 314558us
I20260709 13:18:23.290763 13776 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:23.278611 13855 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:23.302980 14429 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.330957 14426 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:23.512979 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 480) took 1550 ms. Trace:
I20260709 13:18:23.513460 10056 rpcz_store.cc:276] 0709 13:18:21.962965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.509139 (+1546174us) service_pool.cc:224] Handling call
0709 13:18:23.512392 (+  3253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.512595 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.512622 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.512669 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:23.512715 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.512777 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.512911 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.272778 14500 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.469946 10126 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:23.471203 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 476) took 1720 ms. Trace:
W20260709 13:18:23.520813 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85823.7us
I20260709 13:18:23.521314 10049 rpcz_store.cc:276] 0709 13:18:21.750633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.429232 (+1678599us) service_pool.cc:224] Handling call
0709 13:18:23.470374 (+ 41142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.470711 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.470765 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.470836 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:23.470897 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.470980 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.471140 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.471057 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:23.527422  9951 catalog_manager.cc:5697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.480659 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:23.576448 10128 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:23.494786 14503 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:18:23.579777 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:23.580713 10054 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:23.581394 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1883) took 1943 ms (client timeout 1679 ms). Trace:
W20260709 13:18:23.581856 10054 rpcz_store.cc:269] 0709 13:18:21.639967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.664655 (+ 24688us) service_pool.cc:224] Handling call
0709 13:18:23.579566 (+1914911us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:23.581300 (+  1734us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16821632}
I20260709 13:18:23.506057 14510 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.496592 14506 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:18:23.497051 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.425457 14483 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:23.505770 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:23.505921 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.611784 10049 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.617283 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1929) took 1534 ms. Trace:
W20260709 13:18:23.617358 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111574us
I20260709 13:18:23.617878 10049 rpcz_store.cc:276] 0709 13:18:22.082641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.566524 (+1483883us) service_pool.cc:224] Handling call
0709 13:18:23.611548 (+ 45024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.616947 (+  5399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.617200 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.618366 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.619863 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.275063 14457 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.280606 14407 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:23.517345 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 482) took 1551 ms. Trace:
I20260709 13:18:23.264897 14356 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.322538 14462 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:23.628526 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:23.629213 10198 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:23.522754 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:23.629544 10198 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:23.583033 14417 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.510579 14405 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.584260 14426 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.620857 10056 rpcz_store.cc:276] 0709 13:18:21.965848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.514493 (+1548645us) service_pool.cc:224] Handling call
0709 13:18:23.514859 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.515059 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.515084 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.516662 (+  1578us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:23.516808 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.516938 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.517213 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.625191 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:23.568748 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.595208 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89278.5us
W20260709 13:18:23.633309 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 110660us
I20260709 13:18:23.642419 10049 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:23.644912 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 484) took 1170 ms. Trace:
I20260709 13:18:23.647172 10049 rpcz_store.cc:276] 0709 13:18:22.474426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.628152 (+1153726us) service_pool.cc:224] Handling call
0709 13:18:23.642052 (+ 13900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.644641 (+  2589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.644851 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.653458 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:23.645835 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.616328 14483 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:23.641984 10198 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:23.658380 10198 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:23.644982 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 206316us
I20260709 13:18:23.655215 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.627864 14429 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:23.658206 10127 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.802s	user 0.017s	sys 0.001s
I20260709 13:18:23.662272 10056 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:23.652208 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2092) took 2170 ms. Trace:
W20260709 13:18:23.666203 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.663496 10127 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:23.663959 10128 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:23.674118 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.674916 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:23.675227 10130 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:23.636482 14511 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.672173 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:23.636621 14508 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:23.684894 10047 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.004s	sys 0.012s
W20260709 13:18:23.689689 10047 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.013s
I20260709 13:18:23.685047 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:23.684690 10198 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:23.665503 10052 rpcz_store.cc:276] 0709 13:18:21.481862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.084219 (+1602357us) service_pool.cc:224] Handling call
0709 13:18:23.084706 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.085014 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:23.397818 (+312804us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.651665 (+253847us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.651767 (+   102us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:23.651863 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.651992 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.652144 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18133,"raft.queue_time_us":83962,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":46147,"thread_start_us":527509,"threads_started":3,"wal-append.queue_time_us":111590}
I20260709 13:18:23.657560 14500 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.697273 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:23.663184 14483 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:23.726816 14517 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:23.728655 14495 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:23.665652 14095 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.656898 14415 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:18:23.662325 14518 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.688897 14508 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.671643  9951 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 9, leader changed from <none> to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:23.650086 14426 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:23.714823 14381 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:23.731022 14429 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.729624 14434 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:23.777024 10096 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.189s (SENT)
I20260709 13:18:23.774482  9951 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.778234 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:23.748482 14500 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:23.754473 14508 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:23.637158 14510 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.741930 14492 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:23.799031  9951 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 8 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:18:23.799019 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 114050us
I20260709 13:18:23.802075 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.808245 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2091) took 2326 ms. Trace:
I20260709 13:18:23.806945 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.811880 14532 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.812901 14532 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.816777 14532 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:23.815639 14540 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:23.826121 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.696861 14511 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:23.798305 14484 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.005s
I20260709 13:18:23.749861 14095 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.837927 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.794217 14493 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:23.793538 14407 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:23.801406 14500 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:23.834851 14411 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.028s	user 0.005s	sys 0.010s
I20260709 13:18:23.827008 14405 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:23.836433 14495 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:23.845631  9951 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 10, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:23.844308 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104497us
I20260709 13:18:23.859948 10201 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:23.825276 14429 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.841046 14532 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:23.865080 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.830858 14434 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:18:23.848286 14518 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:23.844669 14407 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.848595 14484 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:23.877203  9951 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 9, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:23.865790 14527 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.914853 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 431) took 1803 ms (client timeout 1696 ms). Trace:
I20260709 13:18:23.904809 14407 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:23.915681 14531 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:18:23.915508 10125 rpcz_store.cc:269] 0709 13:18:22.111543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.111697 (+   154us) service_pool.cc:224] Handling call
0709 13:18:23.914539 (+1802842us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400321754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:23.914715 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538495360}
I20260709 13:18:23.916231 14356 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:23.918318 14504 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.323s	user 0.002s	sys 0.000s
I20260709 13:18:23.922585 14426 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:23.924870 10122 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:23.875929 14429 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:23.928584 14434 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.929468 14434 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:23.929597 14072 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:23.929991 14429 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:23.930734 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65790us
W20260709 13:18:23.913471 14530 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.067s	user 0.002s	sys 0.000s
I20260709 13:18:23.932979 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:23.913858 14535 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.929845 14434 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:23.912918 14095 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:23.947366 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 223765us
W20260709 13:18:23.948268 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 480043us
W20260709 13:18:23.948789 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85168.8us
I20260709 13:18:23.953613 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:23.955224 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:23.930199 14527 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.967767 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.968889 10122 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:23.970422 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:23.971362 10125 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:23.913998 14521 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:23.978721 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.980168 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.981768 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:23.985896 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.995874 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.998243 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.999459 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.001132 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.914796 14529 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:23.998971 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46581.7us
I20260709 13:18:23.947088 14381 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:24.003919 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.004679 10130 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:23.966238 14406 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.144s	user 0.011s	sys 0.007s
I20260709 13:18:24.011901 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.013505 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.014523 10130 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:24.015878 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:24.016549 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.006796 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.017932 10127 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:24.017988 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:23.880693 14532 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:24.008091 10122 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:24.018862 10130 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:23.956794 14518 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:24.021643 10126 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:24.021647 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2095) took 2317 ms. Trace:
I20260709 13:18:23.959388 14415 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:18:23.967010 14426 raft_consensus.cc:2764] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
W20260709 13:18:24.070310 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.025388 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:24.026875 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 249834us
I20260709 13:18:24.027944  9951 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 7, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:24.072865 10096 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.217s (SENT)
I20260709 13:18:24.032467 10048 rpcz_store.cc:276] 0709 13:18:21.704186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.383183 (+1678997us) service_pool.cc:224] Handling call
0709 13:18:23.448577 (+ 65394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.566364 (+117787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.566401 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:23.623550 (+ 57149us) log.cc:844] Serialized 26 byte log entry
0709 13:18:23.692941 (+ 69391us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:23.692999 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.693095 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.021337 (+328242us) raft_consensus.cc:1685] finished
0709 13:18:24.021374 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.021570 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16485,"raft.queue_time_us":135358,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":30218,"thread_start_us":155922,"threads_started":2,"wal-append.queue_time_us":117217}
I20260709 13:18:24.040361 10125 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:24.042290 14468 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:18:24.043488 14536 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.073853  9951 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:24.050148 14434 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:23.987282 14531 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:18:24.055430 14541 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:24.062150 14527 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.064961 14381 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.067889 14546 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:24.006264 14356 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.075539 10126 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:24.073354 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.089584 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.035759 14495 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.144371 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119077us
I20260709 13:18:24.144563 14495 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.148082 14513 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:24.148814 14513 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.149371 14513 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:24.079311 14021 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.005s
I20260709 13:18:24.155787 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:24.102154 14532 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.088608 14541 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.103303 14137 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.158015 14541 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:24.159008 14511 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.103770 14510 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.104739 14434 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:24.107793 10096 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 4.187s (SENT)
I20260709 13:18:24.161659 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.108362 14527 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:24.163805 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.165267 14137 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.165871 14137 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.113131 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.165275 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1939) took 1160 ms. Trace:
I20260709 13:18:23.808794 10047 rpcz_store.cc:276] 0709 13:18:21.481384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.056128 (+1574744us) service_pool.cc:224] Handling call
0709 13:18:23.056423 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.056661 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:23.696012 (+639351us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.807807 (+111795us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.807867 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:23.807928 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.808006 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.808169 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":609209,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":207,"thread_start_us":698006,"threads_started":2}
I20260709 13:18:24.167876 10124 rpcz_store.cc:276] 0709 13:18:23.005183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.005315 (+   132us) service_pool.cc:224] Handling call
0709 13:18:24.121059 (+1115744us) spinlock_profiling.cc:243] Waited 991 us on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:24.121328 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.163390 (+ 42062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.163445 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.163486 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:24.163524 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.165044 (+  1520us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b540015ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:24.165091 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.165210 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98402432}
I20260709 13:18:24.169348 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.170270 10047 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:24.172937 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.173257 14426 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.173861 10051 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:24.174103 14426 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.111270 14533 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.146s
I20260709 13:18:24.019668 14529 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.128527 14515 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.259s	user 0.000s	sys 0.002s
I20260709 13:18:24.132421 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.134115 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2085) took 2777 ms. Trace:
I20260709 13:18:24.135888 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:24.191406 10864 connection.cc:256] RPC call timeout handler was delayed by -1.4288886450001428s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 13:18:24.198107 10056 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.195536 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:24.200289 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.201594 10124 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:24.204356 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:24.205590 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97832.7us
I20260709 13:18:24.209261 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.196036 10055 rpcz_store.cc:276] 0709 13:18:21.356914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.958794 (+601880us) service_pool.cc:224] Handling call
0709 13:18:23.300896 (+1342102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.301231 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.301262 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.301326 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:23.301384 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.301466 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.133727 (+832261us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001b7450. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:24.133961 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085656448}
I20260709 13:18:24.137599 14462 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.072990 14500 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.140439 14538 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:18:24.085842 14536 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:24.083017 14483 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.142311 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.143507 14356 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.163730 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.167495 10049 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:24.185345 14433 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.237419 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:24.244768 10048 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:24.185974 14527 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:24.204417 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.225414 10124 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:18:24.252936 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48581.9us
W20260709 13:18:24.253589 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50815.4us
I20260709 13:18:24.253839 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:24.155861 14428 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.002s
I20260709 13:18:24.247614  9951 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 4 to 6, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:24.257584 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:24.259414 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.173660 14021 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.260190 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.263315 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:24.264499 10051 raft_consensus.cc:2368] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:24.267035 10124 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:24.267445 10124 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:24.236161 13776 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.208174 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:24.286454 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.224934 14502 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:24.121397 14508 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.297292 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89243us
I20260709 13:18:24.297629 10128 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:24.299019 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.225644 14468 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.275426 10122 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:24.299928 10196 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:24.275725 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.276942 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.277796 10129 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:24.249758 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:24.216018 10864 connection.cc:256] RPC call timeout handler was delayed by -1.0813186980001319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
I20260709 13:18:24.284571 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.284806 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.303639 10052 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:24.303835 10053 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:24.304447 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.304672 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.167495 14383 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.306211 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.306967 14383 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:24.307425 10200 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:24.307394 14383 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.307940 14383 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.309465 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.310271 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.303835 10056 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:24.289062 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:24.289857 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.239615 14072 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.214959 14521 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:24.284340  9951 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 9, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:24.311427 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.313856 10200 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:24.311925 10201 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:18:24.295743 14545 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.316862 14545 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.317408 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.318056 10201 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:24.312587 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:24.306430 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2099) took 1280 ms. Trace:
I20260709 13:18:24.319682 10197 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10. Re-sending same reply.
I20260709 13:18:24.246872 14537 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.312912 14521 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:24.312942 14072 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:24.319954 10052 rpcz_store.cc:276] 0709 13:18:23.025940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.305572 (+1279632us) service_pool.cc:224] Handling call
0709 13:18:24.305888 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.306084 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.306109 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.306153 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:24.306197 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.306252 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.306370 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.247355 14495 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.321517 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.322829 10202 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:24.323431 14529 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.323525 14502 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:24.339258 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79209.7us
I20260709 13:18:24.313755  9951 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 3 to 8, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 8 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:24.350587 14071 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:24.274598 14204 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:18:24.351274 14071 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.332417 14378 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.351801 14071 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:24.322024 13776 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:24.352315 14378 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.335762 14269 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.355407 14137 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.259055 14433 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.355824 14269 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.358073 14433 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.327411 14527 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.338392 14511 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.333467 14521 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:24.368963 10047 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:24.325225 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:24.266754 14429 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:24.369936 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2102) took 1316 ms. Trace:
I20260709 13:18:24.372372 10047 rpcz_store.cc:276] 0709 13:18:23.053975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.325074 (+1271099us) service_pool.cc:224] Handling call
0709 13:18:24.342357 (+ 17283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.369611 (+ 27254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.369868 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.373005 10205 raft_consensus.cc:2410] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:18:24.268803 14428 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.324110 14468 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:24.344148 14458 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.869s	user 0.014s	sys 0.005s
I20260709 13:18:24.376953  9951 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 1 to 7, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:24.389106 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63953.6us
I20260709 13:18:24.360078 14511 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.396471 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.397039 14511 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.397742 14350 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:24.398602 14350 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:24.399004 14350 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.364707 14545 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.339215 14502 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:24.368417 14495 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:24.411298 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47832.9us
I20260709 13:18:24.402038 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2983) took 1171 ms. Trace:
I20260709 13:18:24.396487  9951 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 6, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:24.413631 10203 rpcz_store.cc:276] 0709 13:18:23.230803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.230942 (+   139us) service_pool.cc:224] Handling call
0709 13:18:23.257307 (+ 26365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.361222 (+103915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.361272 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.361342 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:23.361392 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.401669 (+1040277us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:24.401743 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.401946 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238224256}
I20260709 13:18:24.404320 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:24.407655 10054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.008s	sys 0.030s
I20260709 13:18:24.372103 14429 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:24.416915 10054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.009s	sys 0.031s
I20260709 13:18:24.405274 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:24.439997 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52392.9us
I20260709 13:18:24.445153 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.447558 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.417973 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 483) took 2326 ms. Trace:
I20260709 13:18:24.423955 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.453675 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:24.406797 14546 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:24.414449  9951 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 8, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:24.332856 14537 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:24.417984 14417 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.781s	user 0.015s	sys 0.000s
I20260709 13:18:24.462224 14468 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.461385 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:24.485648 10197 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:24.412448 14502 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.449180 10054 rpcz_store.cc:276] 0709 13:18:22.091814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.582985 (+1491171us) service_pool.cc:224] Handling call
0709 13:18:23.592994 (+ 10009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.593288 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:18:23.780216 (+186928us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.417470 (+637254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.417547 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:24.417614 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.417742 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.417889 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44824,"prepare.queue_time_us":188804,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":20680,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":9156,"thread_start_us":806723,"threads_started":3,"wal-append.queue_time_us":161090}
I20260709 13:18:24.439407 14429 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.443912 14531 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:24.386570 14529 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:18:24.445163 14457 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:24.385881 14532 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:18:24.489362 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.527335 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:24.446219 14495 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:24.451722 14545 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:24.490729 10202 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
W20260709 13:18:24.491317 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48714.5us
I20260709 13:18:24.464496 14510 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:24.499182 10196 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:24.502949 14269 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:24.504777 14536 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.383859 14547 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 13:18:24.465359 14462 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:24.545974 14407 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.003s
I20260709 13:18:24.530164 10048 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:24.522518 13776 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.542461 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.563555 10131 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:24.564783 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.565565 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:24.566540 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.495023 14524 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:24.521229 14469 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:24.552241 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 302731us
I20260709 13:18:24.511545 14550 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:24.549609 14545 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:24.555678 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.807s (SENT) [suppressed 37 similar messages]
W20260709 13:18:24.578454 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 320943us
I20260709 13:18:24.401109 14458 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:24.448132 14202 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:18:24.567847 10131 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:24.588294 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.500118 14446 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:18:24.572331 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.591315 14462 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.592208 14462 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.572781 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 500678us
I20260709 13:18:24.548691 14509 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.543552 14457 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:24.529160 14426 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.583561 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.550943 14072 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.556725 14021 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.592453 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.554430 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.609903 10130 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:24.562100 14356 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.606755 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:24.572293 14532 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.580391 14545 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.577903 14550 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.614842 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.643167 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.651815 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.647202 14426 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:24.656327 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.572366 14407 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.686237 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.613552 14523 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:24.581040 14469 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.700477 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 112268us
I20260709 13:18:24.613998 14416 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:24.683248 10131 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:24.606887 14508 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:24.690825 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.704670 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:24.702749 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:24.723384 10055 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:24.720180 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:24.655835 14379 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:24.650661 14550 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:24.684453 13961 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:24.730993 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.683506 14428 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.734488 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:24.736090 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 291025us
I20260709 13:18:24.737106 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.700995 14501 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:24.740414 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 135462us
I20260709 13:18:24.707844 14523 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.604622 14509 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:24.745550 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 189900us
I20260709 13:18:24.740763 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2101) took 1694 ms. Trace:
I20260709 13:18:24.706646 14416 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:24.695442 14551 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:24.721872 14547 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:24.751518 10096 leader_election.cc:400] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:24.750399 10053 rpcz_store.cc:276] 0709 13:18:23.045961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.322318 (+1276357us) service_pool.cc:224] Handling call
0709 13:18:24.339464 (+ 17146us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.339750 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:24.557444 (+217694us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.740380 (+182936us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.740449 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:24.740519 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.740590 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.740702 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65860,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":548,"thread_start_us":378966,"threads_started":2}
I20260709 13:18:24.723085 14446 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:24.749135 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 2996) took 1449 ms. Trace:
I20260709 13:18:24.650944 14532 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.768544 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:24.741892 14524 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:24.774971 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.745657 14379 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:24.748196 14523 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:24.771443 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.744313 13961 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:24.780433 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:24.780646 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:24.781466 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.736393 14337 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:24.773836 10204 rpcz_store.cc:276] 0709 13:18:23.299897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.713934 (+414037us) service_pool.cc:224] Handling call
0709 13:18:23.751454 (+ 37520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.751957 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.752003 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.115297 (+363294us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:24.115773 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.373951 (+258178us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:24.664853 (+290902us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.736743 (+ 71890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.736863 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.736924 (+    61us) raft_consensus.cc:1685] finished
0709 13:18:24.748889 (+ 11965us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.749050 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82491,"raft.queue_time_us":124877,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":32450,"spinlock_wait_cycles":8771840,"thread_start_us":517248,"threads_started":2,"wal-append.queue_time_us":334404}
I20260709 13:18:24.800244 10892 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:24.754560 14501 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:24.625660 14495 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.802824 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.759773 14551 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:24.785851 14356 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:24.759411 14547 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:18:24.813946 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113300us
I20260709 13:18:24.773759 14446 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.777542 14222 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:24.779435 14021 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:24.786928 14537 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:24.766914 14533 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:18:24.809403 14563 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:18:24.833431 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:24.838996 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:24.740449 14457 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:24.937656 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97412.9us
I20260709 13:18:24.827823 14338 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:24.955041 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 218017us
W20260709 13:18:24.956043 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 175524us
W20260709 13:18:24.956286 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 122974us
I20260709 13:18:24.799576 14383 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.957607 14338 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:24.962507 14569 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:18:24.962589 14338 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:24.962965 14558 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:18:24.964951 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:24.966144 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 127217us
I20260709 13:18:24.967589 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:24.968245 10055 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:18:24.971299 10053 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:24.822019 14381 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:18:24.836323 14021 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:24.975787 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.932762 14529 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:24.828656 14547 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:24.836629 14446 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:24.934820 14508 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:24.982102 10125 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:25.051182 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.051542 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:24.985235 14567 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:18:24.965144 14458 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:24.985560 14555 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:18:24.988533 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.979007 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:25.064301 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85383.7us
I20260709 13:18:24.826747 14551 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.029670 10900 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:24.985905 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.979027 10892 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:24.958037 14562 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:25.036310 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.995371 14462 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.030680 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:24.976737 14420 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:25.067596 14561 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.444s	user 0.000s	sys 0.002s
I20260709 13:18:25.037241 10055 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:24.835283 14416 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:24.994961 14532 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:24.996871 14021 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:25.073318 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:24.988286 14383 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.075103 10122 raft_consensus.cc:2376] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12. Re-sending same reply.
I20260709 13:18:25.079157 14537 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:18:25.080500 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.076418 14420 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.100065 14420 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.083665 14575 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:18:25.046790 14508 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.115793 14508 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:25.117931 14407 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:25.118747 14527 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.760s	user 0.006s	sys 0.004s
I20260709 13:18:25.101289 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.051227 14576 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:18:25.102968 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1951) took 1428 ms. Trace:
W20260709 13:18:25.110422 14071 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.002s
I20260709 13:18:25.099699 14536 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:25.122560 10128 rpcz_store.cc:276] 0709 13:18:23.674478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.676118 (+  1640us) service_pool.cc:224] Handling call
0709 13:18:23.676375 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.298149 (+621774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.102761 (+804612us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:25.102913 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1428273536}
I20260709 13:18:25.055662 14501 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.074355 13855 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:25.068223 14568 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:18:25.124585 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.037079 14379 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.125113 14536 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:25.131922 14536 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.133669 14536 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:25.134605 14521 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.007s
W20260709 13:18:25.135943 14434 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.975s	user 0.014s	sys 0.002s
I20260709 13:18:25.138811 10049 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:18:25.139448 10049 pending_rounds.cc:85] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:18:25.139756 10049 pending_rounds.cc:107] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:18:25.139999 10049 raft_consensus.cc:2889] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.091620 14469 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.127635 10122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:52258
I20260709 13:18:25.036276 13961 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.146334 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.148504 10122 raft_consensus.cc:606] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:25.149145 10122 raft_consensus.cc:614] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:25.149919 10125 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:25.152740 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:25.146843 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.154672 10128 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.141975 14527 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:25.169932 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.128784 13855 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:25.164143 14570 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:18:25.175583 14429 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.667s	user 0.013s	sys 0.000s
I20260709 13:18:25.184473  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:25.195922 10127 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:25.167367 14575 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:25.141534 14521 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.130267 14071 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:25.184841 14541 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.026s	user 0.010s	sys 0.007s
I20260709 13:18:25.197487 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3052) took 1004 ms. Trace:
I20260709 13:18:25.194639  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:25.202118 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.183357 13855 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.205984 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.206789 10895 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:25.204077 10123 rpcz_store.cc:276] 0709 13:18:24.193364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.193522 (+   158us) service_pool.cc:224] Handling call
0709 13:18:24.193990 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:24.194471 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.194501 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:24.236829 (+ 42328us) log.cc:844] Serialized 52 byte log entry
0709 13:18:24.407004 (+170175us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:24.407052 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.407172 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.197233 (+790061us) raft_consensus.cc:1685] finished
0709 13:18:25.197255 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.197419 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6422,"raft.queue_time_us":148,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":40438,"thread_start_us":110875,"threads_started":1,"wal-append.queue_time_us":283551}
W20260709 13:18:25.187809 14546 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.022s	sys 0.035s
W20260709 13:18:25.227145 14546 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.024s	sys 0.036s
I20260709 13:18:25.173961 14338 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.238121 10052 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:25.239836 10128 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:25.241163 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.245450 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.247015 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.247525 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:25.248009 14550 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.003s
I20260709 13:18:25.248584 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.248515 14550 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.249030 14071 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.207554 14575 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:25.232753 14546 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:25.249764 10893 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:25.243285 14578 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:18:25.200343 14429 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.210002 14541 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.289716 10051 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.292315 10054 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:25.273905 14378 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:25.299209 14575 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.300679 14429 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.303041 14338 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:25.305627 14350 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.906s	user 0.006s	sys 0.008s
I20260709 13:18:25.306193 14350 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.313566 14541 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.319274 14531 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:25.320134 14531 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:25.299556 14523 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.343981 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1868) took 1035 ms. Trace:
I20260709 13:18:25.344553 10205 rpcz_store.cc:276] 0709 13:18:24.308903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.309073 (+   170us) service_pool.cc:224] Handling call
0709 13:18:25.343661 (+1034588us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:25.343888 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143302528}
W20260709 13:18:25.351827 14513 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.202s	user 0.015s	sys 0.005s
I20260709 13:18:25.376803 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.411010 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:25.412469 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.414885 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:25.338028 14546 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.356756 14513 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.419799 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:25.404089 14521 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.420929 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:25.435269 14338 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:25.441401 14021 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.442557 14523 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.470144 14379 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.471549 14513 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:25.470847 14379 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.472950 14483 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.474449 14379 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.475333 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.476115 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:25.477228 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.477650 14379 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.477905 10894 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:25.478318 14379 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.481902 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:25.477907 10055 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:25.486030 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.457473 14508 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.460107 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1872) took 1036 ms. Trace:
I20260709 13:18:25.489185 10202 rpcz_store.cc:276] 0709 13:18:24.423518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.423668 (+   150us) service_pool.cc:224] Handling call
0709 13:18:25.459863 (+1036195us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b54002175d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:25.460023 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71624448}
I20260709 13:18:25.460669 14521 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:25.462091 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.428093 14545 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.743s	user 0.009s	sys 0.004s
I20260709 13:18:25.423206 14202 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.465227 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 488) took 1512 ms. Trace:
I20260709 13:18:25.500423 10052 rpcz_store.cc:276] 0709 13:18:23.953033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.342536 (+389503us) service_pool.cc:224] Handling call
0709 13:18:24.342848 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.238679 (+895831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.464937 (+226258us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b540010c2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:25.465146 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052163456}
I20260709 13:18:25.502326 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.457365 14509 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:25.487308 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.428403 14425 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:25.431571 14269 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:25.507041 14545 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.531394 14523 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.532161 14523 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:25.532749 14572 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.474s	user 0.003s	sys 0.000s
I20260709 13:18:25.534572 14521 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.536108 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 1949) took 1369 ms. Trace:
I20260709 13:18:25.536585 10051 rpcz_store.cc:276] 0709 13:18:24.166375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.276666 (+110291us) service_pool.cc:224] Handling call
0709 13:18:25.535874 (+1259208us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54000d8c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:25.536037 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2268509952}
I20260709 13:18:25.537561 14521 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.538691 14420 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.538762 14483 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.540278 14483 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.541492 14420 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.541918 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2100) took 2515 ms. Trace:
I20260709 13:18:25.508394 10124 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.538038 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.542876 10124 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:25.521373 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.543989 10055 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:25.545483 10051 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.542476 10056 rpcz_store.cc:276] 0709 13:18:23.026207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.311566 (+1285359us) service_pool.cc:224] Handling call
0709 13:18:24.322088 (+ 10522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.322332 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:24.335242 (+ 12910us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.529388 (+194146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.529455 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:24.529545 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.529637 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.541681 (+1012044us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b54001b8d50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:25.541835 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13567,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":5524,"raft.queue_time_us":143,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6933,"spinlock_wait_cycles":23267584,"thread_start_us":193650,"threads_started":1,"wal-append.queue_time_us":322863}
I20260709 13:18:25.487890 14235 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:25.484795 14457 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:25.450693 14502 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.940s	user 0.015s	sys 0.000s
I20260709 13:18:25.466250 14501 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.504508 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:25.506956 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 75208us
I20260709 13:18:25.509632 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2112) took 1450 ms. Trace:
I20260709 13:18:25.561571 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.567525 10054 rpcz_store.cc:276] 0709 13:18:24.070997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.518573 (+447576us) service_pool.cc:224] Handling call
0709 13:18:24.548881 (+ 30308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.297077 (+748196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.509354 (+212277us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b5400136854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:25.509558 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1709122304}
I20260709 13:18:25.449949 14350 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.548264 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:25.569729 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:25.566370 10052 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:25.571468 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:25.532918 14545 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.562044 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1965) took 1054 ms. Trace:
I20260709 13:18:25.495005 14137 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.574894 10129 rpcz_store.cc:276] 0709 13:18:24.508041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.511447 (+  3406us) service_pool.cc:224] Handling call
0709 13:18:24.544321 (+ 32874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.546289 (+  1968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.546332 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.547021 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.760525 (+213504us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:24.760579 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.762194 (+  1615us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.532029 (+769835us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:25.544922 (+ 12893us) raft_consensus.cc:1685] finished
0709 13:18:25.544948 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.561941 (+ 16993us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1499,"raft.queue_time_us":2579,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":14256,"spinlock_wait_cycles":7971712,"thread_start_us":185353,"threads_started":1,"wal-append.queue_time_us":282742}
W20260709 13:18:25.524004 14584 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:18:25.529243 14423 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:25.591594 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.527487 13961 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.599462 10056 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:25.494298 14551 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.566277 14509 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.562510 14357 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.605116 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:25.609098 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.612607 10054 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.612692 14379 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.613663 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.613433 14509 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.581563 14466 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:25.615569 14466 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.616035 14466 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.616204 14509 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.610090 14425 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:25.580309 14501 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.619663 14468 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:25.615012 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.621716 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:25.622251 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.624997 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.516921 14202 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.512785 13776 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:18:25.580538 14511 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.608043 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.626374 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:25.634690 10203 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:25.634931 10056 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:25.603076 14523 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.601650 14423 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:25.628818 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.605736 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:25.641954 10051 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.629870 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:25.630203 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:25.633662 10196 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:25.627928 14509 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:25.649256 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.611570 14508 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.636082 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:25.670701 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65415.2us
I20260709 13:18:25.671304 10197 raft_consensus.cc:2368] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13.
I20260709 13:18:25.672753 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:25.637902 10124 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:25.638304 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.673903 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 446) took 1466 ms. Trace:
I20260709 13:18:25.640740 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:25.647886 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.612210 14483 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.584619 14502 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.675798 14545 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:25.676298 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:25.680996 10054 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.674288 10124 rpcz_store.cc:276] 0709 13:18:24.207764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.266455 (+ 58691us) service_pool.cc:224] Handling call
0709 13:18:24.266883 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.518565 (+251682us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b4c001b3180. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2762d0a20 00007fa2762e1a29 00007fa2762a7644 00007fa2762cef2d 00007fa2762ceb93 00007fa277fff18b 00007fa277fffcb5 00007fa277ffaf30 00007fa277ffc7de 00007fa278006b49 00007fa278007416
0709 13:18:25.508035 (+989470us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5400358020. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f8206f 00007fa277ffaf60 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62
0709 13:18:25.673587 (+165552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.673840 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65258112}
I20260709 13:18:25.606379 13961 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.689647 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.691244 10056 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:25.706303 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:25.706470 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.625280 14357 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.710266 10054 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:25.603874 14137 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.682693 14350 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:25.680521 14510 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.740737 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.750119 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.751086 10053 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:25.747903 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:25.753645 10055 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:25.754370 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.764793 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.765883 10054 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.713431 14483 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:25.762995 10056 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:25.726426 10051 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:25.755702 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.756227 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.776269 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.778316 10051 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:25.753674 14021 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.752774 14350 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.753743 14501 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.761986 14545 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:25.800670 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.805609 10056 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:25.818773 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:25.767803 14510 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.803828 14350 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:25.764063 14423 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:25.831058 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 124674us
I20260709 13:18:25.831597 10197 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:25.833654 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.817227 14379 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.815068 14545 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:25.821398 14541 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:25.823230 14446 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.756s	user 0.015s	sys 0.000s
I20260709 13:18:25.837772 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:25.842181 13574 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:25.839057 13776 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:18:25.865301 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:25.842733 14592 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:18:25.838761 14379 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.843909 14502 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:25.826563 14501 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:25.847962 14591 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:18:25.851009 14536 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.853490 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.866184 14021 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.860813 14423 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:25.863834 13855 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.007s
I20260709 13:18:25.877053 10197 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:25.896996 10056 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:25.859445 14202 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:25.881822 10128 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:25.890686 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.900454 10123 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:25.890990 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:25.902040 10130 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:25.892391 10122 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:25.894254 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.896200 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:25.907337 10125 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:25.903607 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:25.905418 10056 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:25.908756 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.909861 10056 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:25.912289 14531 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.007s
I20260709 13:18:25.913978 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.914760 10131 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:18:25.873034 13574 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:25.917339 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:25.919044 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.875236 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:25.920349 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45263us
W20260709 13:18:25.921072 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.878993 14483 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:25.929669 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:25.930473 10130 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:25.936102 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:25.936291 10125 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:25.938128 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:25.906898 14547 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.005s
I20260709 13:18:25.942654 10122 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:25.948036 10123 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:25.952126 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:25.870409 14541 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:25.963023 10095 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.517s (SENT)
I20260709 13:18:25.963912 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:25.931442 14269 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:25.905484 13961 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:25.928998 14424 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:25.922561 14202 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:25.967880 14545 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.879819 14446 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.930168 14483 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:25.912895 14531 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:25.997363 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:25.998252 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:26.014739 10125 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:26.004048 10197 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:25.903337 14493 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:18:26.021284 10131 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:25.923655 14021 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:26.027930 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:25.915947 13855 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.914371 14536 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:25.982158 14357 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:26.036214 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113321us
I20260709 13:18:26.039665 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.066931 10205 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:25.906283 14501 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:18:26.065358 10130 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:25.990872 14137 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:26.071802 10129 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:25.954340 14547 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.030637 14483 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:25.995645 14597 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:26.073160 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.083653 10197 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:26.076043 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:26.085381 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.086920 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:26.094558 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.097800 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:26.087870 14597 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.104796 14597 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:26.075420 14521 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.104954 14137 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.152380 14446 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:26.153862 13855 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:26.082494 14593 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:18:26.102654 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 139688us
I20260709 13:18:26.116501 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:26.168347 10204 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:26.169160 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52789.9us
I20260709 13:18:26.151901 14137 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.064090 13574 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:26.164459 14580 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:18:26.081143 14535 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.096426 14202 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:26.190964 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:26.158058 14493 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:26.159602 14510 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.209017 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:26.106602 14536 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:26.178499 14446 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:26.219882 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 142509us
I20260709 13:18:26.127976 14425 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:18:26.166838 14521 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.208150 14202 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.208441 14493 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.208231 14523 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:26.215276 14379 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.236591 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:26.219551 13574 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:26.239552 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:26.221022 14510 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.208321 14535 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.336704 10204 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:26.340526 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:26.346282 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.343227 14601 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.348392 14601 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:26.349836 14350 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
I20260709 13:18:26.350343 14350 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.351418 10124 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:26.364251 14509 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.000s
I20260709 13:18:26.365034 14509 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:26.447213 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.283022 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:26.463027 10196 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:26.523190 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.284998 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2125) took 1061 ms. Trace:
I20260709 13:18:26.570915 10056 rpcz_store.cc:276] 0709 13:18:25.230692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.913594 (+682902us) service_pool.cc:224] Handling call
0709 13:18:25.914033 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.914365 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:26.190964 (+276599us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.284544 (+ 93580us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.284601 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:26.284661 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.284739 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.284912 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19061,"prepare.queue_time_us":326968,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":1023,"raft.run_wall_time_us":12758,"spinlock_wait_cycles":56814848,"thread_start_us":326064,"threads_started":2}
I20260709 13:18:26.223765 14605 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:26.578243 10202 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:26.582474 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1969) took 2073 ms. Trace:
I20260709 13:18:26.227623 14446 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:26.282416 14466 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.001s
I20260709 13:18:26.601473 10127 rpcz_store.cc:276] 0709 13:18:24.509395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.516450 (+  7055us) service_pool.cc:224] Handling call
0709 13:18:24.516813 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.196451 (+679638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.582058 (+1385607us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:26.582380 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561295616}
I20260709 13:18:26.227087 14337 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:26.228133 14493 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:26.316905 14596 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.247s	user 0.002s	sys 0.000s
W20260709 13:18:26.311897 14511 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.003s
I20260709 13:18:26.611181 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:26.656556 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:26.239949 14541 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.657244 10056 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:26.372879 14601 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.228009 14202 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:26.259591 14531 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:26.657352 10050 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 6.
I20260709 13:18:26.659834 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 499) took 1419 ms. Trace:
W20260709 13:18:26.641386 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 404839us
I20260709 13:18:26.662545 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2110) took 2604 ms. Trace:
I20260709 13:18:26.249231 14606 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:26.667251 10048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.018s	sys 0.014s
I20260709 13:18:26.261303 14137 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:26.671028 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.024s
I20260709 13:18:26.262869 14536 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:26.605705 14423 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.005s
W20260709 13:18:26.654772 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.677331 10056 rpcz_store.cc:276] 0709 13:18:25.240244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.658520 (+1418276us) service_pool.cc:224] Handling call
0709 13:18:26.658998 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.659280 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.659325 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.659392 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:26.659448 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.659536 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.659733 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.261852 14457 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.415s
I20260709 13:18:26.696220 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2131) took 1454 ms. Trace:
I20260709 13:18:26.697048 10050 rpcz_store.cc:276] 0709 13:18:25.242276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.676945 (+1434669us) service_pool.cc:224] Handling call
0709 13:18:26.695520 (+ 18575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.695741 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.695765 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.695818 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:26.695879 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.695961 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.696133 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:26.680648 10123 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.021s	sys 0.017s
I20260709 13:18:26.680583 10049 rpcz_store.cc:276] 0709 13:18:24.058543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.455369 (+396826us) service_pool.cc:224] Handling call
0709 13:18:24.484046 (+ 28677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.164032 (+679986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.661306 (+1497274us) spinlock_profiling.cc:243] Waited 654 ms on lock 0x7b540010a9d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:26.662401 (+  1095us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1504531328}
I20260709 13:18:26.576471 14521 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.677752 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:26.672482 14511 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:26.674628 14337 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:26.680161 10048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.019s	sys 0.014s
W20260709 13:18:26.695312 10131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.015s	sys 0.007s
I20260709 13:18:26.249526 14523 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:26.673461 14605 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.705451 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:26.727279 10196 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:26.688665 14536 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.598253 14510 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.682348 14601 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:26.678658 14541 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:26.685614 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.009s	sys 0.024s
W20260709 13:18:26.712064 10123 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.021s	sys 0.017s
I20260709 13:18:26.737084 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 495) took 1776 ms. Trace:
I20260709 13:18:26.737483 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:26.738363 10123 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:26.718087 10131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.015s	sys 0.007s
I20260709 13:18:26.724696 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2132) took 1469 ms. Trace:
I20260709 13:18:26.700157 14350 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:26.724696 10129 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.001s
I20260709 13:18:26.737510 10047 rpcz_store.cc:276] 0709 13:18:24.960310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.960470 (+   160us) service_pool.cc:224] Handling call
0709 13:18:24.968264 (+  7794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.174055 (+205791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.174087 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:25.174136 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:25.174171 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.736864 (+1562693us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540037b754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:26.736932 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.737031 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472710400}
I20260709 13:18:26.745213 10129 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:26.729012 10130 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:26.707794 14531 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:26.711377 14095 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:26.664250 14466 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.741443 10049 rpcz_store.cc:276] 0709 13:18:25.255781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.699530 (+1443749us) service_pool.cc:224] Handling call
0709 13:18:26.723977 (+ 24447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.724224 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.724262 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.724321 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:26.724374 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.724458 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.724602 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:26.725419 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.754293 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.688910 14606 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.733644 14536 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:26.732050 14511 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.763236 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:26.597865 14550 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:26.764544 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 187556us
W20260709 13:18:26.751441 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 506321us
I20260709 13:18:26.743479 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:26.778956 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2133) took 1506 ms. Trace:
I20260709 13:18:26.779490 10049 rpcz_store.cc:276] 0709 13:18:25.272111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.753285 (+1481174us) service_pool.cc:224] Handling call
0709 13:18:26.778412 (+ 25127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.778619 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.778644 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.778686 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:26.778716 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.778765 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.778878 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.784595 10127 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:26.789973 14466 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.790571 10047 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:26.791594 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 502) took 1506 ms. Trace:
I20260709 13:18:26.792076 10047 rpcz_store.cc:276] 0709 13:18:25.285265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.772331 (+1487066us) service_pool.cc:224] Handling call
0709 13:18:26.790109 (+ 17778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.791265 (+  1156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.791510 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.793597 10047 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:26.793222 14337 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.794409 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2136) took 1360 ms. Trace:
I20260709 13:18:26.795485 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:26.794826 10047 rpcz_store.cc:276] 0709 13:18:25.434366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.793146 (+1358780us) service_pool.cc:224] Handling call
0709 13:18:26.793423 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.794080 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.794342 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.925859 10047 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:26.927209 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2137) took 1490 ms. Trace:
I20260709 13:18:26.927760 10047 rpcz_store.cc:276] 0709 13:18:25.437219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.797581 (+1360362us) service_pool.cc:224] Handling call
0709 13:18:26.925542 (+127961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.926769 (+  1227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.927094 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.777105 14608 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:26.761018 13776 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:26.759775 14619 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.761446 14350 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.742630 14601 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.758402 14545 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:26.742945 14541 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.963243 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:26.756542 14531 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:26.964457 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:26.964983 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.993468 10205 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:26.965399 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:26.949194 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 182253us
I20260709 13:18:26.968529 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.969707 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2129) took 1733 ms. Trace:
I20260709 13:18:26.955427 10047 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:26.997887 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2138) took 1547 ms. Trace:
I20260709 13:18:26.998281 10047 rpcz_store.cc:276] 0709 13:18:25.450623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.954825 (+1504202us) service_pool.cc:224] Handling call
0709 13:18:26.955222 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.997573 (+ 42351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.997824 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:26.963496 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:26.736573 14605 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.988869 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:26.735349 14446 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:26.771675 14511 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.012494 10130 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.018105 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:27.009578 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:27.007009 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.020123 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:27.026510 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:27.004063 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.029330 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.031280 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:27.032603 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.033828 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:26.982968 14466 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.952869 14622 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.038281 10047 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:26.992143 14350 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:26.981724 14608 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:26.987882 14617 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.050801 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 503) took 1578 ms. Trace:
I20260709 13:18:27.080816 10047 rpcz_store.cc:276] 0709 13:18:25.472565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.037294 (+1564729us) service_pool.cc:224] Handling call
0709 13:18:27.038009 (+   715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.050482 (+ 12473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.050742 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:27.042083 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:27.047057 10123 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.049273 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 454) took 1122 ms. Trace:
I20260709 13:18:27.043800 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:27.051882 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.048880 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:27.051853 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:26.990849 14625 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:27.046790 14614 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.261s	user 0.000s	sys 0.000s
W20260709 13:18:27.078254 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 92576.3us
W20260709 13:18:27.088524 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.088729 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46740.1us
I20260709 13:18:27.087183 10122 rpcz_store.cc:276] 0709 13:18:25.927381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.927586 (+   205us) service_pool.cc:224] Handling call
0709 13:18:25.942356 (+ 14770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.955256 (+ 12900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.049011 (+1093755us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b540014c650. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:27.049185 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32589184}
I20260709 13:18:27.095199 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:27.096460 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:27.043253 14415 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:27.040277 14535 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:26.986436 13776 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.075266 14521 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:27.046159 14531 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:27.155107 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 173434us
W20260709 13:18:27.156953 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 61884.2us
I20260709 13:18:27.040982 14629 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:26.985605 14619 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.005650 14545 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.159626 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:27.172132 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79530.7us
I20260709 13:18:27.172870 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:27.198822 10196 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:27.149910 14337 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:27.175408 14137 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:27.203977 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.193603 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:27.195175 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 165043us
I20260709 13:18:27.072934 14466 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:27.193156 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 141415us
I20260709 13:18:27.171700 14605 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.157053 14625 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.076623 14622 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.229622 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 501) took 1985 ms. Trace:
W20260709 13:18:27.269508 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 75977.1us
I20260709 13:18:27.269865 10050 rpcz_store.cc:276] 0709 13:18:25.244293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.698403 (+1454110us) service_pool.cc:224] Handling call
0709 13:18:26.716976 (+ 18573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.747410 (+ 30434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.747447 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.747497 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:26.747533 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.229237 (+481704us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54001b7e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:27.229304 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.229535 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69534976}
I20260709 13:18:27.195209 14627 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.158806 14637 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:27.239813 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:27.189963 14095 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:27.244320 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48339us
W20260709 13:18:27.246277 13574 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.997s	user 0.012s	sys 0.004s
I20260709 13:18:27.240746 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.199393 14523 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.325781 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.098023 14617 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.201611 14510 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.202050 14541 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.264209 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:27.264724 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2123) took 2042 ms (client timeout 2068 ms). Trace:
I20260709 13:18:27.272045 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:27.280313 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2369671380001819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 13:18:27.332942 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1983759960000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 13:18:27.282121 14638 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:18:27.282663 14202 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
I20260709 13:18:27.338055 14202 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.282814 13776 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.339824 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:27.340375 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.277292 14605 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:18:27.264148 14630 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.110s	user 0.002s	sys 0.002s
I20260709 13:18:27.206801 14337 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.286489 10130 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:27.241052 10122 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
W20260709 13:18:27.293264 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.294967 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.298911 10123 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:27.311902 10204 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:27.260572 14458 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:18:27.311813 10054 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:27.326675 10050 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:27.253402 14632 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:27.293324 14606 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.341384 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:27.332985 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60991.8us
W20260709 13:18:27.346314 10055 rpcz_store.cc:269] 0709 13:18:25.222604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.561104 (+338500us) service_pool.cc:224] Handling call
0709 13:18:27.264417 (+1703313us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:27.264650 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425440640}
W20260709 13:18:27.360723 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 161827us
I20260709 13:18:27.360951 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:27.361042 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.362557 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.347466 14541 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:27.288153 14551 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:27.327945 14457 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.017s
I20260709 13:18:27.292801 14544 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:27.334619 14642 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:18:27.385017 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 462) took 1313 ms. Trace:
I20260709 13:18:27.380300 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:27.385598 10055 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:27.375535 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:27.344616 14601 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.350489 14510 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:27.386123 10130 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:27.379245 14457 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.000s	sys 0.019s
I20260709 13:18:27.394490 14544 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.399025 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:27.408594 14202 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:27.409114 14493 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.735s	user 0.005s	sys 0.009s
I20260709 13:18:27.409723 14493 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.377683 14606 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:27.414435 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:27.378453 14643 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:27.403465 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:27.385646 10131 rpcz_store.cc:276] 0709 13:18:26.071538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.073788 (+  2250us) service_pool.cc:224] Handling call
0709 13:18:26.089634 (+ 15846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:26.739936 (+650302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.739987 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.740694 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.776855 (+ 36161us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:26.776908 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.777472 (+   564us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.382625 (+605153us) raft_consensus.cc:1685] finished
0709 13:18:27.382663 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.384939 (+  2276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":296,"raft.queue_time_us":173333,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":34381,"thread_start_us":685429,"threads_started":2,"wal-append.queue_time_us":294628}
W20260709 13:18:27.410832 14483 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.308s	user 0.014s	sys 0.001s
I20260709 13:18:27.470077 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:27.468072 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53734us
I20260709 13:18:27.471370 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:27.473330 14635 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:18:27.417313 14632 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:27.474071 14632 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.474542 14632 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.418200 14633 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:27.479620 14633 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.480446 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:27.481263 10131 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:27.412187 14509 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.409175 14545 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.464600 14458 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:27.486255 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 124653us
I20260709 13:18:27.417038 14617 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.462775 14619 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.465204 14644 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:18:27.487578 14378 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:27.485112 14650 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:27.491511 14458 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:27.491707 14509 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.467597 14643 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:27.505697 14536 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.002s
I20260709 13:18:27.505597 14622 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.500381 14510 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:27.508004 14597 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.325s	user 0.013s	sys 0.002s
I20260709 13:18:27.508268 14608 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.503019 14544 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.510092 14625 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.513531 14644 consensus_queue.cc:1243] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:18:27.510221 14634 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:27.510429 14632 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.504623 14650 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.519117 14634 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:27.521322 14650 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.517709 14597 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.523931 14634 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.526114 14634 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.530833 14632 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.532490 14650 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:27.348168 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84110.8us
I20260709 13:18:27.571729 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:27.469394 10130 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:27.577255 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2122) took 2354 ms. Trace:
I20260709 13:18:27.401505 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:27.579444 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 177869us
I20260709 13:18:27.580753 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.584290 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:27.585615 10131 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:27.487669 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.577983 10051 rpcz_store.cc:276] 0709 13:18:25.222328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.911754 (+689426us) service_pool.cc:224] Handling call
0709 13:18:25.912329 (+   575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.912883 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.912931 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.215827 (+1302896us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b2c000014c8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:27.243163 (+ 27336us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.349627 (+106464us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:27.364236 (+ 14609us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.364482 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.364568 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.575996 (+211428us) raft_consensus.cc:1685] finished
0709 13:18:27.576317 (+   321us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.576910 (+   593us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14401,"raft.queue_time_us":683,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":20080,"spinlock_wait_cycles":9769600,"thread_start_us":57183,"threads_started":1,"wal-append.queue_time_us":206584}
I20260709 13:18:27.482126 14483 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.579712 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 500) took 2337 ms. Trace:
I20260709 13:18:27.401150 14095 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.502069 14378 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.594270 13961 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:18:27.602814 10055 raft_consensus.cc:1217] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Deduplicated request from leader. Original: 7.8->[9.9-9.9]   Dedup: 9.9->[]
I20260709 13:18:27.634759 10055 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:18:27.634513 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:27.634950 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.636107 10131 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:27.636309 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 497) took 2496 ms. Trace:
I20260709 13:18:27.610852 14627 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:27.600771 14541 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:27.515746 14536 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.645283 14379 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:27.646517 14379 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:27.648463 10128 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:27.649827 14070 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.592782 14625 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.650709 14379 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.651003 14070 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.651562 14625 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.638129 14509 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:27.636883 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1965) took 2153 ms. Trace:
I20260709 13:18:27.616559 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.653231 10055 rpcz_store.cc:276] 0709 13:18:25.483933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.386434 (+1902501us) service_pool.cc:224] Handling call
0709 13:18:27.601972 (+215538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.635251 (+ 33279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.636659 (+  1408us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54002364d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:27.636803 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468597504}
I20260709 13:18:27.654146 14070 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:27.654587 10052 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:27.655318 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:27.656255 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:27.656968 10055 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.657390 10052 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.619961 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:27.660588 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1983) took 1780 ms. Trace:
I20260709 13:18:27.637167 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.605226 14137 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.596760 10864 leader_election.cc:400] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13.
I20260709 13:18:27.600445 14523 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:27.601649 14551 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.600986 14458 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.596629 14510 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.665720 10864 leader_election.cc:403] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:27.582697 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:27.661180 10124 rpcz_store.cc:276] 0709 13:18:25.879681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.881591 (+  1910us) service_pool.cc:224] Handling call
0709 13:18:25.881931 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.351834 (+469903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.659827 (+1307993us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54001f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:27.660300 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079307520}
I20260709 13:18:27.606930 14337 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.672248 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:27.638729 14541 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:27.668089 14640 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:18:27.679458 10050 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.680639 14640 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:27.680908 14536 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:27.681210 14640 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:27.682408 14632 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:27.676192 14458 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:27.668702 14523 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:27.683508 10199 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:27.683432 14523 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:27.685166 14523 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.682597 14551 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.687624 14523 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:27.687810 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.689040 10199 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:27.636890 10053 rpcz_store.cc:276] 0709 13:18:25.139755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.139912 (+   157us) service_pool.cc:224] Handling call
0709 13:18:25.167572 (+ 27660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.760586 (+593014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.635601 (+1875015us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b540024ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:27.636100 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341146368}
I20260709 13:18:27.672214 14137 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.618129  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:27.675477 14644 consensus_queue.cc:1175] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d51ad94710c348889d417f7f00032948" responder_term: 10 status { last_received { term: 8 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:27.629933 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:27.631206 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 1984) took 1751 ms (client timeout 1808 ms). Trace:
I20260709 13:18:27.699565 14137 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:27.694928 10129 rpcz_store.cc:269] 0709 13:18:25.880073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.880294 (+   221us) service_pool.cc:224] Handling call
0709 13:18:27.630754 (+1750460us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:27.630995 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349500288}
I20260709 13:18:27.613819 14501 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.711362 14378 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.715188 14204 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:27.718259 14204 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:27.637424 10056 rpcz_store.cc:276] 0709 13:18:25.242477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.689393 (+1446916us) service_pool.cc:224] Handling call
0709 13:18:26.689952 (+   559us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:26.711180 (+ 21228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.711214 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:26.732899 (+ 21685us) log.cc:844] Serialized 52 byte log entry
0709 13:18:27.028420 (+295521us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:27.028463 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.028535 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.578883 (+550348us) raft_consensus.cc:1685] finished
0709 13:18:27.578931 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.579427 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20670,"raft.queue_time_us":34984,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6224,"thread_start_us":257566,"threads_started":1,"wal-append.queue_time_us":578232}
I20260709 13:18:27.694396 10127 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:27.723537 10199 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.729321 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2141) took 2173 ms. Trace:
I20260709 13:18:27.730769 10056 rpcz_store.cc:276] 0709 13:18:25.556306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.723017 (+2166711us) service_pool.cc:224] Handling call
0709 13:18:27.728215 (+  5198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.728813 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.728842 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.728895 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:27.728945 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.729021 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.729209 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:27.732456 14501 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.733767 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2142) took 2177 ms. Trace:
I20260709 13:18:27.733448 14501 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.740134 14501 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.740363 14597 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.720144 14204 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.740475 13776 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:27.693004 14640 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.742152 13776 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:27.742691 13776 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:27.743993 14640 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.748611 10056 rpcz_store.cc:276] 0709 13:18:25.556700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.732068 (+2175368us) service_pool.cc:224] Handling call
0709 13:18:27.732700 (+   632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.733240 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.733280 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.733380 (+   100us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:27.733457 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.733524 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.733668 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:27.691489 14551 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.723764 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.683166 14632 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.754739 14547 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:27.760171 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 177723us
I20260709 13:18:27.760653 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.761547 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:27.763242 14597 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.764115 14608 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:27.763317 14622 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:27.765246 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.744815 13776 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:27.767176 14423 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:27.767782 13776 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:27.781963 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.784711 14608 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.769845 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:27.787453 14619 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.777372 14350 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.788316 14619 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.768795 14423 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:27.790131 14423 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:27.768308 14420 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:18:27.791632 14423 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:27.772390 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 176765us
I20260709 13:18:27.764717 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:27.786741 14608 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:27.765311 14622 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:27.772806 14632 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:27.816646 14423 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:27.772912 14552 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.824240 14610 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:18:27.831174 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:27.832491 10053 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:27.830691 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:27.768837 14597 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.835493 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:27.836452 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.820956 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:27.847605 10052 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:27.849627 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 504) took 2031 ms. Trace:
I20260709 13:18:27.851356 10053 rpcz_store.cc:276] 0709 13:18:25.815631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.831481 (+2015850us) service_pool.cc:224] Handling call
0709 13:18:27.831963 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.844987 (+ 13024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.846456 (+  1469us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:27.827692 14622 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:27.829171 14619 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.852344 10056 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.856115 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:27.857816 10053 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:27.858554 10054 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.863762 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 518) took 1082 ms. Trace:
I20260709 13:18:27.867553 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 1994) took 1970 ms. Trace:
I20260709 13:18:27.872879 10122 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:27.873497 14350 raft_consensus.cc:2764] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:18:27.781729 14541 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:27.874104 14269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.812s	user 0.028s	sys 0.059s
W20260709 13:18:27.890033 14269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.862s	user 0.029s	sys 0.060s
I20260709 13:18:27.890661 14269 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:27.891755 14511 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.875078 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 1963) took 2393 ms. Trace:
I20260709 13:18:27.880281 10130 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:27.884416 10128 rpcz_store.cc:276] 0709 13:18:25.897526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.900785 (+  3259us) service_pool.cc:224] Handling call
0709 13:18:25.901126 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.649123 (+1747997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.867087 (+217964us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54001c7854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:27.867452 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4016996608}
I20260709 13:18:27.895705 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2143) took 2315 ms. Trace:
I20260709 13:18:27.896256 10056 rpcz_store.cc:276] 0709 13:18:25.572934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.780244 (+2207310us) service_pool.cc:224] Handling call
0709 13:18:27.798375 (+ 18131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.887687 (+ 89312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.887975 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:27.868362 14597 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:27.908905 10122 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:27.909695 10055 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:27.907019 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:27.912158 10053 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:27.906994 14611 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.149s	user 0.000s	sys 0.002s
I20260709 13:18:26.996099 10048 rpcz_store.cc:276] 0709 13:18:25.236726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.067208 (+830482us) service_pool.cc:224] Handling call
0709 13:18:26.067616 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.067849 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:26.717322 (+649473us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.967476 (+250154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.967584 (+   108us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:26.969387 (+  1803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.969484 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.969635 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12969,"prepare.queue_time_us":281248,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":13249,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":6397,"thread_start_us":872963,"threads_started":2}
I20260709 13:18:27.915861 14633 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.919324 14536 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:27.919899 14646 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:27.920771 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.954236 10050 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:18:27.947825 14646 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:27.892580 10047 rpcz_store.cc:276] 0709 13:18:25.481132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.082132 (+1601000us) service_pool.cc:224] Handling call
0709 13:18:27.082678 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.083307 (+   629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.083352 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.092382 (+  9030us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.381737 (+289355us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:27.696752 (+315015us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.840945 (+144193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.849846 (+  8901us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.874508 (+ 24662us) raft_consensus.cc:1685] finished
0709 13:18:27.874529 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.874853 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2697,"prepare.queue_time_us":397394,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":85,"raft.queue_time_us":19791,"raft.run_cpu_time_us":500,"raft.run_wall_time_us":37322,"thread_start_us":248329,"threads_started":2,"wal-append.queue_time_us":316513}
I20260709 13:18:27.966920 13855 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:27.967568 13855 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:27.968935 13855 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.973275 14021 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:27.976030 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.919922 14536 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:27.980882 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 109463us
I20260709 13:18:27.983978 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:27.984864 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:27.988770 10127 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
W20260709 13:18:27.989524 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.886787 10893 rpcz_store.cc:276] 0709 13:18:26.781155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.784383 (+  3228us) service_pool.cc:224] Handling call
0709 13:18:27.820487 (+1036104us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:27.863615 (+ 43128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7099776}
I20260709 13:18:27.996053 14536 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.997573 14551 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.997730 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:27.998405 14551 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:27.999635 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.000262 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:28.003266 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.005307 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.970348 13855 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:28.007055 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:27.999677 10056 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:28.017036 10129 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: [12.6-14.7]
I20260709 13:18:28.032881 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2012) took 1321 ms. Trace:
I20260709 13:18:28.033555 14626 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:28.034106 14626 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:28.037741 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 497) took 2406 ms (client timeout 1625 ms). Trace:
W20260709 13:18:28.041159 10202 rpcz_store.cc:269] 0709 13:18:25.629454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.629592 (+   138us) service_pool.cc:224] Handling call
0709 13:18:28.035472 (+2405880us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b5400228b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:28.035684 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662914304}
I20260709 13:18:28.033564 10125 rpcz_store.cc:276] 0709 13:18:26.711188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.859274 (+1148086us) service_pool.cc:224] Handling call
0709 13:18:27.859904 (+   630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.860670 (+   766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.860732 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.861841 (+  1109us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.927592 (+ 65751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:27.927649 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.927750 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:28.032352 (+104602us) raft_consensus.cc:1685] finished
0709 13:18:28.032381 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.032578 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":166,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":230,"thread_start_us":65056,"threads_started":1,"wal-append.queue_time_us":125541}
I20260709 13:18:28.054916 14381 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:28.055812 14466 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.056280 14381 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:28.056843 14381 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.060887 14551 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.064389 10205 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:27.982012 14269 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.058917 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:28.058290 14381 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.024480 14378 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.073576 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 503) took 1099 ms. Trace:
I20260709 13:18:27.999670 14646 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.074234 10205 rpcz_store.cc:276] 0709 13:18:26.971465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.988575 (+1017110us) service_pool.cc:224] Handling call
0709 13:18:28.050219 (+ 61644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.070164 (+ 19945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.070424 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:28.077176 14646 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.082156 14599 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:28.078485 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:28.046701 14626 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:28.113296 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57540.2us
I20260709 13:18:28.099313 14466 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:28.109220 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.117225 10125 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:28.118736 14381 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.133855 10130 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.137705 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:28.144212 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:28.147358 10893 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:18:28.149508 14634 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.014s
I20260709 13:18:28.150914 14593 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:28.155463 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.157040 14424 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:28.155226 14466 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:28.157660 14424 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.158115 14466 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.158996 14424 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.160856 14424 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.160992 10126 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:28.161221 14466 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:28.162949 14662 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:18:28.165144 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106635us
I20260709 13:18:28.166390 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:28.167482 10205 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:28.180063 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.144351 14378 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:28.193637 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.178594 14552 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13.
I20260709 13:18:28.188016 14627 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:28.205049 10196 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:28.201242 10205 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:28.224009 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:28.226104 14626 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:28.201617 14640 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:18:28.245615 14625 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.246830 10899 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:28.255971 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:28.258205 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2011) took 1547 ms (client timeout 1779 ms). Trace:
I20260709 13:18:28.247645 14627 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.261628 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.262574 14021 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.263334 14597 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:28.272863 14605 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.273881 14536 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.277570 14446 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:18:28.284011 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83257.9us
I20260709 13:18:28.295333 10130 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:28.258788 10123 rpcz_store.cc:269] 0709 13:18:26.710943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.743061 (+ 32118us) service_pool.cc:224] Handling call
0709 13:18:28.257751 (+1514690us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54002d1254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:28.257965 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645854080}
W20260709 13:18:28.299700 14668 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.122s	user 0.002s	sys 0.001s
I20260709 13:18:28.302011 14632 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:28.303488 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 83478.3us
I20260709 13:18:28.304430 13776 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:28.305513 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2134) took 2965 ms. Trace:
I20260709 13:18:28.305608 14642 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:28.309265 14621 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.548s	user 0.010s	sys 0.003s
I20260709 13:18:28.281371 14608 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.303375 14605 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:28.323735 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99855.3us
I20260709 13:18:28.329146 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.329824 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3021) took 1360 ms. Trace:
W20260709 13:18:28.330006 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63202.5us
I20260709 13:18:28.330356 14633 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:28.330302 10196 rpcz_store.cc:276] 0709 13:18:26.969499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.205015 (+235516us) service_pool.cc:224] Handling call
0709 13:18:27.244633 (+ 39618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.205648 (+961015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.329456 (+123808us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540039ec54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:28.329736 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2202660096}
I20260709 13:18:28.331177 14633 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.331308 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:28.333343 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.321324 10049 rpcz_store.cc:276] 0709 13:18:25.340489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.780460 (+1439971us) service_pool.cc:224] Handling call
0709 13:18:26.952531 (+172071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.036190 (+ 83659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.036227 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:27.036277 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:27.036327 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.304940 (+1268613us) spinlock_profiling.cc:243] Waited 83.3 ms on lock 0x7b5400235fd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:28.305023 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.305272 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191696128}
I20260709 13:18:28.315657 14466 raft_consensus.cc:2764] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:18:28.341663 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2010) took 1631 ms. Trace:
I20260709 13:18:28.313290 14446 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.344945 10122 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:28.344506 10129 rpcz_store.cc:276] 0709 13:18:26.710666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.701831 (+991165us) service_pool.cc:224] Handling call
0709 13:18:27.989234 (+287403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:28.019492 (+ 30258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.334832 (+315340us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54002d1750. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:28.341556 (+  6724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659719424}
I20260709 13:18:28.348846 10128 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:28.364140 14483 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:28.371098 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.372227 14612 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:28.372750 14612 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:28.373781 14612 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.375798 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 464) took 1403 ms. Trace:
I20260709 13:18:28.376223 10126 rpcz_store.cc:276] 0709 13:18:26.972225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.860005 (+887780us) service_pool.cc:224] Handling call
0709 13:18:27.860515 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.161589 (+301074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.375463 (+213874us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540023c3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:28.375705 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689977600}
I20260709 13:18:28.377506 10047 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:28.323223 14642 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:28.381090 14625 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.382526 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.383409 10052 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:18:28.383618 14670 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:28.389503 14379 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.389776 14619 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:28.371197 14446 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.391348 14350 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:28.391873 14350 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:28.391963 14446 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.393075 14350 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.394037 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.395196 14350 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.395838 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.396150 14416 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:28.397437 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.398212 10125 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:28.399405 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.400751 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:28.401661 10130 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:28.351151 14634 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:18:28.402508 14634 consensus_queue.cc:1243] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:28.350709 14627 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:28.364310 14573 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:18:28.405433 14493 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:28.333598 14021 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.352478 14597 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.407753 14597 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.407780 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:28.409777 14021 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.410967 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 487) took 5132 ms. Trace:
I20260709 13:18:28.412758 10124 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:28.411499 10198 rpcz_store.cc:276] 0709 13:18:23.278176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.712163 (+433987us) service_pool.cc:224] Handling call
0709 13:18:23.779317 (+ 67154us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:23.947842 (+168525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.947884 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:24.541763 (+593879us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:24.550591 (+  8828us) log.cc:844] Serialized 52 byte log entry
0709 13:18:24.736002 (+185411us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:24.808416 (+ 72414us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.820834 (+ 12418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.820924 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.110743 (+289819us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b1800385360. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:25.423769 (+313026us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b1800385360. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:26.345603 (+921834us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b1800385360. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1
0709 13:18:28.266518 (+1920915us) spinlock_profiling.cc:243] Waited 92.7 ms on lock 0x7b1800385360. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:28.410523 (+144005us) raft_consensus.cc:1685] finished
0709 13:18:28.410547 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.410747 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25935,"prepare.queue_time_us":92232,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":96614,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":106832,"spinlock_wait_cycles":281127040,"thread_start_us":376796,"threads_started":3,"wal-append.queue_time_us":267201}
W20260709 13:18:28.414328 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.415004 14632 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.361339 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:28.418102 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.369576 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:28.386699 14643 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:28.342555 14269 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:28.335604 14622 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.429458 10131 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:28.379901 14610 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:18:28.435443 10126 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:18:28.439038 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.436002 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74771.8us
I20260709 13:18:28.440006 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.443356 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.454576 10049 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:28.448921 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.451778 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.389664 14644 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:18:28.460155 10127 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:28.461738 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.461266 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.468809 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:28.472198 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.472893 10054 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:28.444144 14350 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.454238 14552 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:28.457304 14597 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.332681 14633 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.479732 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.452312 14612 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:28.479228 14541 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.483446 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.444679 14632 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:28.490309 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.492503 10052 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:28.495178 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.503933 14071 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:18:28.504819 14071 consensus_queue.cc:1243] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:28.505414 14622 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.506376 14071 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:18:28.507103 14598 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 2.267s	user 0.011s	sys 0.016s
I20260709 13:18:28.510985 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:28.515951 14502 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:28.516743 14502 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.518168 10126 raft_consensus.cc:2376] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13. Re-sending same reply.
I20260709 13:18:28.519309 14137 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.519614 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.519950 14137 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.520781 14541 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.522385 14137 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.520406 10047 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.475880 14350 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.531999 14608 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.582653 14575 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:28.535772 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:28.575376 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.590407 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.591440 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.592020 10196 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:28.539680 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.566634 14642 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.523365 10054 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:28.590781 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.588724 14575 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:28.531967 14502 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.581768 10124 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:28.553793 10197 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:28.523048 14137 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:28.592185 10204 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:18:28.527727 14537 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:18:28.596299 14575 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.522940 14541 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:28.599267 14425 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.536443 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.600063 14425 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.554725 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.601524 10049 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:28.603308 14642 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:28.561160 14675 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:28.599224 14589 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:28.606194 14589 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.593093 10053 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:28.607460 14589 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.600634 10048 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
W20260709 13:18:28.608182 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 238808us
I20260709 13:18:28.608740 14589 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.609259 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.609458 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:28.610613 10200 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:28.611423 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.612728 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.613903 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.615319 14632 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:28.615845 14425 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:28.616173 14632 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:28.615913 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1878) took 2529 ms (client timeout 2595 ms). Trace:
I20260709 13:18:28.616901 10204 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:28.608089 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.564769 14650 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:28.537050 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.592268 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:28.624536 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.545655 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.617455 10197 rpcz_store.cc:269] 0709 13:18:26.086374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.086559 (+   185us) service_pool.cc:224] Handling call
0709 13:18:28.615627 (+2529068us) spinlock_profiling.cc:243] Waited 2.47 s on lock 0x7b54002ddcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:28.615839 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5670780672}
I20260709 13:18:28.525203 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.617446 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.617710 10198 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:28.643170 10056 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:28.599865 14575 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:28.480360 14552 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:28.650388 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:28.650929 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.597543 14137 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.583474 14021 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:28.663086 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:28.649626 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:28.616689 14632 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.674793 10200 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:28.670600 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.671437 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:28.673000 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 57903.6us
I20260709 13:18:28.486353 14634 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.178s
I20260709 13:18:28.527892 14071 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.689316 10128 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:28.675748 10196 raft_consensus.cc:2376] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16. Re-sending same reply.
I20260709 13:18:28.681351 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:28.683101 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72072.6us
W20260709 13:18:28.676838 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 219534us
I20260709 13:18:28.690781 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:28.699337 10130 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:28.668035 14552 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:28.664304 14597 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.673029 14137 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:28.651603 14551 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.679680 14021 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:28.716727 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.729171 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.685595 14608 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:28.806413 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.786842 10049 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:28.730048 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.683965 14632 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:28.754364 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:28.818230 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65935us
I20260709 13:18:28.819023 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.718005 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.751230 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.717159 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:28.715844 10125 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:28.816859 14622 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:28.822758 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3006) took 2882 ms. Trace:
I20260709 13:18:28.823051 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:28.823427 14673 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:18:28.824044 14407 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:18:28.824932 14619 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:28.825379 14619 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.825970 14619 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:28.778412 10051 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:28.754303 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:28.820359 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:28.811328 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 300532us
W20260709 13:18:28.816691 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.784454 14379 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.768870 14605 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.777441 14541 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:28.802744 10053 raft_consensus.cc:2376] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12. Re-sending same reply.
I20260709 13:18:28.808310 14021 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:28.831872 10895 pending_rounds.cc:85] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 7
I20260709 13:18:28.550328 14378 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 4 to 8, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:28.716806 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:28.834439 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80268.5us
I20260709 13:18:28.841162 10895 pending_rounds.cc:107] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:18:28.827457 14627 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.825429 14407 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:28.753943 14597 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:28.858656 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.813834 14608 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:28.844488 10895 raft_consensus.cc:2889] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:28.844803 10055 raft_consensus.cc:2376] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11. Re-sending same reply.
W20260709 13:18:28.834940 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117027us
I20260709 13:18:28.842720 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:28.849284 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:28.858341 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.689360 14071 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.823305 10203 rpcz_store.cc:276] 0709 13:18:25.794339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.794596 (+   257us) service_pool.cc:224] Handling call
0709 13:18:25.811774 (+ 17178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.816926 (+  5152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.816963 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:25.817487 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.028048 (+210561us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:26.217590 (+189542us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.223972 (+  6382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.224065 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:28.506306 (+2282241us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:28.618059 (+111753us) raft_consensus.cc:1685] finished
0709 13:18:28.650397 (+ 32338us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.676783 (+ 26386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8190,"prepare.queue_time_us":223042,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":13121,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":16111,"spinlock_wait_cycles":21288704,"thread_start_us":339337,"threads_started":2,"wal-append.queue_time_us":386769}
I20260709 13:18:28.859772 10054 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:28.838100 14510 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:28.873324 10047 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:28.877274 10892 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 4 to 8, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:28.858744 14407 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:28.859580 14627 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:28.883424 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.896968 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:28.885411 14597 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:28.863039 14608 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:28.888825 10899 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:18:28.890597 14625 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.006s
I20260709 13:18:28.900898 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:28.901742 10122 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:28.921113 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.925051 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:28.926429 10055 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:28.935827 14407 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.937528 14407 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.903013 14627 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.930963 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:28.866465 14350 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:28.944218 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:28.895206 14468 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.858101 14379 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.951380 14510 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:28.902519 14446 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
I20260709 13:18:28.865688 14541 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.838299 14551 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:28.944857 14625 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:28.951988 14425 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.958534 14468 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.980158 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 115625us
I20260709 13:18:28.964255 14466 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:28.954071 14680 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:28.973562 14502 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:28.987005 10131 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.964193 14589 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:28.982322 10056 raft_consensus.cc:2376] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11. Re-sending same reply.
I20260709 13:18:28.981828 14468 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:28.979864 14425 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.022696 14425 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.025635 14508 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:28.999382 13574 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:29.009357 14204 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:29.012614 14551 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:29.012902 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.015319 14510 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.016472 14589 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:29.043663 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47816.1us
I20260709 13:18:29.045147 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:29.046480 14597 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:29.018383 14379 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:28.996697 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.054271 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 157876us
W20260709 13:18:29.024224 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.997622 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66763us
I20260709 13:18:29.026736 14446 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:29.034647 14542 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:29.040735 14458 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:29.063807 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:29.072489 14589 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.073621 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:29.077318 14612 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.078009 14612 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.080746 14612 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.061342 14458 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:29.090171 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:29.057411 14542 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.092957 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:29.092571 14407 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:29.097801 10204 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:18:29.061813 14021 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.083979 14632 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.109120 13574 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.109133 14381 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:29.109853 14508 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:29.110597 14423 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:29.110939 14458 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.112382 14416 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:29.131026 13574 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:29.132010 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.112849 14622 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:29.133049 14622 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:29.133517 13574 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.134016 14021 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.134812 10126 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:29.121583 14551 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:29.136021 14551 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.135555 14542 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.136381 14551 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.137323 14542 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:29.113821 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 51988.2us
I20260709 13:18:29.139822 14508 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:29.139889 14466 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:29.140703 14466 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:29.143937 14416 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:29.144474 14416 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.145493 14416 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.122043 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:29.146917 14416 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:29.152768 14680 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:29.140413 14508 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.154325 14508 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.154978 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:29.124651 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 157607us
I20260709 13:18:29.084905 14379 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:29.130414 14597 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:29.151528 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.160434 10122 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
W20260709 13:18:29.160640 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46132.4us
I20260709 13:18:29.161300 13574 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.162081 13574 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.163777 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.158332 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.174053 10124 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.106806 14552 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.175796 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:29.183056 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90229.6us
I20260709 13:18:29.153121 14541 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.189366 10131 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:29.171883 14508 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.170862 14458 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:29.203333 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.170238 14680 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:29.183749 14269 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.751s	user 0.014s	sys 0.000s
I20260709 13:18:29.182665 14021 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:29.209759 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:29.177217 14502 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.215641 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:29.219655 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:29.220729 10053 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:29.223219 14423 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:29.224619 14423 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:29.225023 14423 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.226052 14423 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.200488 14642 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:29.227600 14423 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:29.227263 14502 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.228144 14642 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.230000 14502 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.230585 14642 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.231408 10056 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:18:29.235492 14622 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:29.239190 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.250655 14552 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:29.259200 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.239751 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.255265 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.244834 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.196019 14682 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:29.202613 14612 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.266240 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.270109 14095 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:29.271173 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:29.271432 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:29.259063 14204 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:29.274012 13574 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.212507 14269 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:29.279031 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 124120us
W20260709 13:18:29.279074 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72876.4us
I20260709 13:18:29.277649 10123 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:29.259743 14021 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:29.237139 14466 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.284102 14021 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.285499 14466 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:29.271380 14612 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.289713 14612 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.291405 10126 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:29.293342 14597 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:29.278398 14682 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.296415 10122 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:29.297688 10095 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:29.298170 10127 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:29.298884 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.279143 14095 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.300737 14095 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.327881 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:29.332638 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51743.9us
I20260709 13:18:29.335616 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:29.335776 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.340212 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.339524 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:29.346256 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.325857 14378 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.268611 14608 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.326849 10129 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 4 to 8, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:29.268054 14619 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:29.355762 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58172.5us
I20260709 13:18:29.358219 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:29.359256 14690 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:29.360059 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.360229  9949 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 11 to 16, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.362488 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.363379 10128 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:29.367889 14483 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:29.368460 14483 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:29.368870 14483 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:29.370464 14619 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.371068 14608 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.338289 14597 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:29.343489 14502 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:29.379623 14597 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.380481 14632 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.307148 14627 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.383177 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.383668 14627 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:29.384439 10125 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:29.386715 10125 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:29.330997 14204 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.361593 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:29.389225 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46900.2us
I20260709 13:18:29.389076 10131 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.363536 10124 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:29.352532 10019 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.692s (SENT)
I20260709 13:18:29.392022 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:29.400132 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.400583 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 106463us
I20260709 13:18:29.348351 14269 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:29.348534 14379 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:29.403214 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61137.8us
I20260709 13:18:29.404081 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.410725 10123 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.403163 14625 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:29.415936 14625 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:29.417086 14625 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.419994 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.422030 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:29.427234 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.345975 14541 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:18:29.419912 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.435521 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:29.407299 14379 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:29.468657 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48841.3us
I20260709 13:18:29.351709 14551 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:29.475957 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54016.6us
I20260709 13:18:29.391469 14378 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.470479 14379 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:29.480444 14680 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.481105 14137 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.002s
I20260709 13:18:29.481042 14680 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.481629 14137 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.483777 10048 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:29.485857 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.487216 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.487684 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.488381 14468 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.489291 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.461036 14627 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:29.491242 14627 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.491667 14627 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:29.493638 14378 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:29.490671 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.495749 14552 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:29.496009 10124 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.496470 14552 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.496920 14552 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:29.497370 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.498387 10124 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.470678 14682 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.410753 14269 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.513782 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.516160 10127 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:29.517055 14407 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:29.517405 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.563947 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.517583 14423 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:29.517899 14622 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.518625 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.572973 10130 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.520668 14551 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.521083 14537 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:29.522984 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:29.581835 10895 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:18:29.526331 14612 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:29.526861 14552 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.527473 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:29.528187 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.424926 14416 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:18:29.532196 14269 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.404907  9949 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 9 to 12, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.467092 14071 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:18:29.541757 14619 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:29.520466 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85007us
W20260709 13:18:29.479280 14691 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
W20260709 13:18:29.545861 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.490707 14468 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.558509 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.571491 10131 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.573973 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:29.576112 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 235027us
I20260709 13:18:29.617136 10123 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:29.653367 10127 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.654361 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.550495 14204 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.668939 14608 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.669879 14608 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.679289 14551 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:29.697846 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.703681 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.722347 14552 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:29.733529 14682 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.738001 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.762s (SENT) [suppressed 22 similar messages]
W20260709 13:18:29.742234 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 168379us
I20260709 13:18:29.745843 14575 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.746275 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.746752 14575 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.634889 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.637799 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.620098 10125 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:29.612605 14537 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.656869 14703 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:29.770718 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.771837 10127 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:29.799657 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.800892 10123 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:29.807256  9951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 206) took 24117 ms (client timeout 15000 ms). Trace:
I20260709 13:18:29.808496 14407 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:29.734508 14682 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.956209 14682 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.957003 14682 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.957542 14682 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.815694 10131 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:29.958945 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.959955 10126 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:29.960480 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.825603 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.961809 10128 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:29.826592 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 185546us
I20260709 13:18:29.837064 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:29.773279 14537 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:29.829579 13574 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:29.802274 14552 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.839890 14680 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.966678 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.856110 14541 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:29.857131 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.854136 14446 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:29.859887 14608 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:29.814699 14468 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:18:29.845711  9949 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:29.881541 14551 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:29.887208 14095 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.887208 14643 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:29.963282 14378 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:18:29.896508 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:29.898612 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:29.900689 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:29.981033 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144088us
I20260709 13:18:29.983306 10204 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:29.902714 14137 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.810490 14483 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.911720 14575 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:29.919983 14466 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:29.935667 14502 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.045s	sys 0.067s
I20260709 13:18:29.949808 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:29.940964 14508 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.005s
I20260709 13:18:29.654942 14619 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.960899 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.854128 14627 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:29.968715 13574 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:29.972410 14552 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.973796 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:29.973875 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.048070 10128 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:30.048017 10123 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:29.975944 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:29.894171 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.030300 10124 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12. Re-sending same reply.
I20260709 13:18:30.050421 10198 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:18:29.990080 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.027s	sys 0.043s
W20260709 13:18:30.051231 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.027s	sys 0.044s
I20260709 13:18:30.007449 14589 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:18:30.051625 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 647399us
I20260709 13:18:30.016798 14633 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.970041 14446 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:30.008020 10127 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:29.979055 14541 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:29.975984 14551 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:29.979460 14709 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:29.995426 14416 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:29.926141 14379 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.065893 14446 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.067591 14379 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.068271 14071 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:30.068696 14071 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:30.068575 14379 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:18:30.019546 14502 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.046s	sys 0.067s
I20260709 13:18:30.069763 14071 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.070976 14608 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.071635 14608 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.046644 14466 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.083000 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.044095 13574 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.056252 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:30.084309 10129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.060s
W20260709 13:18:30.084725 10129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.016s	sys 0.060s
I20260709 13:18:30.059899 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:30.085378 10123 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.084539 10196 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:29.978932 14095 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.063827 10125 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12.
I20260709 13:18:30.064373 10130 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:30.067710 14446 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.070295 14502 consensus_queue.cc:1243] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:30.075695 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.042403 14407 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.990513 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.077764 14551 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.079764 14709 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.080929 14541 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.049418 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.052706 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:30.054316 10131 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:30.100903 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.015151 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:30.088058 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.101661 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.091406 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.102620 10128 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:18:30.074122 14608 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.028683 14643 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:30.099385 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 154795us
I20260709 13:18:30.060913 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:29.954907  9951 rpcz_store.cc:269] 0709 13:18:05.714837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.730372 (+ 15535us) service_pool.cc:224] Handling call
0709 13:18:29.806886 (+24076514us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:25.170542 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:25.181263 (+ 10721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:25.181323 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:18:25.181346 (+    23us) tablet.cc:662] Decoding operations
0709 13:18:25.307910 (+126564us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:25.308002 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:25.308030 (+    28us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:18:25.483607 (+175577us) tablet.cc:701] Row locks acquired
0709 13:18:25.483670 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:18:25.483882 (+   212us) write_op.cc:270] Start()
0709 13:18:25.484079 (+   197us) write_op.cc:276] Timestamp: P: 1783603105483801 usec, L: 0
0709 13:18:25.484102 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:25.488305 (+  4203us) log.cc:844] Serialized 9056 byte log entry
0709 13:18:26.235598 (+747293us) op_driver.cc:464] REPLICATION: finished
0709 13:18:26.795751 (+560153us) write_op.cc:301] APPLY: starting
0709 13:18:26.795853 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:27.720872 (+925019us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:27.721083 (+   211us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:28.158558 (+437475us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:28.193938 (+ 35380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:18:28.196582 (+  2644us) write_op.cc:312] APPLY: finished
0709 13:18:28.206617 (+ 10035us) log.cc:844] Serialized 204 byte log entry
0709 13:18:28.359600 (+152983us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:28.496424 (+136824us) write_op.cc:454] Released schema lock
0709 13:18:28.643602 (+147178us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:28.813015 (+169413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:28.814828 (+  1813us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1317982,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":555802,"apply.run_cpu_time_us":70509,"apply.run_wall_time_us":1959603,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100083,"num_ops":23,"prepare.queue_time_us":135013,"prepare.run_cpu_time_us":132855,"prepare.run_wall_time_us":666045,"raft.queue_time_us":172210,"raft.run_cpu_time_us":17076,"raft.run_wall_time_us":509889,"replication_time_us":734525,"thread_start_us":1021703,"threads_started":5,"wal-append.queue_time_us":650206,"wal-append.run_cpu_time_us":11339,"wal-append.run_wall_time_us":2223061}]]}
I20260709 13:18:30.111693 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:30.102696 10203 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
W20260709 13:18:30.112385 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59770.8us
I20260709 13:18:30.132695 10127 raft_consensus.cc:2376] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15. Re-sending same reply.
I20260709 13:18:30.133755 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:30.134948 13574 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:30.145367 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.146492 10126 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:18:30.154265 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 172116us
I20260709 13:18:30.096701 14642 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.161872 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.162479 10131 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:30.081988 14633 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.159046 10124 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:30.164409 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:30.166188 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.086567 14682 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:30.166888 10196 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:30.166229 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:30.135932 14541 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:30.184029 14541 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.184541 14466 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.184689 14541 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.098950 14407 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.185987 10198 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:30.186244 14682 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:30.187179 14407 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.167793 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.190431 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:30.175417 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 437448us
I20260709 13:18:30.166265 14204 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.191114 10125 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:30.162995 14612 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.168373 14642 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.190744 10126 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:30.149505 14468 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:30.127529 14071 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:30.194890 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.160018 14379 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.161541 14551 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:30.187669 14682 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:18:30.229205 10019 leader_election.cc:336] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.788s (SENT)
I20260709 13:18:30.229848 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.195102 14713 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.230885 10125 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:30.103670 14597 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.196578 14204 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.238809 10123 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:30.212416 14575 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:30.243193 10127 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:30.244709 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87624.8us
I20260709 13:18:30.234802 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.207170 14379 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.247395 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.223577 14682 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.248772 10196 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:30.255388 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.255539 10125 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:30.259478 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2059) took 1010 ms. Trace:
I20260709 13:18:30.276127 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.260669 14552 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.261955 14709 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.264037 10053 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
W20260709 13:18:30.266216 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.268289 10048 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:30.260224 14095 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:30.269999 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.196609 14612 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.274849 10122 rpcz_store.cc:276] 0709 13:18:29.248556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.297332 (+ 48776us) service_pool.cc:224] Handling call
0709 13:18:29.297750 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:29.563402 (+265652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.563457 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.877224 (+313767us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b2c0011b9a8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:29.877847 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:18:30.112507 (+234660us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:30.112596 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.112782 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.259031 (+146249us) raft_consensus.cc:1685] finished
0709 13:18:30.259063 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.259392 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112755,"raft.queue_time_us":105226,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5048,"spinlock_wait_cycles":6818048,"thread_start_us":395037,"threads_started":2,"wal-append.queue_time_us":354788}
W20260709 13:18:30.275605 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 111306us
I20260709 13:18:30.269057 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.270797 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:30.271135 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104960us
I20260709 13:18:30.240906 14721 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:30.287750 10052 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:30.284996 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:30.284534 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:30.291759 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45089.9us
I20260709 13:18:30.253000 14379 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.252862 14597 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.303534 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.289367 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:30.271560 14483 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.312692 10125 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:30.310001 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.273146 14718 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:30.097790  9949 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 14 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:30.279307 14682 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:30.318010 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.343873 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.344775 10205 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:30.279603 14575 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:30.324858 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.288023 14721 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:30.325796 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.326521 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.331601 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.333506 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:30.334296 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.335153 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.298130 14612 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.287204 14378 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:30.337779 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:30.347491 10055 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:30.304577 14698 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:30.307607 14680 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:30.366771 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:30.325549 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.349511 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.245287 14713 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.304950 14633 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:30.294935 14714 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:30.353865 10050 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:30.330469 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.351537 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.354990 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:30.400643 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70291.2us
I20260709 13:18:30.356076 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.303929 14627 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.300237 14642 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.362038 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.402346 10898 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:30.384603 10196 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:30.400751 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:30.409200 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.409950 10047 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:30.360400 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:30.363162 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.365115 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.413550 10892 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:18:30.360200 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.367595 10052 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.358783 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.372500 14575 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.375459 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.418421 10205 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:30.377339 14541 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.414122 10202 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:30.377852 10899 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:30.379542 14542 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:30.380616 14204 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:30.382159 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.402721 14619 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.361784 14698 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:30.442262 14698 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.443585 14698 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.443768 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.385608 14680 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:30.395622 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57985.5us
W20260709 13:18:30.414759 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54483.4us
I20260709 13:18:30.414916 13574 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:30.446650 13574 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:30.408622 14423 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.446707 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.415671 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.412670 10199 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:30.427500 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.412493 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:30.429394 10200 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:30.392515 14597 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:30.455173 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53704.3us
I20260709 13:18:30.456091 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:30.456822 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.430200 10047 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:30.405227 14714 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.457513 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:30.433266 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.434114 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.406631 14726 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:30.459458 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.447752 13574 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.460882 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.440922 14541 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.445463 14680 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.438860 10204 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:30.431605 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.464861 10197 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.464890 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.459354 14714 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:30.460278 10051 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:30.469462 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:30.471400 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:30.473628 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.475317 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.477416 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.477993 10047 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:30.474485 10055 raft_consensus.cc:2368] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:30.462929 14726 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.479511 14071 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:30.479915 14726 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:30.480340 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.431864 14643 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:30.480756 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:30.481529 14715 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:30.466468 14458 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:30.481884 10900 raft_consensus.cc:2410] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:18:30.482174 14458 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:30.473913 10205 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:30.480756 10054 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:30.490128 10019 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.927s (SENT)
W20260709 13:18:30.490250 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.490617 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.490747 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.453871 14725 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:30.491660 10051 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:30.501163 10199 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:30.474670 14552 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:18:30.490978 10020 connection.cc:256] RPC call timeout handler was delayed by -1.3823412009999174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:18:30.505905 10019 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.998s (SENT)
I20260709 13:18:30.497902 14541 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.511030 10201 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:18:30.511443 14541 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.494596 14642 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:30.512391 14642 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.512831 14642 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:30.512988 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:30.508092 14705 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:30.474856  9949 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:30.516978 10019 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.006s (SENT)
W20260709 13:18:30.517256 14707 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.548s	user 0.002s	sys 0.005s
I20260709 13:18:30.517136  9949 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.460758 14612 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:18:30.520116 14541 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.520766 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:30.520895 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.521387 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:30.472673 14712 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.522697 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.530575 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.498006 14680 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.497362 14703 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.501681 14723 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
W20260709 13:18:30.534360 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63070.7us
I20260709 13:18:30.535570 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.537902 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.531096 10203 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:30.539381 10123 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:30.633379 10053 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:30.550697 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.553205 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:30.557853 10095 connection.cc:256] RPC call timeout handler was delayed by -2.4363598120000916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:18:30.638743 10095 connection.cc:256] RPC call timeout handler was delayed by -2.3949598120000246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:18:30.639990 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:30.552721 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.642050 10130 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:30.560604 10897 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:18:30.562160 10124 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:30.562574 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.566247 10051 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:30.566437 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:30.572391 10127 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:30.560281 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.591423 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.649226 10048 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:30.651973 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:30.629556 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:30.540268 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:30.654626 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.541464 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:30.594605 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73923us
I20260709 13:18:30.656864 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:30.667196 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:30.670110 10204 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:30.673491 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.675202 10123 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:30.662156 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:30.559439 10052 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:30.621866 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.680095 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.680891 10201 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:18:30.620549 14446 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.005s
I20260709 13:18:30.585227 14423 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:18:30.592861 14095 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.594942 14705 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.700201 10047 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:30.592732 14542 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.566293 14137 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.620234 14718 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:30.690964 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.570072 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:30.692150 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 52328.3us
I20260709 13:18:30.685344 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:30.712540 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50497.9us
I20260709 13:18:30.712942 10047 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:18:30.713343 10047 pending_rounds.cc:85] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:18:30.713546 10047 pending_rounds.cc:107] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:18:30.713701 10047 raft_consensus.cc:2889] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:30.696156 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.629709 14713 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.566227 14204 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.622493 14716 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:30.690824 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:30.702564 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.702834 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.705616 10202 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:30.683243 14589 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:30.673624 14625 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:30.698877 14643 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:30.678120 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:30.691046 14709 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:30.584930 14703 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.726215 10055 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 6.
I20260709 13:18:30.710353 14137 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.663499 14378 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:18:30.691264 14627 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:30.706449 14542 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.732566 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:30.779318 14597 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.780064 14597 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.783785 14137 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:30.784325 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:30.784624 14137 raft_consensus.cc:2764] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:30.785130 10053 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:30.786429 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 53993.3us
I20260709 13:18:30.786711 10053 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:30.733870 14204 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.729753 14739 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:30.726480 14716 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.727061 14589 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.739544 10051 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:30.715847 10050 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:30.750607 14737 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:30.707311 14095 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:30.740319 10204 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:30.763842 10197 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
W20260709 13:18:30.768932 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90944.1us
I20260709 13:18:30.904294 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:30.904417 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.905934 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:30.778031 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:30.907629 10198 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:30.770888 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:30.733590 14730 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:18:30.734472 14733 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:30.871006 10047 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:18:30.954465 10047 pending_rounds.cc:85] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 4
I20260709 13:18:30.885442 14738 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.873142 14423 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.662551  9949 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:30.873533 14739 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.954872 10047 pending_rounds.cc:107] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:18:30.901471 10055 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:18:30.958598 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 441611us
I20260709 13:18:30.923714 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.937435 10197 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:30.876772 14625 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:30.712010 14705 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.886302 14716 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:30.909472 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:30.772329 14642 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:30.899307 14718 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:30.974255 10051 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:18:31.005959 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.008832 10204 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:30.956331 14700 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:31.010790 10050 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:30.902319 14737 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.894186 14589 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:31.011965 10047 raft_consensus.cc:2889] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:31.020828 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 111971us
I20260709 13:18:30.959303 14709 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:30.877338 14627 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:31.009579 10019 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.132s (SENT)
I20260709 13:18:30.788714 14740 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.203s
W20260709 13:18:31.094988 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.011983 14739 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.030689 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.105715 10197 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:31.032018 14095 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.087423 10053 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 4 to 8, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:30.976693 14423 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.037914 14468 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:31.027277 14718 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.043881 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.129460 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.130708 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:31.101842 14726 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.006s
I20260709 13:18:30.976557 14597 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.083305 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:31.143067 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60264.7us
I20260709 13:18:30.977788 14741 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:31.014699 14537 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.040s	user 0.010s	sys 0.006s
I20260709 13:18:31.087072 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:31.087548 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:31.034240 14711 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.722s	user 0.006s	sys 0.000s
I20260709 13:18:31.095286 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:31.109282 10050 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:31.108681 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.158962 10048 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:18:31.134204 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124679us
W20260709 13:18:31.159813 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72850us
I20260709 13:18:30.977577 14542 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.090080 14551 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.086335 14737 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:31.144326 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.082993 14625 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.146096 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.170724 10123 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:31.150022 10131 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:30.980463 14738 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.150776 10130 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:18:31.169535 14551 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.178646 14551 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.151376 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:31.151924 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.102406 14612 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:31.180564 14612 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.168648 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.181911 10204 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:31.161154 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.183697 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.184652 14612 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.185659 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.186110 14612 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:31.186581 14612 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.187760 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.186972 14612 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:31.147461 14741 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:31.160501 14416 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.148917 14468 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.147854 14726 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:31.251389 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100116us
W20260709 13:18:31.229393 14714 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.745s	user 0.015s	sys 0.000s
W20260709 13:18:31.252671 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 157522us
I20260709 13:18:31.260548 10197 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:31.160337 14537 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.252244 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.164815 14752 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:31.231530 14720 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:18:31.261287 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.221861 14741 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.280428 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:31.210081 14748 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:31.290031 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129744us
W20260709 13:18:31.299059 10196 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.001s
I20260709 13:18:31.289750 14714 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.297803 14703 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.287497 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:31.285953 14713 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:31.201763 14625 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.285449 14752 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.258940 14753 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:31.135273  9949 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.303735 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.336393 10896 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:31.353992 10204 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.336393 10200 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:31.338547 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 177440us
W20260709 13:18:31.314924 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53739.9us
I20260709 13:18:31.356079 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.325317 10892 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:31.242116 14709 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.359733 10125 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:31.300520 10196 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:31.398785 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:31.352030 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 271478us
W20260709 13:18:31.361471 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.400326 10124 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:31.401981 10048 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:18:31.402707 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.410163 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.410259 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.449018 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.449379 10047 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:31.350422 14744 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:31.414366 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55737us
I20260709 13:18:31.455753 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.489588 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:31.490247 10051 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:31.490448 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.304407 14703 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:31.477175 10196 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:31.477756 10130 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:31.479269 10204 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:31.303594 14712 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:31.329553 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:31.353456 14759 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.455753 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:31.484359 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:31.355131 14642 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.494524 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:31.482333 10199 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.002s
I20260709 13:18:31.499641 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:31.475811 14729 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:18:31.411945 14748 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:31.505484 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:31.511304 14378 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:31.512060 14095 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:31.513557 14705 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.514648 14718 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:31.520390 14755 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:31.496279 14744 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:31.495159 14745 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 13:18:31.496096 14597 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.525326 14468 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.449622 14542 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:18:31.409524 14416 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
W20260709 13:18:31.567723 14698 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.124s	user 0.015s	sys 0.000s
I20260709 13:18:31.530052 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:31.531569 14726 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.411242 14633 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:18:31.557402 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1901) took 1241 ms. Trace:
I20260709 13:18:31.668740 10203 rpcz_store.cc:276] 0709 13:18:30.316194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.316678 (+   484us) service_pool.cc:224] Handling call
0709 13:18:31.557167 (+1240489us) spinlock_profiling.cc:243] Waited 67.3 ms on lock 0x7b540012aa54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:31.557309 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154734336}
I20260709 13:18:31.341162 14713 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.562242 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3058) took 1043 ms. Trace:
I20260709 13:18:31.564064 14537 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.603720 14458 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.559002 13574 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:31.538952 14423 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.608542 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2068) took 1594 ms. Trace:
I20260709 13:18:31.609678 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2177) took 1104 ms. Trace:
W20260709 13:18:31.502307 14682 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.141s	user 0.015s	sys 0.001s
I20260709 13:18:31.646931 10199 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:31.647789 10124 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 7: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:31.493287 14379 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.141s	user 0.005s	sys 0.010s
I20260709 13:18:31.649531 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:31.556524 14716 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.529s	user 0.006s	sys 0.005s
I20260709 13:18:31.655421 14763 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:31.657661 14204 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.658345 10051 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:31.558683 14627 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.503703 14703 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.565968 14625 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.553357 14738 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.548108  9949 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 14, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.505077 14714 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.503397 14712 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:31.665145 10131 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:31.648389 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 192721us
I20260709 13:18:31.667959 10054 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
W20260709 13:18:31.550462 14680 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.971s	user 0.008s	sys 0.005s
W20260709 13:18:31.669791 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 185526us
I20260709 13:18:31.743947 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:31.552031 14757 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:31.571009 10130 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:31.670145 10893 raft_consensus.cc:2368] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:31.677955 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.666529 14612 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.505260 14749 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 13:18:31.754061 10203 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:31.687225 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.617712 14761 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.663056 14744 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.614711 14748 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:31.650811 14597 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.714818 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1912) took 1170 ms. Trace:
I20260709 13:18:31.707003 13574 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:31.760309 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:31.713410 10049 rpcz_store.cc:276] 0709 13:18:30.505006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.710717 (+205711us) service_pool.cc:224] Handling call
0709 13:18:30.729055 (+ 18338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.772760 (+ 43705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.772817 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.897312 (+124495us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.157105 (+259793us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:31.452726 (+295621us) log.cc:844] Serialized 14 byte log entry
0709 13:18:31.488857 (+ 36131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.488947 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.609420 (+120473us) raft_consensus.cc:1685] finished
0709 13:18:31.609443 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.609609 (+   166us) inbound_call.cc:177] Queueing success response
0709 13:18:31.682982 (+ 73373us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18003a5700. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
Metrics: {"mutex_wait_us":60447,"prepare.queue_time_us":332457,"prepare.run_wall_time_us":63,"raft.queue_time_us":82420,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":47691,"spinlock_wait_cycles":46244096,"thread_start_us":393179,"threads_started":2,"wal-append.queue_time_us":335307}
I20260709 13:18:31.715582 10202 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:31.712687 14379 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.671690 10201 rpcz_store.cc:276] 0709 13:18:30.518942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.519097 (+   155us) service_pool.cc:224] Handling call
0709 13:18:30.519521 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.681435 (+161914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.561861 (+880426us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540039d854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:31.562109 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370308864}
I20260709 13:18:31.760818 10199 rpcz_store.cc:276] 0709 13:18:30.544621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.770521 (+225900us) service_pool.cc:224] Handling call
0709 13:18:31.714745 (+944224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:31.744855 10054 pending_rounds.cc:85] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:18:31.713253 14458 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.768532 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.763089 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.755023 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1914) took 1210 ms. Trace:
I20260709 13:18:31.755419 10204 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:31.687355 14769 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.782678 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:31.685801 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.758603 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2080) took 1206 ms. Trace:
I20260709 13:18:31.712940 14682 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.686435 14713 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.718444 14742 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:31.795945 14750 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:31.796726 14742 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.797210 14742 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.770464 10054 pending_rounds.cc:107] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 11.8
W20260709 13:18:31.773554 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.779999 10203 rpcz_store.cc:276] 0709 13:18:30.544952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.753314 (+1208362us) service_pool.cc:224] Handling call
0709 13:18:31.753682 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.754746 (+  1064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.754955 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:31.776947 13574 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.720320 14703 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:31.781272 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.778831 10201 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:31.798882 13574 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:31.797958 14551 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.006s
I20260709 13:18:31.718485 14716 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.801920 10201 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:31.802582 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1923) took 1129 ms. Trace:
I20260709 13:18:31.706946 10126 rpcz_store.cc:276] 0709 13:18:30.014246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.207755 (+193509us) service_pool.cc:224] Handling call
0709 13:18:30.229430 (+ 21675us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:30.241464 (+ 12034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.242461 (+   997us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:30.284486 (+ 42025us) log.cc:844] Serialized 52 byte log entry
0709 13:18:30.405999 (+121513us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:30.453391 (+ 47392us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.471771 (+ 18380us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.472022 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.472119 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.301434 (+829315us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b1800383980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:31.304003 (+  2569us) spinlock_profiling.cc:243] Waited 8.81 ms on lock 0x7b1800383980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:31.608258 (+304255us) raft_consensus.cc:1685] finished
0709 13:18:31.608282 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.608469 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17718,"prepare.queue_time_us":24951,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":105,"raft.queue_time_us":18624,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":16591,"spinlock_wait_cycles":76782464,"thread_start_us":82552,"threads_started":1,"wal-append.queue_time_us":145034}
I20260709 13:18:31.802919 10201 rpcz_store.cc:276] 0709 13:18:30.673120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.801199 (+1128079us) service_pool.cc:224] Handling call
0709 13:18:31.801554 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.802323 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.802519 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:31.783500 10128 rpcz_store.cc:276] 0709 13:18:30.552746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.633783 (+ 81037us) service_pool.cc:224] Handling call
0709 13:18:30.634258 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.634679 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.634709 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.655192 (+ 20483us) log.cc:844] Serialized 26 byte log entry
0709 13:18:30.758744 (+103552us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:30.758816 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.758931 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.758351 (+999420us) raft_consensus.cc:1685] finished
0709 13:18:31.758373 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.758535 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1428,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":24407,"spinlock_wait_cycles":1920,"thread_start_us":67762,"threads_started":1,"wal-append.queue_time_us":226245}
W20260709 13:18:31.763254 14643 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.009s	user 0.006s	sys 0.008s
I20260709 13:18:31.807557 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.771440 14744 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:18:31.762548 14737 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.561s	user 0.006s	sys 0.005s
I20260709 13:18:31.666620 14759 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.788831 14763 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:31.800402 14551 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.837558 14551 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.802806 14716 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.841780 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.843068 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:31.843428 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:31.843883 10128 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:18:31.844234 10124 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:31.844678 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.846611 14752 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.783768 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1907) took 1241 ms. Trace:
I20260709 13:18:31.857738 10196 rpcz_store.cc:276] 0709 13:18:30.542650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.685301 (+1142651us) service_pool.cc:224] Handling call
0709 13:18:31.783691 (+ 98390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:31.853533 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:31.797777 10054 raft_consensus.cc:2889] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:31.797394 10131 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:31.872423 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.873330 10131 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:31.873704 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 368093us
I20260709 13:18:31.874866 10123 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:31.876973 10196 raft_consensus.cc:1217] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Deduplicated request from leader. Original: 10.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:18:31.876653 14721 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:31.877465 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.877604 10196 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:18:31.878235 14721 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.878402 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3060) took 1178 ms. Trace:
I20260709 13:18:31.879130 10196 rpcz_store.cc:276] 0709 13:18:30.700166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.858583 (+1158417us) service_pool.cc:224] Handling call
0709 13:18:31.876852 (+ 18269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.878106 (+  1254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.878330 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:31.881029 14705 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.881829 14740 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:31.881646 14705 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.882320 14740 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:31.884680 14749 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:31.885349 14749 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.887828 14740 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.889441 14740 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:31.893349 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.893141 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.895079 14542 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:18:31.895354 14772 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:31.898406 14772 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.898783 14772 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:31.900373 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.901155 10047 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:31.848585 14737 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.903779 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.904558 10051 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:31.905795 14737 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.908464 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.910581 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:31.911674 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.912577 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:31.912904 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:31.914098 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.915548 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:31.916172 10123 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:31.917073 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2184) took 1401 ms. Trace:
I20260709 13:18:31.917517 10053 rpcz_store.cc:276] 0709 13:18:30.515770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.870476 (+354706us) service_pool.cc:224] Handling call
0709 13:18:30.870901 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.087289 (+216388us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:31.647046 (+559757us) log.cc:844] Serialized 14 byte log entry
0709 13:18:31.758839 (+111793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:31.758895 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:31.758942 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.916814 (+157872us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54001b8d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:31.916872 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.917012 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1707,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":13570,"spinlock_wait_cycles":489595648,"thread_start_us":111345,"threads_started":1,"wal-append.queue_time_us":138940}
I20260709 13:18:31.914052 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.670147 14741 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.808331 10202 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:31.968230 10203 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:18:31.970531 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78397.5us
I20260709 13:18:31.808423 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:31.769687 14680 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.768169 14767 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:31.811913 10049 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:31.753989 14738 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:18:31.817039 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.979511 10122 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:31.819526 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3046) took 1464 ms. Trace:
I20260709 13:18:31.786965  9949 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 9, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:18:31.819911 14748 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.820946 14763 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.983268 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2077) took 1445 ms. Trace:
I20260709 13:18:31.980029 10200 rpcz_store.cc:276] 0709 13:18:30.355543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.453372 (+ 97829us) service_pool.cc:224] Handling call
0709 13:18:30.468700 (+ 15328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.354692 (+885992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.819208 (+464516us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:31.819445 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994964352}
I20260709 13:18:31.983892 10122 rpcz_store.cc:276] 0709 13:18:30.537331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.538087 (+   756us) service_pool.cc:224] Handling call
0709 13:18:31.816856 (+1278769us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:31.983182 (+166326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2594688}
I20260709 13:18:31.805063 14718 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.987962 14718 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.988526  9949 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.990540 14718 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.825116  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:31.826000 14597 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.826704 14750 consensus_queue.cc:1243] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:31.760331 14761 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.827919 14744 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:31.828552 14589 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.732s	user 0.007s	sys 0.006s
I20260709 13:18:32.009490 14589 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.010514 10050 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:32.010965 10050 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:32.011850 14682 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.012351 14589 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:31.830466 14379 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.012756 14749 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.784288 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1920) took 1187 ms. Trace:
I20260709 13:18:32.013790 10205 raft_consensus.cc:2368] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:32.013736 10204 rpcz_store.cc:276] 0709 13:18:30.596558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.686840 (+1090282us) service_pool.cc:224] Handling call
0709 13:18:31.784216 (+ 97376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:32.015288 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.016829 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.017529 10204 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:32.017264 14712 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.018646 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:31.854646 13574 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.856263 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:32.031217 10204 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:32.031376 13574 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:31.860456 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:32.033280 10129 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:31.861858 14407 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:31.862736 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:31.863632 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.036545 10130 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:32.036368 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 180264us
W20260709 13:18:31.864070 14734 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.906s	user 0.003s	sys 0.000s
I20260709 13:18:31.865540 14633 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:32.049047 13574 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:31.866763 10125 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:31.846521 14612 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:31.870913 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:31.871321 10899 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:31.779961 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.053369 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.045221 14597 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:31.950192 10124 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:31.918671 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:32.057900 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 139296us
I20260709 13:18:31.951097 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3063) took 1250 ms. Trace:
I20260709 13:18:31.951339 14204 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:31.952078 14739 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.000s
I20260709 13:18:31.952549 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:31.956873 14759 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:31.963207 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109778us
I20260709 13:18:31.962080 10123 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:31.966527 10169 leader_election.cc:400] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:31.974119 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1922) took 1334 ms. Trace:
W20260709 13:18:31.974434 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 167442us
I20260709 13:18:31.969684 14446 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:31.812048 14643 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:31.987208 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1906) took 1444 ms. Trace:
I20260709 13:18:31.997478 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.003988 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.008116 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2173) took 1508 ms. Trace:
I20260709 13:18:31.851272 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:32.020331 10050 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.022424 10205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1909) took 1479 ms (client timeout 1814 ms). Trace:
I20260709 13:18:31.980402 14680 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:31.979363 14737 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.037657 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:32.035445 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.998653 14378 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.040410 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:32.042951 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:32.044112 14407 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.044601 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:31.992702 14763 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.047626 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:31.806082 10201 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:32.092360 10048 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:32.055486 10203 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:32.058825 10196 rpcz_store.cc:276] 0709 13:18:30.701128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.880023 (+1178895us) service_pool.cc:224] Handling call
0709 13:18:31.880387 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.923322 (+ 42935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.950688 (+ 27366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:31.950750 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:31.950801 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.950877 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.951028 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:32.064167 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:32.078979 10197 rpcz_store.cc:276] 0709 13:18:30.542435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.127458 (+585023us) service_pool.cc:224] Handling call
0709 13:18:31.986882 (+859424us) spinlock_profiling.cc:243] Waited 90.4 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:31.987072 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207864064}
I20260709 13:18:32.079859 10200 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:32.080457 10205 rpcz_store.cc:269] 0709 13:18:30.543089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.870488 (+327399us) service_pool.cc:224] Handling call
0709 13:18:32.022127 (+1151639us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400217ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:32.022341 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310828800}
I20260709 13:18:32.074061 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.080122 10127 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:32.099097 10204 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:32.092332 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 1924) took 1394 ms. Trace:
I20260709 13:18:32.067987 10169 leader_election.cc:403] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:32.099818 10201 rpcz_store.cc:276] 0709 13:18:30.698234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.805386 (+1107152us) service_pool.cc:224] Handling call
0709 13:18:31.805743 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.092017 (+286274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.092261 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:32.068336 10202 rpcz_store.cc:276] 0709 13:18:30.633448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.798686 (+1165238us) service_pool.cc:224] Handling call
0709 13:18:31.799040 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.967518 (+168478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.967789 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:32.070231 10130 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:32.071098 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:32.101346 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 115067us
I20260709 13:18:32.063854 14682 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:32.075527 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 123091us
I20260709 13:18:32.113685 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.092916 10055 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:32.064826 10049 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:18:32.116707 10049 pending_rounds.cc:85] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 5
I20260709 13:18:32.117031 10049 pending_rounds.cc:107] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:32.117308 10049 raft_consensus.cc:2889] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.102743 13574 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:32.079806 10054 rpcz_store.cc:276] 0709 13:18:30.499874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.500084 (+   210us) service_pool.cc:224] Handling call
0709 13:18:30.514518 (+ 14434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.880810 (+1366292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.007715 (+126905us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54003cfad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:32.008035 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648130432}
I20260709 13:18:32.082612 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:32.113126 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 146796us
I20260709 13:18:32.091529 10050 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:32.092291 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49530.5us
I20260709 13:18:32.073699 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:32.111714 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.112509 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:31.998608 14703 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.124969 10899 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:32.124811 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:32.126044 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.052765 14742 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.184898 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 91915.1us
I20260709 13:18:32.193208 14759 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:32.113093 14458 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:32.205299 14759 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:32.205688 14458 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.206001 14759 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:32.206254 14458 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:32.090591 14750 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.173079 14705 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.080456 14612 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:32.090404 14095 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:18:32.209751 14742 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:32.075730 14739 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.122242 14204 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:32.210668 14204 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.122532 14756 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:32.211055 14204 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.211406 14756 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.211906 14756 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.212342 14739 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.183593 14716 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:32.213330 14716 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.111835 14744 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.080018 14709 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 13:18:32.161782 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:32.217752 10048 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:32.171285 14446 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:32.164518 14712 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:32.100291 14737 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:32.220338 14737 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.126538 14552 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.197069 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:32.214367 14716 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:32.199528 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 162007us
I20260709 13:18:32.226285 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:32.227653 14619 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:32.226322 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.232184 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:32.184515 14779 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:18:32.236337 14407 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:32.236687 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.238260 10197 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:32.167719 14761 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.192821 14680 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.187014 14772 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.244966 14772 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:32.172089 14741 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.250427 10199 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:32.262102 14767 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:32.269913 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:32.322491 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52679.8us
I20260709 13:18:32.114035 14378 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.284361 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 172774us
I20260709 13:18:32.329689 14767 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:32.287693 14752 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:32.288779 14709 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:32.331470 14709 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:32.332880 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.334374 14752 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.086021 14589 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.197s
W20260709 13:18:32.296742 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89879.2us
I20260709 13:18:32.291720 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.296689 14756 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.299372 14458 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:32.304814 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 107848us
I20260709 13:18:32.305663 13574 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:32.311787 14682 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:32.344512 13574 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.345052 13574 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:32.312978 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 188228us
I20260709 13:18:32.080164 14423 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.234s
I20260709 13:18:32.319222 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.348155 10203 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:32.348578 10205 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.351774 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.320192 14742 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.285032 14718 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.358554 14682 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.332803 14709 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.334534 14767 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:32.290320 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 216708us
W20260709 13:18:32.303689 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 183710us
I20260709 13:18:32.336480 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.336850 10197 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:32.337810 10055 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
W20260709 13:18:32.339581 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84738us
W20260709 13:18:32.131639 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.990684  9949 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 15, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 15 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.346606 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:32.349234 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.358161 14756 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:32.359686 14682 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:32.387100 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.393451 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.389799 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.358142 14458 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.367506 14759 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.395723 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:32.358150 14378 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:32.371095 14407 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:32.376215 14722 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:18:32.409891 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60762.9us
I20260709 13:18:32.395951 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:32.411310 10055 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:32.397461 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:32.398136 10053 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:32.399174 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52623us
I20260709 13:18:32.381336 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:32.413719 10047 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.414378 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.416106 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:32.416594 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.417893 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.418561 10128 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:18:32.388334 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51921.7us
I20260709 13:18:32.401585 10200 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:32.402381 10198 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:32.409861 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:32.420658 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.400822 10050 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:18:32.422526 10050 pending_rounds.cc:85] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:18:32.422945 10050 pending_rounds.cc:107] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:18:32.422845 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:32.423564 10050 raft_consensus.cc:2889] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.429942 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.433949 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.434791 10124 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:32.406908 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:32.430867 10130 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:32.448572 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.455483 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.456533 10196 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:32.459635 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 107963us
I20260709 13:18:32.460986 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.453042 14137 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:32.453974 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:32.487854 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:32.492352 10047 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:32.495802 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.496532 10198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3061) took 1796 ms (client timeout 1873 ms). Trace:
W20260709 13:18:32.497025 10198 rpcz_store.cc:269] 0709 13:18:30.700481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.953600 (+253119us) service_pool.cc:224] Handling call
0709 13:18:32.496283 (+1542683us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002ddcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:32.496454 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2489810176}
W20260709 13:18:32.502725 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88411.5us
I20260709 13:18:32.505901 10900 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:32.506148 10124 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:32.509692 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:32.516049 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:32.542119 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 135337us
I20260709 13:18:32.546237 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:32.547848 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86979.9us
I20260709 13:18:32.363399 14742 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.552456 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.553691 10124 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:32.555703 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46169.3us
I20260709 13:18:32.453504 14619 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:32.398336 14761 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:32.564412 10124 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 8.
I20260709 13:18:32.453438 14718 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:32.559859 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:32.619971 10200 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:32.557797 14785 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:18:32.498775 14740 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:32.595038 10197 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:18:32.600064 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:32.605568 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:32.629628 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:32.608412 10055 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
W20260709 13:18:32.633505 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87388.2us
I20260709 13:18:32.557986 14446 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:32.519573 14468 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.558158 14772 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:32.598596 13574 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.574780 14716 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:32.613997 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.458299 14756 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:32.429276 14709 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:32.648185 13574 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:32.476114 14737 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.482304 10130 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
W20260709 13:18:32.641776 14775 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:18:32.560791 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:32.618919 10899 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:32.463263 14597 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:32.668056 14597 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:32.668546 14772 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:32.669342 14597 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.669698 14772 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.540071 14783 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:18:32.644914 14552 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:32.518276 14706 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:18:32.610035 10203 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:32.519484 14752 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:32.671864 14446 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:32.672304 14752 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.673022 14446 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.478211 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:32.558849 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.675616 10122 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.643581 10124 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:32.682301 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:32.644497 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:32.620735 10126 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:18:32.636808 14712 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:32.661389 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:32.698215 10130 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.627442 14416 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.664395 10123 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:32.591614 10204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:18:32.653475 14763 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.002s
I20260709 13:18:32.674815 14752 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:32.700426 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56026.9us
I20260709 13:18:32.700917 10124 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:32.618276 10196 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:32.708717 10204 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.704986 10200 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:32.733836 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:32.734946 14754 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:32.391789  9949 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.747391 10122 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:32.750622 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:32.751961 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 302244us
I20260709 13:18:32.751942 10052 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:32.662370 14756 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.753165 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:32.747886 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.756448 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:32.752085 10130 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:32.641394 14718 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:32.839609 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 86560.5us
I20260709 13:18:32.846148 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.746687 14712 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.645226 14767 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.870608 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 122923us
W20260709 13:18:32.843137 14204 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.631s	user 0.004s	sys 0.008s
I20260709 13:18:32.837304 14751 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.826218 14703 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:32.736615 14416 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.737175 14763 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:32.867180 14779 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:18:32.896257 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49074.6us
I20260709 13:18:32.843791 14732 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:32.869834 14269 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:32.898438 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:32.900844 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 152886us
I20260709 13:18:32.921931 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:32.879674 14716 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.869270 14619 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:32.933142 10052 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:32.930440 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:32.885746 14633 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.886809 14791 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:32.909958 14759 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:32.950196 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.917297 14795 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:32.923410 14378 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:32.919293 14732 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.920089 14269 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:32.956067 14792 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 13:18:32.976413 14784 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:18:32.940011 14796 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.050247 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.947491 14643 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:32.977644 14742 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:33.056632 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 112414us
I20260709 13:18:33.068763 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:33.069635 10205 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.688s	user 0.010s	sys 0.001s
I20260709 13:18:33.070792 10205 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:32.977329 14806 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:33.071816 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.890614 14204 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:33.052421 14794 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:18:33.034941 14795 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:33.048813 14721 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.976709 14712 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.051769 14759 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:33.108021 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57880.2us
I20260709 13:18:33.166359 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:32.957757 14633 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:32.924458 14756 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:32.980134 14791 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.060709 14682 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.158267 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.081141 14796 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.104840 14737 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.107254 14597 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:32.927228 14716 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.113876 14795 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:33.153532 14742 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.156842 14790 raft_consensus.cc:2764] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:18:33.155809 14806 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:18:33.163415 14732 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.159595 14619 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.160491 14552 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:32.979602  9949 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.168646 14759 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.151099 14721 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.172690 14793 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.179143 14643 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:33.235972 14718 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.220844 14597 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:33.239298 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2205) took 1272 ms. Trace:
I20260709 13:18:33.158202 14269 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.240636 14269 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:33.240510 14597 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.243369 14793 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.246536 14793 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:33.247277 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 747625us
I20260709 13:18:33.229852 14767 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:18:33.243490 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 583070us
W20260709 13:18:33.230062 14709 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.003s
I20260709 13:18:33.229924 14619 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.251955 14619 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.227207 14749 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:33.254289 14709 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.254509 14748 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.255087 14709 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.255331 14748 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.230058 14759 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:33.256431 14814 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:33.257697 14709 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:33.258211 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 658172us
I20260709 13:18:33.257855 14552 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.259902 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.260663 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:33.261787 10049 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:33.250945 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:33.240038 10051 rpcz_store.cc:276] 0709 13:18:31.967320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.967465 (+   145us) service_pool.cc:224] Handling call
0709 13:18:31.981719 (+ 14254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:32.286154 (+304435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:32.286191 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:32.286825 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:18:32.566232 (+279407us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:32.566302 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.238493 (+672191us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54002369d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:33.238551 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:33.238666 (+   115us) raft_consensus.cc:1685] finished
0709 13:18:33.238679 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.239166 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91856,"raft.queue_time_us":94168,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":487567616,"thread_start_us":278839,"threads_started":1,"wal-append.queue_time_us":461945}
W20260709 13:18:33.266510 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 344604us
I20260709 13:18:33.268992 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:33.229182 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 145044us
I20260709 13:18:33.270885 10095 leader_election.cc:400] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:33.272336 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.273119 10131 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:33.274415 10130 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:33.271311 10095 leader_election.cc:403] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:33.284309 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.192098 14682 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.254421 14721 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:33.302588 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51731.7us
I20260709 13:18:33.226806 14807 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:33.321184 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50421.9us
W20260709 13:18:33.328667 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68855.8us
I20260709 13:18:33.211997 14742 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.332357 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:33.327346 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.295439 14680 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:33.208952 14809 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 13:18:33.284528 14763 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.341922 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:33.349843 10019 leader_election.cc:400] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12.
I20260709 13:18:33.352408 10019 leader_election.cc:403] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:18:33.278853 14749 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:18:33.303632 14748 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.319494 14468 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:18:33.344754 14137 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.850s	user 0.013s	sys 0.009s
I20260709 13:18:33.365561 10049 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:33.360054 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:33.329699 14718 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.369820 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:33.393733 14137 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:33.424695 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54984.6us
W20260709 13:18:33.370463 10095 connection.cc:256] RPC call timeout handler was delayed by -2.0489021719999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 13:18:33.330191 14752 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:33.330040 14740 raft_consensus.cc:2764] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:18:33.311502 14378 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:33.341804 14721 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:33.320007 14619 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:33.348799 14737 raft_consensus.cc:2764] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:18:33.354285 14703 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:18:33.403230 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119011us
W20260709 13:18:33.419626 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 253360us
W20260709 13:18:33.428387 10095 connection.cc:256] RPC call timeout handler was delayed by -1.6997021719998884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19440000000000002s for a response
I20260709 13:18:33.399590 14763 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:33.400368 14793 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:33.430490 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70565.1us
I20260709 13:18:33.432819 14709 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:33.430109 14718 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.433243 14423 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:33.433938 10899 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:33.403949 14468 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:33.403311 14748 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12.
I20260709 13:18:33.413692 14204 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.446934 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:33.289860  9949 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 9 to 13, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 13 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:33.448187 14721 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.473464 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:33.474808 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 125109us
I20260709 13:18:33.493894 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:33.469055 14812 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:18:33.461092 14458 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:18:33.491760 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:33.491421 10095 connection.cc:256] RPC call timeout handler was delayed by -1.5791021719999208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 13:18:33.483538 14796 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.485883 14619 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.506930 10896 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:33.526317 14619 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.528753 14791 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.512023 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.477185 10894 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:33.513319 14643 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.530838 14643 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:33.531747 14823 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:18:33.533420 14643 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:33.518107 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.540869 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.514213 14721 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:33.541318 14763 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:33.542471 14763 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.544013 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2103) took 1035 ms. Trace:
I20260709 13:18:33.349601 14790 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:18:33.544521 10125 rpcz_store.cc:276] 0709 13:18:32.508450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.509214 (+   764us) service_pool.cc:224] Handling call
0709 13:18:32.558365 (+ 49151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.657305 (+ 98940us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:32.836053 (+178748us) log.cc:844] Serialized 14 byte log entry
0709 13:18:33.252800 (+416747us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b180016db60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:18:33.280593 (+ 27793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:33.280654 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:33.280712 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.543708 (+262996us) spinlock_profiling.cc:243] Waited 98.6 ms on lock 0x7b5400354154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:33.543766 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.543940 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45772,"prepare.queue_time_us":260073,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":12144,"raft.queue_time_us":32659,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":9540,"spinlock_wait_cycles":240169216,"thread_start_us":218930,"threads_started":2,"wal-append.queue_time_us":229289}
I20260709 13:18:33.548171 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:33.547560  9949 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 11, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 11 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:33.549181 14741 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.549805 14741 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.551241 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.551553 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:33.552438 10205 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:33.552305 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:33.552598 14741 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.519497 10892 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:33.524863 14204 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.477864 14378 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.553712 14204 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.560931 14204 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.553354 14741 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:33.562529 14741 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:33.529176 14619 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.517860 14682 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.537194 14718 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.591529 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:33.592417 10124 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:33.592545 14741 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.595484 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:33.596426 10899 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:33.596629 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.597275 10126 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:33.597862 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:33.519801 10095 connection.cc:256] RPC call timeout handler was delayed by -1.5152021720000448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 13:18:33.539369 14416 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.553123 10200 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:33.600270 10095 connection.cc:256] RPC call timeout handler was delayed by -1.0157021719999193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27040000000000003s for a response
I20260709 13:18:33.564385 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.601018 10095 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.601346 10123 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:33.518752 14831 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.553850 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.604627 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:33.554153 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.605365 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.605728 10205 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:33.554559 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.607249 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.607566 10196 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:33.565712 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.588304 14737 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:33.588587 14468 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:33.589722 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77842.5us
I20260709 13:18:33.609150 10197 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:33.610405 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.611837 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.590368 14835 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:33.602892 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.605052 10203 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:33.579310 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.613816 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.620119 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:33.554811 14820 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:18:33.623301 10124 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:33.627897 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.515165 14423 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:33.631680 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.635278 10127 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:33.632457 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.632607 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.503923 14793 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.615252 14752 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:33.621178 14718 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:33.639217 14737 consensus_queue.cc:1243] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:18:33.639819 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.641170 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.642757 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.644363 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:33.646313 14633 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.621086 14831 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.647526 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.648514 10198 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:33.649628 14831 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.627374 14458 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:33.650393 14796 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:33.650781 14458 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:33.651965 14458 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.639266 14718 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:33.653059 14718 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:33.597383 14095 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:33.656138 14095 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.554910 14742 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:33.555466 14709 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:33.665236 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.668437 10198 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:33.659842 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:33.549672  9949 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.647631 14829 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:33.650822 14796 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.621470 14790 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:33.627025 14682 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:33.664086 14740 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.604439 14619 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.671099 10095 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:33.668248 14752 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:33.684384 10096 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.545s (SENT)
I20260709 13:18:33.722095 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:33.628549 14791 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:33.663129 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.663902 14737 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:33.696760 14796 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:33.702744 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.706545 10053 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:33.708911 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.709115 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.576860 14759 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:33.715842 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 203354us
I20260709 13:18:33.704386 14137 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:33.664822 14809 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:33.718072 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.745947 10052 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:33.674690 10200 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:33.718369 10198 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:18:33.665066 14095 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.668162 14835 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:33.723537 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63862.4us
I20260709 13:18:33.724915 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:33.725236 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62244.8us
I20260709 13:18:33.698686 10205 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:33.714826 14703 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:33.755751 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53119.6us
I20260709 13:18:33.714566 14682 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.741760 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.739459 10049 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:33.739024 10047 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:33.753252 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:33.761160 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.721761 14752 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:33.763561 14763 raft_consensus.cc:2764] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:18:33.765069 14423 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.781075 10129 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:33.754045 14703 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.760202 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.735803 14542 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:33.743837 14680 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.720595 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:33.750697 14716 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:33.798494 10893 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:33.799708 14759 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.768519 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.793568 14619 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:18:33.759543 14709 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.801548 14619 consensus_queue.cc:1243] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:33.802160 14619 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:33.802481 14619 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:33.793987 14795 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.004s
I20260709 13:18:33.771194 10200 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:18:33.725723 14829 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:33.778573 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.735814 14767 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:33.805853 14763 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.806613 14763 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:33.806632 14767 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:33.810230 14767 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:33.782457 10126 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:33.810642 14767 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:33.792722 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51647.7us
I20260709 13:18:33.812366 14793 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.794229 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.816123 14742 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:33.803920 14795 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.816656 14742 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.818493 14742 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.819170 14795 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:33.775740 14752 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.815635 14619 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.820765 14619 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:33.784205 14597 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:33.816195 14763 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:33.833318 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.777043 14682 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:33.834172 10056 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:33.831537 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.845996 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:33.846354 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:33.847388 10203 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:33.745687 14137 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:33.842079 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:33.848359 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.854079 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:33.859354 10122 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:33.850657 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:33.838546 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:33.867962 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.865319 10056 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:18:33.881803 10124 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:33.885525 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.889539 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.889665 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.869042 10054 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:33.892472 10124 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:33.894164 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.895462 10050 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:33.896754 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.898497 10050 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:33.834412 14680 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:33.784853 14589 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:33.900978 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62556.2us
I20260709 13:18:33.935932 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.902760 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:33.863034 14761 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.218s	user 0.009s	sys 0.007s
I20260709 13:18:33.904662 10198 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:18:33.905498 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.906877 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:33.939558 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.940515 10131 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:33.910782 14741 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.976812 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:33.910380 14446 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:33.842331 14542 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:33.925449 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:33.862535 14137 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.931425 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:33.904134 14843 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.137s	user 0.004s	sys 0.000s
I20260709 13:18:33.937232 10056 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:33.831432 14718 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:33.910727 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.002107 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.002938 10200 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:33.942973 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.977761 10054 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:33.924701 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.002518 10198 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:18:34.005128 10127 raft_consensus.cc:2368] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:33.987289 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:33.988265 10202 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:33.989878 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:33.996493 10196 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:34.006531 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:33.938709 14761 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.010s	sys 0.007s
I20260709 13:18:34.010456 10204 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:18:33.996214 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.012120 10055 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:34.013037 10131 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:34.012159 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.000150 10199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:34.012044 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.014562 10199 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:33.999900 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 273208us
I20260709 13:18:34.014789 10053 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:34.014956 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:34.017318 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80870.2us
I20260709 13:18:34.019260 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.020056 10128 raft_consensus.cc:2368] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.023398 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:33.998469 14552 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:34.025199 14552 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.027851 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.028836 10055 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:33.990121 14706 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:34.031728 14706 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.032696 10900 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:33.989384 14446 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.052496 10200 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:34.043049 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:34.043196 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.062938 10052 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:33.942761 14721 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:34.049116 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.048993 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:34.068324 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.021368 14378 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.074625 14378 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.077206 14378 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:34.079216 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.796s (SENT)
I20260709 13:18:34.079898 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:34.080868 14597 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:34.081424 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.614s (SENT)
I20260709 13:18:34.081990 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:34.082929 14597 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.083246 14739 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:34.085637 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.087037 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:34.087278 14815 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:34.088389 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.088606 14619 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:34.089640 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.090516 14737 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.091104 14737 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.092782 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:34.093302 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.094523 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.094730 14759 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:34.095293 10128 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.096290 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.096625 14742 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.099485 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.020475 14767 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.101238 10131 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:34.102944 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.104236 10204 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:18:34.104725 10204 pending_rounds.cc:85] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 10
I20260709 13:18:34.104609 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.105885 10124 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.106159 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.108922 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:33.941880 14589 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.088819 10198 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:34.095197 14759 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:34.105327 10204 pending_rounds.cc:107] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:18:34.108116 10056 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:34.109548 10125 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:34.117297 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.111750 10204 raft_consensus.cc:2889] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.012395 14761 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:34.115924 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.112324 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.110255 10196 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:34.115414 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:34.065938 14816 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:18:34.134229 10128 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:34.126945 10203 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:34.132230 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.133168 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.139698 10056 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:33.995146 14741 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.071153 14446 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.140769 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.106463 14716 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:34.004354 14542 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.153157 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.118254 14759 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:34.155166 10196 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:34.157240 10128 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:34.158262 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.162737 10056 raft_consensus.cc:2376] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12. Re-sending same reply.
I20260709 13:18:34.164458 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.165795 10056 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:34.167102 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.182860 10204 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:34.169699 14848 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:34.138327 14643 raft_consensus.cc:993] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:34.110962 14742 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:34.203742 10096 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.878s (SENT)
I20260709 13:18:34.204594 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:34.207851 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 158960us
W20260709 13:18:34.209233 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.156407 14716 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.190307 10125 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:34.191561 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:34.197873 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 197957us
I20260709 13:18:34.198182 10050 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:34.199636 10126 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:34.212848 10096 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.944s (SENT)
I20260709 13:18:34.253929 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:34.174255 14748 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:34.162779 10124 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:34.233238 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.234009 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:34.176446 14767 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.213510 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.232429 10127 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:34.234751 14716 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:34.219271 14742 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:34.215804 14703 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.209597 14597 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:34.316926 10056 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 7.
I20260709 13:18:34.318655 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:34.324208 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:34.336113 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.042s (SENT)
I20260709 13:18:34.330844 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.339349 10056 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:34.340742 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.342062 10056 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:34.352393 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 119316us
I20260709 13:18:34.353943 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.305431 14619 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.356477 14612 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:18:34.382136 14589 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.384265 14612 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:34.332291 14793 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:18:34.385516 14793 consensus_queue.cc:1243] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:18:34.390204 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66072.7us
I20260709 13:18:34.396761 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.370028 14633 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:34.364951 14824 raft_consensus.cc:1081] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:34.255717 14761 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.375159 14718 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:34.416054 14857 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:18:34.426187 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.427420 10053 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:34.416781 10020 leader_election.cc:400] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
W20260709 13:18:34.410265 14844 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
I20260709 13:18:34.401618 14797 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:34.408836 14416 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:34.410841 14706 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.432678 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.440097 10047 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:34.386619 14680 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.402559 14552 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.411010 14589 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.414675 14633 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.427963 10020 leader_election.cc:403] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:34.438676 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:34.438923 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 305827us
I20260709 13:18:34.414446 10127 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:18:34.437350 14796 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.000s
I20260709 13:18:34.452176 10019 leader_election.cc:400] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.426069 14742 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:34.436214 14721 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.431386 14737 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.454342 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.438092 14797 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.462345 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.438436 14416 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:34.458582 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61941.1us
I20260709 13:18:34.455780 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:34.463446 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 129566us
I20260709 13:18:34.464823 10053 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:34.465272 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:34.467926 14850 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:18:34.468602 14810 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:34.469136 14859 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:18:34.469210 14810 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:34.469909 14810 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:34.472769 14137 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.464607 14542 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:34.475013 14542 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.475477 14542 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:34.478506 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.452085 14633 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:34.468039 14858 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:18:34.480254 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.488034 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.464236 14416 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:34.488250 10900 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:18:34.488793 10053 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:34.486756 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.488834 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.493069 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:34.493925 10096 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.785s (SENT)
I20260709 13:18:34.464005 14797 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.503605 14680 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.462096 14468 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.498019 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.499183 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.487948 14706 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.453912 14589 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.462433 14741 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.490532 10202 pending_rounds.cc:85] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 11
I20260709 13:18:34.509152 10202 pending_rounds.cc:107] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:18:34.509665 10202 raft_consensus.cc:2889] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.509919 10019 leader_election.cc:400] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.484275 14793 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:18:34.505865 10053 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:34.515462 10056 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:34.458012 14796 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.488852 14772 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:34.519948 10053 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:34.516880 14718 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.494716 14864 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:34.531523 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.539803 10053 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:34.540936 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 493465us
I20260709 13:18:34.517259 14803 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.531955 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:34.531663 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.544559 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:34.533774 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.547439 10047 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:34.519630 14772 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.531538 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.517383 14861 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.537261 14706 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:34.517743 14468 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.555028 14680 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:34.552919 14269 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:34.599594 14269 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:34.600066 14269 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:34.600329 14737 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:34.559250 14795 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:34.553465 14718 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:34.562464 14772 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:34.549964 14803 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.565914 14589 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.565755 14797 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.611003 10203 raft_consensus.cc:606] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.611580 10203 raft_consensus.cc:614] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:34.566849 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:34.612094 14803 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:34.612573 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45836.5us
I20260709 13:18:34.612816 14803 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:34.613406 14803 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:34.612124 14797 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.570767 14841 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:34.579346 10126 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:34.536837 14864 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.615080 14864 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.614454 14803 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.588864 10122 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:18:34.580236 14682 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.002s
I20260709 13:18:34.624917 14682 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:34.598644 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67174.6us
I20260709 13:18:34.627529 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:34.628314 10019 leader_election.cc:400] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.629027 14772 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.620038 14796 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.631614 14682 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.586051 14706 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:34.635636 14706 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.636364 14772 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.643630 10197 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:18:34.620646 14865 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
W20260709 13:18:34.622259 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 76953.9us
I20260709 13:18:34.658414 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:34.550791 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:34.661918 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 111229us
I20260709 13:18:34.629344 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:34.659515 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45623.4us
I20260709 13:18:34.663048 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.669520 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:34.670709 10127 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:18:34.671195 10127 pending_rounds.cc:85] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
I20260709 13:18:34.671429 10127 pending_rounds.cc:107] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:18:34.671334 10197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:34.671885 10127 raft_consensus.cc:2889] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.672184 10197 raft_consensus.cc:606] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.672770 10128 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:34.661012 14807 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.674945 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.675161 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:34.675832 10128 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.676014 10129 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.677474 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:34.679677 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:34.591847 10125 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
W20260709 13:18:34.683724 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54473.2us
I20260709 13:18:34.684748 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.683771 10125 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:34.686146 10125 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:34.704551 10055 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:34.704825 10126 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:18:34.704875 10203 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:34.708122 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.659560 14748 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:34.719686 10019 leader_election.cc:400] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:34.676852 14709 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:34.533907 14742 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.721372 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.695006 14763 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:34.707584 14835 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:34.740100 10048 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:34.733079 10047 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:34.703933 14680 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:34.737589 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.715603 14552 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:18:34.692665 14807 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:34.746750 10129 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:34.730072 10055 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:34.673745 14682 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:18:34.746121 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.753315 10128 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:34.741580 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.661635 14864 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.732240 10125 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.710817 14095 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:18:34.694494 14867 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:18:34.739375 14835 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.717417 14797 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:34.726747 14799 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:18:34.801666 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.818986 10048 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:34.806520 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2127) took 1216 ms (client timeout 1616 ms). Trace:
I20260709 13:18:34.767793 14709 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:34.711024 14810 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:34.819990 10130 rpcz_store.cc:269] 0709 13:18:33.590017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.597601 (+  7584us) service_pool.cc:224] Handling call
0709 13:18:34.806240 (+1208639us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540036b854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:34.806422 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144587520}
I20260709 13:18:34.820436 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:34.821745 10125 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.823459 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.777143 14552 consensus_queue.cc:1243] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:34.824194 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:34.826869 10047 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.828022 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:34.828876 10203 raft_consensus.cc:606] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:34.829427 10203 raft_consensus.cc:614] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:34.778083 14763 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:34.747581 14866 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:34.825004 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113977us
I20260709 13:18:34.784667 14741 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:34.747334 14863 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:18:34.854995 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.856773 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.841567 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:34.857746 10204 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:18:34.823534 14706 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.822783 14137 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.860105 14706 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.860296 14269 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:34.862269 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.860181 14709 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.811075 14680 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.836797 10055 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:34.844245 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:34.878438 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:34.849804 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:34.815397 14835 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:34.841845 10130 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:34.834209 14612 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:34.891752 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.893168 10199 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:34.869913 10048 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
W20260709 13:18:34.832970 14446 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.005s
I20260709 13:18:34.874111 10047 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:34.911199 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.879563 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:34.879357 10055 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.780900 14633 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.939049 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:34.940857 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:34.941706 10199 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:34.920714 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:34.949819 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:34.960662 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:34.961108 10204 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:34.859005 14416 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.862325 14807 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.965369 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:34.966032 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86603.7us
I20260709 13:18:34.963727 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:34.963707 14446 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:34.857319 14864 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:34.975891 10048 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:18:34.988698 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.224s (SENT) [suppressed 21 similar messages]
I20260709 13:18:35.004876 10047 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:18:35.011376 14416 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.895910 14680 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.013356 14633 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:34.882702 14748 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:34.963819 14874 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:18:35.014307 14680 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.015657 10047 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:35.016063 14763 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:35.017607 14716 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.000s
I20260709 13:18:35.018090 14716 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.019215 14732 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:35.019850 14732 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:35.031342 10201 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:18:35.035326 14542 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.559s	user 0.013s	sys 0.000s
I20260709 13:18:35.035914 14542 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.038682 14542 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:35.040390 14269 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:34.935971 14552 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:34.867625 14682 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 13:18:35.050701 10200 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:35.044291 10130 pending_rounds.cc:85] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:18:35.051720 10130 pending_rounds.cc:107] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:18:35.052037 10130 raft_consensus.cc:2889] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.050894 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2241) took 1149 ms. Trace:
W20260709 13:18:35.004911 14759 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.001s
I20260709 13:18:35.065562 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:35.005016 14879 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:35.064453 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:35.059568 10055 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:35.102689 10199 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:35.127359 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 381317us
I20260709 13:18:35.146466 10055 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:35.168567 14807 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.169587 14807 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.172406 14716 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.174094 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:35.174818 10198 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:35.178169 14732 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.050275 14737 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:35.064492 14552 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:35.189499 10198 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:35.049271 14863 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
I20260709 13:18:35.181931 14864 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.186437 14761 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.201232 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2240) took 1299 ms. Trace:
I20260709 13:18:35.106142 14759 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.204696 10053 rpcz_store.cc:276] 0709 13:18:33.901442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.540742 (+639300us) service_pool.cc:224] Handling call
0709 13:18:34.541078 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.541995 (+   917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.542032 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.542627 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.786971 (+244344us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:35.200201 (+413230us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.200597 (+   396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.200712 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.200785 (+    73us) raft_consensus.cc:1685] finished
0709 13:18:35.200804 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.201134 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228,"prepare.queue_time_us":46751,"prepare.run_wall_time_us":49,"raft.queue_time_us":1111,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":7685,"spinlock_wait_cycles":23335168,"thread_start_us":591672,"threads_started":2,"wal-append.queue_time_us":275648}
I20260709 13:18:35.205816 14864 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.234532 10124 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:35.235409 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:35.236160 10053 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:35.206604 14740 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.229125 14866 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.352614 14866 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.252691 14740 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.210816 14761 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.365737 14761 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.253157 14552 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.254451 14867 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:18:35.362429 14740 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.379694 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:35.380963 10054 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:35.382151 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:35.385082 10054 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:35.259635 14716 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:18:35.320164 14881 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:18:35.321301 14853 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.066s	user 0.014s	sys 0.004s
I20260709 13:18:35.322214 14137 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:35.322865 14882 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:35.320273 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:35.393007 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72826.2us
I20260709 13:18:35.396217 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:35.396489 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:35.398075 14882 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:35.398651 14882 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:35.399982 14866 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:35.399668 14882 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:35.327072 14883 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:18:35.345799 14759 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.347734 14446 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:35.350091 14864 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.351326 14807 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.240420 14709 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:35.405880 14709 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.408515 14709 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:35.409025 14709 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.409483 14709 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.317950 14703 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:35.414135 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:35.417830 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:35.419301 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:35.445829 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.465962 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43888.9us
I20260709 13:18:35.426376 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:35.472491 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:35.473389 14835 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:18:35.473707 10049 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:35.475744 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:35.492725 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 244843us
I20260709 13:18:35.493939 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:35.495324 14867 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:35.495970 14867 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:35.497255 14867 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.500437 10054 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:35.479298  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:35.508085 10196 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:35.501897 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:35.509718 10054 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:35.509846 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:35.506814  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:35.525887 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2260) took 1001 ms. Trace:
I20260709 13:18:35.528903 10054 rpcz_store.cc:276] 0709 13:18:34.524664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.524878 (+1000214us) service_pool.cc:224] Handling call
0709 13:18:35.525272 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.525507 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.525536 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.525578 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:35.525625 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.525683 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.525820 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.532439 10054 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:35.535496 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2261) took 1006 ms. Trace:
I20260709 13:18:35.537082 10054 rpcz_store.cc:276] 0709 13:18:34.529225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.531443 (+1002218us) service_pool.cc:224] Handling call
0709 13:18:35.531922 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.535070 (+  3148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.535411 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.503352 14703 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.543001 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2258) took 1026 ms. Trace:
I20260709 13:18:35.549317 10049 rpcz_store.cc:276] 0709 13:18:34.516970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.518446 (+1001476us) service_pool.cc:224] Handling call
0709 13:18:35.542210 (+ 23764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.542473 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.542512 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.542581 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:35.542633 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.542732 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.542906 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.524075 14740 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.564621 10203 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:35.474501 14835 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.052556 10052 rpcz_store.cc:276] 0709 13:18:33.901818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.575874 (+674056us) service_pool.cc:224] Handling call
0709 13:18:34.576315 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.576564 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:34.787259 (+210695us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.048716 (+261457us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.048802 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:35.048880 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.050643 (+  1763us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.050826 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":250257,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":1922,"thread_start_us":468956,"threads_started":2}
I20260709 13:18:35.524155 14721 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 13:18:35.589008 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.593936 10050 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:35.554567 14703 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.568571 14740 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.631248 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:35.637470 10203 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:35.595605 14807 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.596908 14709 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.589993 14861 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:35.651435 10049 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:35.646054 14807 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:35.672744 14887 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:18:35.664783 14892 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.669792 14759 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.685871 14446 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.704941 14407 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:18:35.771461 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 248022us
I20260709 13:18:35.699637 14892 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.711786 14759 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.791360 14759 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.792017 14759 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:35.774853 14882 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:35.797902 14882 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.763183 14446 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.800707 14882 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.801015 14892 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.801565 14446 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:35.784380 14861 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.786976 14761 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:18:35.802577 14861 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:35.792346 10202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.092s	sys 0.184s
W20260709 13:18:35.804067 10202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.092s	sys 0.184s
I20260709 13:18:35.805022 14861 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.783743 14740 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:35.805567 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 548) took 1315 ms. Trace:
I20260709 13:18:35.773108 14407 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.808131 14407 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:35.806254 14468 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:35.811851 14891 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
W20260709 13:18:35.812366 14889 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.146s	user 0.000s	sys 0.003s
W20260709 13:18:35.817873 14898 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:18:35.821012 14703 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:35.820616 14893 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.121s	user 0.000s	sys 0.003s
I20260709 13:18:35.844129 14752 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:35.846237 14796 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:35.847035 14796 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:35.848254 14796 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:35.852910 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2251) took 1370 ms. Trace:
I20260709 13:18:35.853904 10047 rpcz_store.cc:276] 0709 13:18:34.482101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.006287 (+524186us) service_pool.cc:224] Handling call
0709 13:18:35.006728 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.016720 (+  9992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.850754 (+834034us) spinlock_profiling.cc:243] Waited 8.37 ms on lock 0x7b5400136354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:35.851102 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19246336}
I20260709 13:18:35.859803 14796 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:35.867589 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:35.868949 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.876821 10131 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:35.870878 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2148) took 1290 ms. Trace:
I20260709 13:18:35.878145 10124 rpcz_store.cc:276] 0709 13:18:34.580461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.581726 (+  1265us) service_pool.cc:224] Handling call
0709 13:18:34.582001 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.235400 (+653399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.870483 (+635083us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540023b4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:35.870768 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500013440}
I20260709 13:18:35.880072 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.880818 10124 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:35.871389 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.873073 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:35.884394 10128 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:35.885530 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.886193 10128 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:35.864560 10126 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:35.888643 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.892529 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.893744 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:35.900247 10128 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:35.904489 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 549) took 1211 ms. Trace:
W20260709 13:18:35.863970 14878 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.848s	user 0.015s	sys 0.023s
I20260709 13:18:35.833415 14832 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:35.805971 10202 rpcz_store.cc:276] 0709 13:18:34.489684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.489875 (+   191us) service_pool.cc:224] Handling call
0709 13:18:34.490260 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:34.515230 (+ 24970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.515289 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:34.531823 (+ 16534us) log.cc:844] Serialized 52 byte log entry
0709 13:18:34.794198 (+262375us) spinlock_profiling.cc:243] Waited 958 us on lock 0x7b18002eb460. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:18:35.804515 (+1010317us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:35.804865 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 13:18:35.805108 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.805189 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.805248 (+    59us) raft_consensus.cc:1685] finished
0709 13:18:35.805352 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.805502 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4452,"raft.queue_time_us":8679,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":11111,"spinlock_wait_cycles":2202368,"thread_start_us":1010080,"threads_started":1,"wal-append.queue_time_us":1010756}
I20260709 13:18:35.908296 10123 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:35.908825 10047 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:35.905033 10052 rpcz_store.cc:276] 0709 13:18:34.693573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.588876 (+895303us) service_pool.cc:224] Handling call
0709 13:18:35.589393 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.592710 (+  3317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.592757 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.596036 (+  3279us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.891872 (+295836us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:35.891915 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.891988 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.904191 (+ 12203us) raft_consensus.cc:1685] finished
0709 13:18:35.904215 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.904423 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2776,"raft.queue_time_us":3235,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3029,"thread_start_us":248232,"threads_started":1,"wal-append.queue_time_us":286989}
I20260709 13:18:35.913333 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 551) took 1183 ms. Trace:
I20260709 13:18:35.907982 10124 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:35.913805 10047 rpcz_store.cc:276] 0709 13:18:34.729778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.891394 (+1161616us) service_pool.cc:224] Handling call
0709 13:18:35.908622 (+ 17228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.912927 (+  4305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.913147 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.916167 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2265) took 1107 ms. Trace:
I20260709 13:18:35.916695 10047 rpcz_store.cc:276] 0709 13:18:34.809223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.914977 (+1105754us) service_pool.cc:224] Handling call
0709 13:18:35.915613 (+   636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.915828 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.915853 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.915896 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:35.915935 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.915990 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.916100 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.918654 10047 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:35.919409 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 556) took 1107 ms. Trace:
I20260709 13:18:35.919871 10047 rpcz_store.cc:276] 0709 13:18:34.812476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.917903 (+1105427us) service_pool.cc:224] Handling call
0709 13:18:35.918285 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.919078 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.919342 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.921890 10047 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:35.915163 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 138441us
I20260709 13:18:35.924499 14892 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:35.925655 14761 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:35.926352 14761 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:35.914179 14807 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:35.927405 14761 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:35.939575 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 551) took 1297 ms. Trace:
I20260709 13:18:35.940651 10124 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:35.943166 10892 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:18:35.944626 10196 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:35.945614 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 554) took 1175 ms. Trace:
I20260709 13:18:35.940056 10205 rpcz_store.cc:276] 0709 13:18:34.642454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.645022 (+  2568us) service_pool.cc:224] Handling call
0709 13:18:34.645559 (+   537us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:34.646142 (+   583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.646184 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:34.646995 (+   811us) log.cc:844] Serialized 52 byte log entry
0709 13:18:34.838749 (+191754us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:34.861446 (+ 22697us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.861675 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.861772 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.812500 (+950728us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:35.939200 (+126700us) raft_consensus.cc:1685] finished
0709 13:18:35.939330 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.939505 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":497,"prepare.queue_time_us":22980,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":18422,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":12724,"spinlock_wait_cycles":53584640,"thread_start_us":190972,"threads_started":1,"wal-append.queue_time_us":320797}
I20260709 13:18:35.949754 10124 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:35.947316 10095 leader_election.cc:400] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:35.951083 10123 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:35.911824 14832 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:35.953641 10095 leader_election.cc:403] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:18:35.949858 10131 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:35.953673 10052 rpcz_store.cc:276] 0709 13:18:34.769942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.913613 (+1143671us) service_pool.cc:224] Handling call
0709 13:18:35.944837 (+ 31224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.945221 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.945257 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.945305 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:35.945348 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.945412 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.945551 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.954735 10047 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:35.958956 10052 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.961669 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:35.977339 14807 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:35.979228 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:35.986430 10123 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:35.992332 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 548) took 1313 ms. Trace:
I20260709 13:18:35.992861 10049 rpcz_store.cc:276] 0709 13:18:34.678923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.550468 (+871545us) service_pool.cc:224] Handling call
0709 13:18:35.550828 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.652020 (+101192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.992068 (+340048us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b540024ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:35.992261 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227752320}
I20260709 13:18:35.993702 10051 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:18:35.994292 10051 pending_rounds.cc:85] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:18:35.994621 10051 pending_rounds.cc:107] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:18:35.994995 10051 raft_consensus.cc:2889] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.950523 14722 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:18:35.958443 14712 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:18:36.006577 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2257) took 1507 ms. Trace:
I20260709 13:18:36.011006 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.007071 10055 rpcz_store.cc:276] 0709 13:18:34.499463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.147910 (+648447us) service_pool.cc:224] Handling call
0709 13:18:35.184793 (+ 36883us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.303216 (+118423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.303270 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.303334 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:35.303382 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.006263 (+702881us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001b7454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:36.006330 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.006512 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271404416}
I20260709 13:18:36.017889 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.018069 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.021808 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:36.031327 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.032356 10055 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:36.032629 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.038051 10202 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:36.039948 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 527) took 1071 ms. Trace:
I20260709 13:18:36.040373 10130 rpcz_store.cc:276] 0709 13:18:34.968992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.971476 (+  2484us) service_pool.cc:224] Handling call
0709 13:18:35.044065 (+ 72589us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:35.067419 (+ 23354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.067490 (+    71us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:35.068328 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 13:18:35.491836 (+423508us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:35.491905 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.492011 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.005205 (+513194us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b240028f868. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274d9d9c4 00007fa274d9bacb 00007fa274f56dc9 00007fa274f56a55 00007fa274f55fb0 00007fa277bcc488 00007fa277bccbe8 00007fa277ba4398 00007fa277b9f76a 00007fa277b9ebb6
0709 13:18:36.011815 (+  6610us) raft_consensus.cc:1685] finished
0709 13:18:36.011837 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.039867 (+ 28030us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14689,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":49693,"spinlock_wait_cycles":42014976,"thread_start_us":423018,"threads_started":1,"wal-append.queue_time_us":578364}
I20260709 13:18:36.043650 10052 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:35.972963 14916 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10.
I20260709 13:18:36.036540 14752 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:36.083305 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 116379us
I20260709 13:18:36.050758 14739 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:18:36.087646 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.052723 14458 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:18:36.062453 14863 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:18:36.069018 14552 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:18:36.115847 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3097) took 1225 ms. Trace:
I20260709 13:18:36.117385 10200 rpcz_store.cc:276] 0709 13:18:34.890317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.890492 (+   175us) service_pool.cc:224] Handling call
0709 13:18:34.936641 (+ 46149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.051261 (+114620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.115547 (+1064286us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:36.115780 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261485952}
I20260709 13:18:36.094370 14807 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:36.075644 14752 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.114995 14934 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:18:36.138137 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73408.3us
I20260709 13:18:36.083487 14916 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:18:36.110800 14763 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.147218 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.129318 14807 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:36.160022 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43772.6us
I20260709 13:18:36.157712 14737 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.162093 14737 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.159749 14863 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:18:36.163566 14807 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.165325 14807 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:36.166493 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:36.172029 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:36.149932 14416 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:36.145309 14925 raft_consensus.cc:2764] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:18:36.177983 14936 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:36.148909 14703 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.166291 14751 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:36.149869 14934 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:36.181259 14703 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.159235 14763 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.236819 14916 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:36.176858 14742 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:36.274569 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:36.282905 10050 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2264) took 1464 ms (client timeout 1911 ms). Trace:
I20260709 13:18:36.276173 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.284409 10056 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:36.277200 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.286286 10053 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:36.286830 10096 connection.cc:256] RPC call timeout handler was delayed by -1.1043290760001128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 13:18:36.278013 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:36.299450 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 127515us
I20260709 13:18:36.288264 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:36.288372 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111883us
I20260709 13:18:36.300177 10055 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:36.300531 10056 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:36.283751 10050 rpcz_store.cc:269] 0709 13:18:34.808968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.856483 (+ 47515us) service_pool.cc:224] Handling call
0709 13:18:36.204370 (+1347887us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54002e8e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:36.273013 (+ 68643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693151488}
I20260709 13:18:36.301919 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:36.302127 10096 connection.cc:256] RPC call timeout handler was delayed by -1.1011797529999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31930000000000003s for a response
W20260709 13:18:36.281086 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.302932 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:36.303992 10892 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.178798 14936 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.308156 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.322580 10049 raft_consensus.cc:2410] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:18:36.314167 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:36.323730 10202 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:36.324235 10202 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.283516 10052 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:36.320621 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.329242 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.325518 10202 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.341861 10202 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:36.342710 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.343521 10892 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:36.278718 14925 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:36.283129 14742 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:36.338246 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:36.298754 14721 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.329998 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:36.353114 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.329841 14942 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.330550 14866 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:36.365278 14703 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:36.366422 14752 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:36.367633 14926 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:36.376612 14877 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.368s	user 0.007s	sys 0.000s
I20260709 13:18:36.378660 14810 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:36.380151 14932 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:36.380630 14925 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.383409 14763 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:36.391973 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.393836 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.395063 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.420926 10128 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.421674 10126 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:36.403488 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73546.3us
I20260709 13:18:36.310088 14796 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:18:36.346127 14916 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.433741 14916 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.411319 14633 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:36.436254 14633 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.437249 14633 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.398824 14742 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:36.398248 14721 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:36.408056 14807 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:36.405290 10122 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:36.422953 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.407243 10131 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:36.407672 10123 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:36.348601 14737 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.408808 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.413431 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:36.384902 14878 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:36.416913 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.413061 14942 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.482621 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.482949 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.465780 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.485438 10122 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:36.505262 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.516702 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.523874 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.522914 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.532244 10123 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:36.534351 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:36.534504 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47220.6us
I20260709 13:18:36.535199 10126 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:36.537030 10128 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:36.537729 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.540630 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.542441 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.537657 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.545904 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.546422 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.548388 10128 raft_consensus.cc:2376] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12. Re-sending same reply.
I20260709 13:18:36.549505 10131 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:36.549708 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 244239us
I20260709 13:18:36.550323 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.554456 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.556197 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:36.557161 10123 raft_consensus.cc:2376] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10. Re-sending same reply.
I20260709 13:18:36.563818 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.567309 10131 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:36.570693 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.555814 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:36.426416 14926 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.578933 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.531836 14831 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:36.582556 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:36.595088 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65910.4us
I20260709 13:18:36.472623 14643 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:18:36.468994 14807 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.555848 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:36.578840 10123 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:36.598266 14934 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:36.598678 14916 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:36.574210 14871 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:36.601091 10127 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:36.583930 10126 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:36.593976 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.612555 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:36.596868 14870 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.440s	user 0.018s	sys 0.051s
I20260709 13:18:36.613353 10122 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:18:36.615653 14943 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:36.600742 14721 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.618088 14866 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:36.618577 14926 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.581898 14416 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:36.619437 14416 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:36.620059 14416 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.625643 14942 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.573275 14751 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:18:36.628657 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.629166 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.582217 14737 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.599089 14934 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:36.615919 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.600311 14831 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.599051 14938 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:36.611696 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:36.598500 10123 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:36.594506 14763 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:36.614535 10128 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:36.615293 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.609575 14925 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.635911 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:36.601128 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45371.5us
I20260709 13:18:36.642371 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.642874 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.645637 10052 pending_rounds.cc:85] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:18:36.645282 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.637674 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.648753 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.648624 10123 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.625509 14936 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.692550 10047 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:36.692813 10123 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:36.646374 14703 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:36.616406 14870 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.459s	user 0.018s	sys 0.051s
I20260709 13:18:36.619096 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2262) took 2009 ms. Trace:
I20260709 13:18:36.647001 14737 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:36.684437 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.694635 10050 raft_consensus.cc:2368] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:36.652109 14824 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:36.702320 10054 rpcz_store.cc:276] 0709 13:18:34.609887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.550021 (+940134us) service_pool.cc:224] Handling call
0709 13:18:35.550448 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.645735 (+ 95287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.645788 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.655685 (+  9897us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.715309 (+ 59624us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:35.715373 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.618553 (+903180us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b54002355d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:36.618618 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.618680 (+    62us) raft_consensus.cc:1685] finished
0709 13:18:36.618697 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.619010 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":155,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":114,"spinlock_wait_cycles":218084608,"thread_start_us":47705,"threads_started":1,"wal-append.queue_time_us":171828}
I20260709 13:18:36.704496 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.706156 10050 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:36.706549 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.669116 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:36.682013 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.694913 10052 pending_rounds.cc:107] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:18:36.648336 14866 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.636327 14752 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:36.728442 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59438.5us
I20260709 13:18:36.729782 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:36.730613 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.731747 10055 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:36.654939 14740 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:36.674197 14871 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.737905 10202 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:18:36.744714 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86785.6us
I20260709 13:18:36.724918 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:36.748397 10123 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:18:36.748862 10053 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:36.717525 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:36.750037 10054 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:36.711290 10052 raft_consensus.cc:2889] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:36.739975 10049 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:36.738199 10047 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:36.754381 10047 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:18:36.739550 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86857.7us
I20260709 13:18:36.749140 10123 pending_rounds.cc:85] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:18:36.767966 10123 pending_rounds.cc:107] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:18:36.769979 10123 raft_consensus.cc:2889] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:36.775725 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.786072 10050 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:36.792685 14954 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.793644 14954 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.796941 14954 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.803258 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:36.807744 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46034.6us
I20260709 13:18:36.741358 14936 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:36.808928 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.741231 14925 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:36.815666 14936 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:36.705462 14763 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:18:36.669307 14935 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:18:36.709028 14703 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.738777 14752 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.781898 14831 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.823693 14633 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:18:36.826197 14943 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:36.834458 14752 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.822024 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.744100 14866 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:36.844945 14950 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.195s	user 0.000s	sys 0.003s
I20260709 13:18:36.851598 14871 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.800478 14914 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:18:36.800233 14892 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:18:36.788537 14926 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:18:36.810537 14916 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:18:36.801479 14938 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
W20260709 13:18:36.865690 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62543.3us
I20260709 13:18:36.867255 10096 leader_election.cc:400] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:36.867789 10096 leader_election.cc:403] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:18:36.866523 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44628.6us
I20260709 13:18:36.869673 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.849684 14954 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:36.830365 14925 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.871177 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.874246 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.881320 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.882944 14752 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:36.883613 14737 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:36.884359 14890 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.217s	user 0.005s	sys 0.000s
I20260709 13:18:36.886483 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2047) took 2754 ms. Trace:
I20260709 13:18:36.789686 14934 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:36.886940 10048 rpcz_store.cc:276] 0709 13:18:34.131856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.002915 (+871059us) service_pool.cc:224] Handling call
0709 13:18:35.003408 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.003749 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.003791 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.045266 (+ 41475us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.565798 (+520532us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:35.565838 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.565918 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.883644 (+1317726us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:36.886087 (+  2443us) raft_consensus.cc:1685] finished
0709 13:18:36.886113 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.886406 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3402624,"thread_start_us":441349,"threads_started":1,"wal-append.queue_time_us":587729}
I20260709 13:18:36.880466 14936 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:36.889840 14936 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:36.890404 14936 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:36.823649 14945 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:18:36.848466 14824 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:36.892107 14936 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.864177 14926 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:36.882638 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:36.895804 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 545) took 3178 ms. Trace:
I20260709 13:18:36.896262 10051 rpcz_store.cc:276] 0709 13:18:33.717306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.717568 (+   262us) service_pool.cc:224] Handling call
0709 13:18:33.717942 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.995758 (+2277816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.895532 (+899774us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54002e95d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:36.895724 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5232379264}
I20260709 13:18:36.897156 14703 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:36.897733 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.898675 14933 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:36.899159 14933 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.900151 14933 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.902287 14953 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:36.902863 14953 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:36.904394 14953 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:36.905993 14953 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:36.907352 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.908592 14269 raft_consensus.cc:993] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:36.882230 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.908453 14943 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:36.909313 14269 raft_consensus.cc:1081] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:36.910352 14960 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:36.910835 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:36.911837 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:36.915006 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:36.918150 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.920173 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:36.872327 14763 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:36.951341 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53688.4us
I20260709 13:18:36.911978 14793 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:36.954320 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:36.871698 14961 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:36.957446 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.957912 10203 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:36.958675 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:36.959581 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.974500 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.960525 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.880997 14807 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:36.956930 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.874959 14740 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:18:36.915897 14737 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:36.923754 14943 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:36.988406 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 178669us
I20260709 13:18:36.923715 14960 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:36.994748 10200 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:18:36.962327 14793 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:37.000201 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45948.1us
I20260709 13:18:36.984336 14763 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:37.008627 10203 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:18:37.011444 10204 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:18:37.009959 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:37.019546 10131 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:37.024408 10200 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:18:37.027328 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:36.980180 14963 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:37.001837 14938 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:36.984447 14892 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:18:37.053750 10170 leader_election.cc:400] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:37.054430 10170 leader_election.cc:403] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:18:37.057569 10123 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:37.013227 14916 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:37.062357 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.028126 14807 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:37.022264 14763 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:37.070267 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.075795 10123 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:18:37.085927 14956 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
W20260709 13:18:37.088451 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 167835us
I20260709 13:18:37.095477 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:37.001366 14943 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.061934 14916 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:37.095096 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.094772 14793 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.110950 14970 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:37.139139 14970 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:37.139379 14793 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.139678 14970 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.085094 14871 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
W20260709 13:18:37.111882 14742 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.006s
I20260709 13:18:37.142309 14742 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:37.143671 14416 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.006s
I20260709 13:18:37.079516 14763 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:37.144402 14416 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.147716 10202 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:37.003945 14960 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.130122 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:37.114053 14866 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.138118 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.121387 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.099817 14807 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:37.129396 14737 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.158607 14964 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.159242 14972 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:37.163754 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.166909 14826 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:37.167333 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 545) took 2351 ms. Trace:
I20260709 13:18:37.167522 10056 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:37.168334 14975 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:37.168795 14925 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.169124 10047 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:37.170689 14960 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.130208 14916 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:37.176286 14968 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:37.176220 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.185153 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:37.246615 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61857.2us
I20260709 13:18:37.193073 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.196655 14709 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:37.199555 14742 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.208104 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.209466 10893 rpcz_store.cc:276] 0709 13:18:34.816201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.816740 (+   539us) service_pool.cc:224] Handling call
0709 13:18:34.841175 (+ 24435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.546303 (+1705128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.546335 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.546382 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:36.546428 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.167039 (+620611us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:37.167089 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.167234 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3920386944}
I20260709 13:18:37.199450 14943 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:37.215629 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 162106us
I20260709 13:18:37.254908 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:37.260456 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:37.262019 14841 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:37.262301 14904 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:37.202525 14807 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.263475 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:37.264509 14841 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:37.264967 14841 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:37.265429 14841 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.252954 14975 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:37.240967 14826 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.270859 10201 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:37.275493 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.293346 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:37.296219 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.209805 14763 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.210160 14737 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.305837 14737 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.257146 14709 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.312311 10199 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:37.274246 14416 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:37.315477 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:37.319720 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 404773us
I20260709 13:18:37.270416 14826 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:37.320405 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 109217us
I20260709 13:18:37.238067 14964 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.325793 10200 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:37.245077 14970 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.329329 10199 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:37.330369 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 419676us
I20260709 13:18:37.302870 14914 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:37.326059 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:37.323956 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:37.336133 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:37.314236 14797 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:18:37.316819 14742 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:18:37.330973 14269 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.375159 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:37.376199 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:37.376724 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:37.376998 10047 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:37.377612 10051 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:18:37.378798 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:37.379673 10053 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11. Re-sending same reply.
I20260709 13:18:37.345134 14894 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:18:37.360298 14914 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:37.371706 14795 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:37.395587 14749 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:37.410709 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.448303 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.451611 10048 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:37.450114 14269 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:37.372251 14832 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.461123 14914 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.463769 14988 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 13:18:37.465106 14976 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:18:37.466809 14980 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:18:37.467939 14979 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:18:37.468811 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 132779us
I20260709 13:18:37.506043 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:37.469671 14936 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:37.470459 14942 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:37.435889 14810 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:18:37.411458 14795 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:37.504386 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56603.2us
I20260709 13:18:37.418606 14749 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:37.510588 14974 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:18:37.519338 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:37.520141 10123 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.514904 14925 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:37.517376 14892 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:18:37.517870 14807 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:37.504717 14709 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:37.525272 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71921.2us
I20260709 13:18:37.530789 10123 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:37.531780 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:37.527792 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:37.532104 14938 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:18:37.543404 14807 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:37.543745 14981 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:18:37.549126 14953 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:37.535849 14722 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:18:37.550045 14971 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.549937 14953 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.550712 14971 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.553375 14971 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:37.555169 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 83376.5us
I20260709 13:18:37.524679 14759 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:37.563496 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:37.545058 14983 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.547629 14763 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.576860 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:37.564608 14963 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:37.577926 14963 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:37.579324 14742 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:37.578858 14963 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.579916 14742 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:37.580406 14963 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.580844 14742 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.536859 14964 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:37.559832 14970 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:37.578729 14984 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:18:37.564087 14831 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.014s	sys 0.000s
I20260709 13:18:37.567595 14269 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:37.578847 14953 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.543430 14795 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:37.593531 14831 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.015s	sys 0.000s
W20260709 13:18:37.611498 14959 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.552s	user 0.001s	sys 0.003s
I20260709 13:18:37.616509 14759 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:37.617252 14759 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.582973 14983 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.619295 14971 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.619889 14971 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.617276 14933 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.625988 14926 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:37.626150 14916 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.627679 14825 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:37.628362 14825 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.579030 14977 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.634181 14977 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.635964 14749 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:37.636230 14945 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:37.636693 14945 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:37.636850 14977 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.602304 14269 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.637753 14945 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.640980 14269 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:37.642711 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.644622 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:37.645589 10893 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:18:37.647569 14612 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:37.648223 14612 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.626538 14926 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:37.647643 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:37.650566 10198 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:37.651312 14937 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:37.639040 14945 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:37.669489 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.671150 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.671293 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.673288 10204 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:37.676227 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:37.653812 14935 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:37.655941 14988 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:18:37.686661 14935 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:37.682893 14793 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:37.692778 14825 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.713985 10122 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:37.705767 14835 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.695571 14841 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.722512 14759 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:37.722832 14925 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.699878 14926 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.753252 14935 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.754488 14926 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:37.755064 14935 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:37.722541 14793 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:37.768934 14793 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:37.830687 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 149746us
I20260709 13:18:37.759038 14925 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.834100 14916 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.842401 14996 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:37.880169 14971 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:37.885205 14990 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
I20260709 13:18:37.874909 14996 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:37.888266 14996 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.939930 10125 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:18:37.942227 10125 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:37.943291 10125 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.952905 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2269) took 1310 ms. Trace:
I20260709 13:18:37.953653 10049 rpcz_store.cc:276] 0709 13:18:36.642109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.643229 (+  1120us) service_pool.cc:224] Handling call
0709 13:18:36.654502 (+ 11273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.751234 (+ 96732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.952045 (+1200811us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b5400086e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:37.952531 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195759872}
I20260709 13:18:37.966780 10200 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:37.979957 14904 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.968977 14841 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:37.982137 10122 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:18:37.984982 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:37.987464 14925 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:37.988262 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:37.990226 10202 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:18:37.997534 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.002851 14904 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.026480 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:38.027676 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.046066 14612 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.051816 14925 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:38.052350 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.053352 14759 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.054234 14759 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.053608 14925 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.055352 14945 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.056222 14945 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.060956 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 1957) took 1103 ms. Trace:
I20260709 13:18:38.061543 10201 rpcz_store.cc:276] 0709 13:18:36.956960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.957127 (+   167us) service_pool.cc:224] Handling call
0709 13:18:38.060477 (+1103350us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54001bf654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:38.060753 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686062336}
I20260709 13:18:38.065979 14832 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:38.078228 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.084013 10205 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:18:38.059019 14945 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.100517 14761 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:38.102243 14712 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:38.103608 14712 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:38.105851 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 781773us
I20260709 13:18:38.109611 14759 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.113648 14423 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:38.115765 14964 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:38.116307 14964 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:38.097474 14945 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.117756 14964 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.117950 14945 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:38.119445 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.122611 14994 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.123878 14994 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.125058 14612 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.126072 14612 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.126600 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:38.127393 14994 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.127895 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:38.128296 14925 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:38.123155 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.118098 14759 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.137825 10125 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.163967 14709 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:38.164348 10894 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:18:38.164835 14709 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:38.165316 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:38.166558 10128 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:38.163811 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:38.167917 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:38.260890 10125 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.168165 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:38.272219 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.299707 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:38.255858 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83502.7us
W20260709 13:18:38.285907 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.306079 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 153806us
W20260709 13:18:38.268599 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 118096us
I20260709 13:18:38.270236 14740 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.304068 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:38.310592 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:38.305608 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3148) took 1912 ms (client timeout 1584 ms). Trace:
I20260709 13:18:38.260319 14964 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:38.300576 14945 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:38.272835 14712 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:38.181140 14861 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.858s	user 0.010s	sys 0.026s
I20260709 13:18:38.279327 14612 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.273247 14748 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.189904 14759 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:38.209794 14938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.007s	sys 0.007s
I20260709 13:18:38.262296 15005 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.297644 14904 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:38.391847 10127 rpcz_store.cc:269] 0709 13:18:36.393223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.393419 (+   196us) service_pool.cc:224] Handling call
0709 13:18:38.305066 (+1911647us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540023c3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:38.305374 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413861632}
I20260709 13:18:38.279979 15020 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:38.420137 10128 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:38.429648 14926 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.671s	user 0.023s	sys 0.004s
W20260709 13:18:38.432251 14938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.014s	sys 0.018s
W20260709 13:18:38.447746 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.436726 14861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.317s	user 0.014s	sys 0.047s
I20260709 13:18:38.166970 14709 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.464560 10130 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 7. Current term is 9. Ops: []
I20260709 13:18:38.478556 14423 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:38.479588 14423 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.480705 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 533) took 1464 ms. Trace:
I20260709 13:18:38.482479 14423 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:38.486601 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2293) took 1028 ms. Trace:
W20260709 13:18:38.511754 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.487149 14748 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.488607 14826 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:18:38.489322 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 131916us
I20260709 13:18:38.489630 14892 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:38.519182 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65143.8us
I20260709 13:18:38.492312 14983 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.495471 14926 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:38.503329 10126 rpcz_store.cc:276] 0709 13:18:37.016548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.016742 (+   194us) service_pool.cc:224] Handling call
0709 13:18:38.307211 (+1290469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.307959 (+   748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.307999 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.308081 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:38.308160 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.308273 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.480322 (+172049us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540023c3d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:38.480450 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2966421760}
I20260709 13:18:38.468792 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:38.519536 14892 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:38.549357 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:38.515479 14826 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:38.580058 15019 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.595935 10130 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:18:38.596568 10130 pending_rounds.cc:85] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:18:38.532464 14740 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.598026 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:38.511482 10054 rpcz_store.cc:276] 0709 13:18:37.374917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.380162 (+  5245us) service_pool.cc:224] Handling call
0709 13:18:37.395772 (+ 15610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.396122 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:37.410675 (+ 14553us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.884558 (+473883us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:38.207055 (+322497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.207243 (+   188us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:38.359936 (+152693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.377029 (+ 17093us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.403188 (+ 26159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":627333,"prepare.queue_time_us":486246,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":760281,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":168,"spinlock_wait_cycles":38082432,"thread_start_us":163149,"threads_started":2,"wal-append.queue_time_us":128940}
W20260709 13:18:38.617527 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.620739 14748 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.473341 15005 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.602598 10130 pending_rounds.cc:107] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:38.628837 10130 raft_consensus.cc:2889] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:38.484402 15020 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.595103 15019 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.651803 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:38.606787 14983 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:38.654488 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.654181 14983 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.657220 14983 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:38.596155 14793 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 13:18:38.651862 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58704us
W20260709 13:18:38.680464 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45754.6us
I20260709 13:18:38.703172 15023 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:38.739756 14926 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.740931 14926 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.755934 14945 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.775808 14926 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:38.783804 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.813365 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95629us
I20260709 13:18:38.779397 14945 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.829999 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:38.830825 15019 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.833840 10052 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:18:38.834326 10052 pending_rounds.cc:85] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
W20260709 13:18:38.834498 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.834614 10052 pending_rounds.cc:107] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:18:38.835222 10052 raft_consensus.cc:2889] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:38.821051 14761 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.720s	user 0.021s	sys 0.003s
I20260709 13:18:38.836607 10124 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:18:38.837039 10124 pending_rounds.cc:85] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 4
I20260709 13:18:38.837343 10124 pending_rounds.cc:107] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:18:38.837587 10124 raft_consensus.cc:2889] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:38.837736 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 535) took 1810 ms. Trace:
I20260709 13:18:38.838876 14826 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.841356 14795 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.841552 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 282583us
I20260709 13:18:38.842653 10056 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:38.844072 14942 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.844636 15013 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:38.844749 14942 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.845345 14955 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:38.845448 15013 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:38.846202 15013 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:38.846923 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.842311 14795 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:38.848712 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 237795us
W20260709 13:18:38.836792 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 114459us
I20260709 13:18:38.831866 14709 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.734884 14759 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.846252 14955 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:38.721426 14938 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:38.900414 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 457336us
I20260709 13:18:38.911422 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.728317 14904 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.165s
I20260709 13:18:38.915042 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.838208 10131 rpcz_store.cc:276] 0709 13:18:37.028944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.029159 (+   215us) service_pool.cc:224] Handling call
0709 13:18:37.033431 (+  4272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.067798 (+ 34367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.067848 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:37.068522 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.026164 (+957642us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18001d2780. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:18:38.163325 (+137161us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:38.163416 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.837169 (+673753us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b54003d4854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:38.837230 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.837311 (+    81us) raft_consensus.cc:1685] finished
0709 13:18:38.837329 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.837532 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11939,"raft.queue_time_us":4652,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3247,"spinlock_wait_cycles":85257856,"thread_start_us":67310,"threads_started":1,"wal-append.queue_time_us":1173421}
I20260709 13:18:38.844800 14740 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.934303 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.926133 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:38.955746 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87924.2us
I20260709 13:18:38.734337 14892 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:38.964133 10201 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:38.971943 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:38.975579 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:38.976539 10131 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:38.976969 15005 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.978412 14759 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:38.975886 14976 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
W20260709 13:18:38.984202 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136439us
W20260709 13:18:38.985033 15034 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:38.986430 14749 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:38.986892 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 122333us
I20260709 13:18:38.987087 14749 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:38.988481 15029 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 13:18:38.990046 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:38.988577 14749 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:38.991618 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2301) took 1456 ms. Trace:
I20260709 13:18:38.993988 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:39.004115 15020 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.982746 14938 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:38.983937 15027 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
W20260709 13:18:39.007263 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.016150 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.029572 14761 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.030525 14761 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.033389 14761 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.984068 15024 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.034113 10198 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:18:39.034586 15024 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.035398 14861 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:39.036748 14712 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.037736 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.037515 14712 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.038342 14740 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.038614 10202 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:18:39.039311 14740 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.040840 10204 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:39.041131 14904 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:39.041623 14904 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:39.042037 14740 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:38.983968 14945 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.040892 10205 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:38.969201 14709 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.037438 15024 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.055434 14938 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:39.056314 15026 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
I20260709 13:18:39.056493 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:39.057221 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75707.3us
I20260709 13:18:38.997260 10048 rpcz_store.cc:276] 0709 13:18:37.535610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.536660 (+  1050us) service_pool.cc:224] Handling call
0709 13:18:37.537171 (+   511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:38.386791 (+849620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.401090 (+ 14299us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:38.456652 (+ 55562us) log.cc:844] Serialized 52 byte log entry
0709 13:18:38.586929 (+130277us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:38.859545 (+272616us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.859857 (+   312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.900852 (+ 40995us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.991043 (+ 90191us) raft_consensus.cc:1685] finished
0709 13:18:38.991069 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.991421 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":597706,"raft.queue_time_us":738365,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":30724,"thread_start_us":380146,"threads_started":3,"wal-append.queue_time_us":239447}
I20260709 13:18:39.058396 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.058655 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:39.059072 15033 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:18:39.059712 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2200) took 1583 ms. Trace:
I20260709 13:18:39.017222 14759 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.060153 10123 rpcz_store.cc:276] 0709 13:18:37.476554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.949156 (+472602us) service_pool.cc:224] Handling call
0709 13:18:37.949812 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.950432 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.950465 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.162694 (+212229us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.375570 (+212876us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:38.375630 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.375747 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.059314 (+683567us) raft_consensus.cc:1685] finished
0709 13:18:39.059336 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.059486 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":106157,"threads_started":1,"wal-append.queue_time_us":424317}
I20260709 13:18:39.061821 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:39.062326 14759 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.064184 15024 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:39.064973 10201 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:39.065483 10201 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:39.069476 14709 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.071693 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:39.025708 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65595.8us
I20260709 13:18:39.072861 10056 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:39.069598 14712 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.077253 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:39.083077 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2300) took 1551 ms. Trace:
I20260709 13:18:39.042078 14904 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:39.096099 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3127) took 2023 ms. Trace:
I20260709 13:18:39.066879 10201 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.103848 10200 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:39.098070 10197 rpcz_store.cc:276] 0709 13:18:37.072771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.239148 (+166377us) service_pool.cc:224] Handling call
0709 13:18:37.239547 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.269270 (+ 29723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.269320 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.269383 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:37.269438 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.091719 (+1822281us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:39.095413 (+  3694us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.095763 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67585792}
I20260709 13:18:39.133642 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.038014 14861 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:18:39.088157 10051 rpcz_store.cc:276] 0709 13:18:37.531248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.545418 (+ 14170us) service_pool.cc:224] Handling call
0709 13:18:37.545821 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.256150 (+710329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.256227 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.458646 (+202419us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.560699 (+102053us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:38.560744 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.572387 (+ 11643us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.075871 (+503484us) raft_consensus.cc:1685] finished
0709 13:18:39.075899 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.082852 (+  6953us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":602481,"raft.queue_time_us":727436,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":204,"thread_start_us":180217,"threads_started":2,"wal-append.queue_time_us":343926}
I20260709 13:18:39.150321 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.210991 10201 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:39.081977 14759 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:39.173156 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:39.303490 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.256538 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:39.178304 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:39.315100 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.227815 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.154409 10202 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
I20260709 13:18:39.209976 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:39.239782 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:39.329943 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.146376 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88132.8us
I20260709 13:18:39.336150 10056 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:39.338281 10126 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:39.338481 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.239315 14742 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.337150 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.339838 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.340591 10897 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:39.341768 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.343353 10894 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:39.343726 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.339874 14742 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.345861 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.321121 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3152) took 2885 ms. Trace:
I20260709 13:18:39.338354 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:39.351413 10124 rpcz_store.cc:276] 0709 13:18:36.394919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.395073 (+   154us) service_pool.cc:224] Handling call
0709 13:18:36.417303 (+ 22230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.838237 (+2420934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.254801 (+416564us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54001270d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:39.279913 (+ 25112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5562192256}
I20260709 13:18:39.355883 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.356418 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.354741 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.358012 14721 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.343025 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.362521 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.338898 14936 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.364218 14936 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.368901 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2303) took 1832 ms. Trace:
I20260709 13:18:39.369534 10050 rpcz_store.cc:276] 0709 13:18:37.536367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.537624 (+  1257us) service_pool.cc:224] Handling call
0709 13:18:37.538062 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.463754 (+925692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.469961 (+  6207us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.575940 (+105979us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.885865 (+309925us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:39.119835 (+233970us) log.cc:844] Serialized 14 byte log entry
0709 13:18:39.159231 (+ 39396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.188786 (+ 29555us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.368487 (+179701us) raft_consensus.cc:1685] finished
0709 13:18:39.368513 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.368687 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":633430,"prepare.queue_time_us":478972,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":773009,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":46414,"thread_start_us":671798,"threads_started":3,"wal-append.queue_time_us":762843}
I20260709 13:18:39.359309 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:39.328920 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 338980us
I20260709 13:18:39.188870 15024 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.292831 14832 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.226s	user 0.016s	sys 0.007s
I20260709 13:18:39.250880 14795 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.256865 15019 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.175474 15017 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:39.247546 14955 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.147363 14945 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:39.133543 14964 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.705s	user 0.008s	sys 0.008s
I20260709 13:18:39.173987 14709 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:39.255474 15020 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.394435 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 184611us
W20260709 13:18:39.398908 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.472399 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.471426 14861 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:39.197257 14938 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.393567 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.393201 14761 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.393131 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.485567 14992 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.506038 14797 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:39.505699 14938 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.252170 14807 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:39.393699 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:39.458660 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.255728 14752 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.238587 14542 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:39.502650 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.503463 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68940.9us
W20260709 13:18:39.507815 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 251460us
I20260709 13:18:39.476392 14945 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.481596 14748 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:39.478978 14759 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.456895 14936 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.506386 14761 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.426029 15023 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:39.520915 10198 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:39.393945 14983 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.448771 14742 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.509716 14807 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:39.556710 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.533999 14945 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:39.505746 14861 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:39.568758 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 196122us
I20260709 13:18:39.575088 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.522526 15024 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.505875 14992 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.579202 15040 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:39.590220 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.566653 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:39.592658 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 195176us
I20260709 13:18:39.502110 15017 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.543087 14759 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.596252 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:39.538865 15019 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.589169 15040 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:39.560599 14936 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:39.573889 14709 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.553714 14542 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:39.624039 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 260114us
I20260709 13:18:39.560731 14748 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.561626 15020 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.614387 10202 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
W20260709 13:18:39.621804 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.599460 15045 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:39.569932 14752 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.580312 14983 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.652172 14742 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:39.652936 10900 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
W20260709 13:18:39.659725 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 80425.1us
W20260709 13:18:39.660600 14904 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.551s	user 0.015s	sys 0.000s
I20260709 13:18:39.661176 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.662158 14938 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.661208 14904 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.665877 14904 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.670010 14904 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:39.622932 15040 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:39.671484 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.683s (SENT)
I20260709 13:18:39.672012 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:39.671175 15042 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.673769 15042 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.676810 14748 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.677834 15042 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.681015 14983 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.681722 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:39.681962 14749 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.001s
I20260709 13:18:39.683624 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.688001 10198 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:39.690817 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.642045 14964 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:39.691100 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.691425 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:39.692672 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.694219 15017 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.695209 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:39.698313 15019 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.686040 14709 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.698611 15021 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:39.688690 14938 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:39.812255 10201 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:39.709820 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:39.709537 14807 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.713119 14542 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:39.716168 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93330us
I20260709 13:18:39.717212 14709 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.717888 15020 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.719514 14759 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.720813 14761 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.704312 14983 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:39.719578 15023 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:39.726070 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.726414 14861 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.732471 14882 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.701793 14749 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.740069 14752 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.705935 14964 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:39.746011 15024 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.748798 14748 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.814548 10205 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:39.706660 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:39.825279 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 115604us
W20260709 13:18:39.825325 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.833012 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.724023 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.708426 15021 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:39.853220 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.848209 14841 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.842504 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:39.824553 14904 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:39.860985 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 197931us
I20260709 13:18:39.859200 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:39.862820 14964 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:39.866035 10202 raft_consensus.cc:2410] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:18:39.857924 15006 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:39.876168 10198 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:18:39.876740 14882 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.877857 14748 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.876088 14841 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.927711 14841 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:39.882908 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.885789 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 542) took 1152 ms. Trace:
I20260709 13:18:39.930270 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.930692 10130 rpcz_store.cc:276] 0709 13:18:38.733914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.820095 (+ 86181us) service_pool.cc:224] Handling call
0709 13:18:38.870336 (+ 50241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.911879 (+ 41543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:38.911931 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.959064 (+ 47133us) log.cc:844] Serialized 26 byte log entry
0709 13:18:39.354727 (+395663us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b18000271c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:18:39.602924 (+248197us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:39.602974 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.603055 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.885412 (+282357us) raft_consensus.cc:1685] finished
0709 13:18:39.885434 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.885708 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10170,"raft.queue_time_us":6737,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":28607,"spinlock_wait_cycles":10892672,"thread_start_us":247949,"threads_started":1,"wal-append.queue_time_us":843492}
I20260709 13:18:39.933020 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:39.933389 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:39.933776 10050 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:39.886118 15040 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:39.935047 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:39.929302 14882 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:39.937093 10130 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:39.939219 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:39.939638 14742 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.940371 10130 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 13:18:39.940332 14742 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:39.944190 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.891896 10204 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:39.902998 15023 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:39.912822 14964 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:39.913553 15051 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:18:39.906992 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:39.908802 14542 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.972800 14748 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:39.946930 15013 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.097s	user 0.010s	sys 0.009s
I20260709 13:18:39.973004 14542 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.973708 15013 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.976212 15051 consensus_queue.cc:1243] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:39.977463 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.978406 10205 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
W20260709 13:18:39.981031 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.909003 15042 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:39.910194 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.916352 14907 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:39.916920 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:39.892403 14709 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:18:39.921828 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.925144 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.840317 14938 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.950095 14882 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:39.886637 14721 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
W20260709 13:18:39.960803 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 118603us
W20260709 13:18:39.961323 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.989233 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.166s (SENT) [suppressed 15 similar messages]
I20260709 13:18:39.935875 14749 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.898151 15006 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.971813 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:39.972015 15019 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:39.972221 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 105137us
I20260709 13:18:39.924114 15021 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.935376 14945 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:39.961784 15023 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.995740 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:39.998025 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:40.008971 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.962167 14742 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:39.976214 15013 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:40.003837 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 541258us
I20260709 13:18:40.010841 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:40.005911 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:40.012470 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.015573 10899 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 7.
I20260709 13:18:40.029201 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:40.032562 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:40.034080 10203 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:18:40.002409 15006 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.036113 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.041941 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.036514 10129 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:40.002410 14904 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:40.010495 15023 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:40.052657 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.054467 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:40.055265 10199 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:40.056407 10202 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:18:40.058251 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:40.061791 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.046211 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 129390us
I20260709 13:18:40.000389 15051 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:40.032186 14994 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.001559 14882 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.065953 15021 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.037283 14542 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.017892 14983 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.032148 15052 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:40.048115 14936 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:40.105063 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:40.066241 15017 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:40.045737 14795 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:40.109478 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.110158 10202 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:40.111423 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:40.114046 14938 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:40.114723 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 197237us
I20260709 13:18:40.040726 15013 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:18:40.030383 15042 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.043154 14761 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:18:40.137672 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:40.144111 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:40.144593 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84778.6us
I20260709 13:18:40.143020 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:40.151894 10203 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:40.171489 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:40.105422 14936 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.173941 10129 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:40.178045 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:40.192978 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.145779 14795 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.104908 14994 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.115042 14983 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.150092 15052 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:40.206748 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77960.3us
I20260709 13:18:40.181107 14936 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.176697 15017 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:40.211280 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.212363 14742 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:40.211974 15017 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:40.209756 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 164077us
I20260709 13:18:40.216463 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:40.209134 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68752.8us
W20260709 13:18:40.216552 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 184618us
I20260709 13:18:40.221920 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:40.222820 10125 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:40.227736 10203 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:40.236138 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.238771 14945 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:40.301213 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61612.1us
I20260709 13:18:40.307147 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:40.307996 10125 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.216039 15052 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.323344 10050 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:40.315892 15058 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:18:40.317706 15061 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:40.318316 14983 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.320472 15024 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:18:40.335255 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:40.328117 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.334249 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:40.346477 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 148709us
I20260709 13:18:40.344897 10127 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:40.360101 15069 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:40.361079 14882 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.363981 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:40.365820 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.366165 14937 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:40.367524 10203 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:40.356182 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.369388 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.372650 15066 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:40.375695 15052 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.376425 15042 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:40.338323 14748 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:40.351429 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.326404 14742 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:18:40.386085 14542 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:40.338961 15019 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.390166 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76238.7us
I20260709 13:18:40.395011 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.397831 15078 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:40.399895 15078 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.401098 15061 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:40.401635 15061 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:40.402071 15061 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.394418 15019 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.405886 15019 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.398981 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.390631 15024 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:40.391466 14882 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:40.402982 15078 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:40.435065 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69418.7us
I20260709 13:18:40.426640 14936 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.436983 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.438184 15024 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:40.438603 14936 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:40.445271 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.439005 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:40.420217 14904 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:18:40.434382 14742 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:40.445763 14826 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.426470 14882 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:40.462589 15019 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:40.469048 15019 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:40.469646 15019 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.477607 14752 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:40.479719 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.439056 15024 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:18:40.467562 14882 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:40.461028 15080 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:40.485984 15019 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.458940 14795 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.489521 14797 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:40.490268 14797 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.444623 15023 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.465844 14742 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.509024 14742 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.503433 10125 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:40.530498 15060 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
I20260709 13:18:40.537525 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 562) took 3371 ms. Trace:
I20260709 13:18:40.538025 15023 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.541286 10053 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:40.542147 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2317) took 1292 ms. Trace:
I20260709 13:18:40.545251 14964 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:40.545053 15006 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:40.552052 10899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 349) took 2427 ms (client timeout 2144 ms). Trace:
W20260709 13:18:40.552655 10899 rpcz_store.cc:269] 0709 13:18:38.124217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.151890 (+ 27673us) service_pool.cc:224] Handling call
0709 13:18:40.551715 (+2399825us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54006f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:40.551906 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3796345600}
I20260709 13:18:40.554150 10896 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:40.555451 15019 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:40.556146 14907 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.556068 15061 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.556898 14907 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.557623 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 549) took 1396 ms. Trace:
I20260709 13:18:40.558070 10123 rpcz_store.cc:276] 0709 13:18:39.161002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.174706 (+ 13704us) service_pool.cc:224] Handling call
0709 13:18:39.175118 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.485392 (+310274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.485434 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:39.485502 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:39.485552 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.557345 (+1071793us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:40.557398 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.557536 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687415680}
I20260709 13:18:40.558924 15061 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.538157 10055 rpcz_store.cc:276] 0709 13:18:37.165534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.168055 (+  2521us) service_pool.cc:224] Handling call
0709 13:18:37.168410 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.246768 (+ 78358us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:37.259711 (+ 12943us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.881734 (+622023us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1
0709 13:18:37.882237 (+   503us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:38.103606 (+221369us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa27bed858f 00007fa274f17571 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd
0709 13:18:38.498526 (+394920us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:18:38.757433 (+258907us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:38.758646 (+  1213us) log.cc:844] Serialized 26 byte log entry
0709 13:18:38.758965 (+   319us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:38.953751 (+194786us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.954263 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.359979 (+405716us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:39.368352 (+  8373us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b18002b78c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17557 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:40.536892 (+1168540us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:40.536956 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.537016 (+    60us) raft_consensus.cc:1685] finished
0709 13:18:40.537033 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.537264 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16613,"prepare.queue_time_us":620027,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":90,"raft.queue_time_us":21324,"raft.run_cpu_time_us":652,"raft.run_wall_time_us":123041,"spinlock_wait_cycles":422965504,"thread_start_us":364104,"threads_started":2,"wal-append.queue_time_us":1587312,"wal-append.run_cpu_time_us":2835,"wal-append.run_wall_time_us":1011519}
I20260709 13:18:40.542493 10053 rpcz_store.cc:276] 0709 13:18:39.249628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.251249 (+  1621us) service_pool.cc:224] Handling call
0709 13:18:40.540518 (+1289269us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edff8c 00007fa274edfef6 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:40.540728 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.541835 (+  1107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.542079 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10662656}
I20260709 13:18:40.562224 15019 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.562961 15019 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.563460 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.564551 15006 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.565423 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.566039 15019 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.567518 14942 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.568202 14807 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.572388 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3141) took 1519 ms. Trace:
I20260709 13:18:40.572927 10204 rpcz_store.cc:276] 0709 13:18:39.052919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.053128 (+   209us) service_pool.cc:224] Handling call
0709 13:18:39.088349 (+ 35221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.968515 (+880166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.572065 (+603550us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54001d0454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:40.572310 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847427456}
I20260709 13:18:40.573419 14797 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.566932 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.584374 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.585628 10055 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:18:40.593880 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 167526us
I20260709 13:18:40.595705 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.596902 15094 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:40.597465 15094 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.589689 14882 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.597921 15094 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.601734 14882 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.602382 14742 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.604315 15051 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:40.604892 15051 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:40.605690 14742 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.606097 15051 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.610494 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:40.612160 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 188307us
I20260709 13:18:40.623471 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.623621 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:40.624908 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.633279 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.611042 14797 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:40.634390 10197 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:40.614116 14749 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:40.641652 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3146) took 1530 ms. Trace:
I20260709 13:18:40.619042 15051 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.641011 14793 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:40.645716 10197 rpcz_store.cc:276] 0709 13:18:39.111105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.264210 (+153105us) service_pool.cc:224] Handling call
0709 13:18:40.623421 (+1359211us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:40.641545 (+ 18124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12121984}
W20260709 13:18:40.634771 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.657080 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.665349 10055 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:18:40.652833 14793 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:40.668309 14793 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.669898 14793 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:40.675527 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.679771 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.672538 14749 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:40.682395 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:40.687099 10895 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 7.
I20260709 13:18:40.691668 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:40.691745 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:40.692734 10124 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:40.694410 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:40.694871 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:40.695703 10898 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.700525 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:40.725708 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53342.1us
I20260709 13:18:40.723832 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:40.699476 15017 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:40.729485 15017 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.727651 14964 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:40.732631 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:40.715516 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:40.734400 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:40.735082 10124 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:40.731349 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:40.688032 14882 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:40.752451 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.739607 15073 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:40.744102 10055 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:18:40.767155 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:40.768889 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:40.773180 10122 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:18:40.775982 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2316) took 1526 ms. Trace:
I20260709 13:18:40.773886 10122 pending_rounds.cc:85] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
I20260709 13:18:40.777688 10122 pending_rounds.cc:107] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:18:40.776532 10049 rpcz_store.cc:276] 0709 13:18:39.249129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.251294 (+  2165us) service_pool.cc:224] Handling call
0709 13:18:40.752253 (+1500959us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:40.752439 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.768093 (+ 15654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.768126 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:40.768175 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:40.775653 (+  7478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.775749 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.775909 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13764736}
I20260709 13:18:40.778542 10122 raft_consensus.cc:2889] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:40.779712 14732 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:40.780247 14732 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:40.780666 14732 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.781014 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.782336 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:40.784195 15100 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:18:40.786887 15017 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.771998 15043 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:40.787688 15043 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:40.787897 10047 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:40.788160 15043 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.788806 14468 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:40.791216 14861 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:40.791653 14861 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:40.792188 15076 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:40.682440 14749 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.794919 14732 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:40.796737 10052 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:40.797480 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 567) took 1205 ms. Trace:
I20260709 13:18:40.783213 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.783236 10200 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:40.797822 10052 rpcz_store.cc:276] 0709 13:18:39.592504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.795940 (+1203436us) service_pool.cc:224] Handling call
0709 13:18:40.796322 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.797183 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.797420 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:40.805806 10124 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
W20260709 13:18:40.813010 10200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3147) took 1666 ms (client timeout 1847 ms). Trace:
I20260709 13:18:40.771641 14882 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:40.813619 10200 rpcz_store.cc:269] 0709 13:18:39.147027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.239246 (+ 92219us) service_pool.cc:224] Handling call
0709 13:18:40.782146 (+1542900us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:40.812906 (+ 30760us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12336000}
I20260709 13:18:40.835373 14964 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:40.839805 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.811545 14732 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:40.847364 10052 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 13:18:40.852945 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:40.854276 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:40.859714 10049 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:40.862685 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:40.863870 10123 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.840742 15094 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.883461 14892 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:40.886111 15094 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:40.884281 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 45249.4us
I20260709 13:18:40.884426 15084 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.888116 15084 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.890412 15084 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.884752 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:40.899648 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.901408 10898 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:40.895263 14797 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.906930 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:40.909880 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.917201 10048 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:18:40.927536 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:40.928056 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.928457 10055 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:40.929591 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.930449 14907 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.931932 10052 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:18:40.932893 15080 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:40.933233 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.933997 14964 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.934090 10052 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:40.937382 14861 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.904631 14612 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:40.940861 10123 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:40.941398 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.942129 10050 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:40.943898 10126 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:40.944514 14942 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.945073 14942 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:40.946195 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2218) took 1255 ms. Trace:
I20260709 13:18:40.933463 15080 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:40.948487 14861 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:40.949288 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.951704 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:40.952868 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.954216 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:40.954599 10900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 557) took 3310 ms (client timeout 1805 ms). Trace:
I20260709 13:18:40.954879 10050 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:18:40.955312 10900 rpcz_store.cc:269] 0709 13:18:37.644111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.647291 (+  3180us) service_pool.cc:224] Handling call
0709 13:18:40.954284 (+3306993us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54001e7154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:40.954496 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4504573696}
I20260709 13:18:38.422217  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:40.934808 15105 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:40.958998 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2219) took 1267 ms. Trace:
I20260709 13:18:40.961076 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.979394 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 583) took 1472 ms. Trace:
W20260709 13:18:40.995283 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84006.8us
W20260709 13:18:40.983023 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 76186.3us
W20260709 13:18:40.991451 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.946655 10126 rpcz_store.cc:276] 0709 13:18:39.690948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.691101 (+   153us) service_pool.cc:224] Handling call
0709 13:18:40.945916 (+1254815us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:40.946130 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2849941376}
I20260709 13:18:40.930886 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.000648 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.995451 10202 rpcz_store.cc:276] 0709 13:18:39.507512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.146079 (+638567us) service_pool.cc:224] Handling call
0709 13:18:40.181195 (+ 35116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.208556 (+ 27361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.208610 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:40.208669 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:40.208712 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.978932 (+770220us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:40.979033 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.979293 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5071616}
I20260709 13:18:41.010643 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.002975 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:41.012125 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.012775 10054 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 7.
I20260709 13:18:41.013691 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.014248 10054 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:41.015480 15101 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:41.017334 15101 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:41.018409 15101 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.019685 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:41.020387 10130 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.023015 10123 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:41.024231 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:41.025049 10124 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:41.029682 15108 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:41.030622 15108 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:41.031836 15108 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.032351 14994 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:40.993093 10122 rpcz_store.cc:276] 0709 13:18:39.691391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.694637 (+  3246us) service_pool.cc:224] Handling call
0709 13:18:39.694997 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.779143 (+1084146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.958514 (+179371us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400389ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:40.958888 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2478274304}
W20260709 13:18:41.043521 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.044821 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.982199 14882 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:41.053231 10050 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:41.049794 10054 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 7: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:41.060883 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:41.062266 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.063560 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:40.986450 15105 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:41.070330 10054 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:41.074402 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.075477 10126 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:41.082196 10197 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:41.087761 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.088626 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.089581 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.091724 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.093670 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.099329 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:41.087525 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.089669 10052 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:41.100533 10204 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:40.920425 14797 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.098922 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.099042 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.115516 10196 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:41.077502 15072 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:18:41.121328 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:41.098897 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.130290 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.123304 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.118988 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 551) took 1444 ms. Trace:
I20260709 13:18:41.134747 10131 rpcz_store.cc:276] 0709 13:18:39.674917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.675072 (+   155us) service_pool.cc:224] Handling call
0709 13:18:39.675611 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.675880 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:18:40.029154 (+353274us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.408707 (+379553us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:18:41.100681 (+691974us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b180034a0e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287
0709 13:18:41.118354 (+ 17673us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.118454 (+   100us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:41.118530 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.118653 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.118885 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18464,"prepare.queue_time_us":396684,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":14293,"raft.queue_time_us":83779,"raft.run_cpu_time_us":2161,"raft.run_wall_time_us":119664,"spinlock_wait_cycles":34368512,"thread_start_us":587280,"threads_started":3,"wal-append.queue_time_us":208137}
I20260709 13:18:41.138756 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.139726 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:41.142076 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:41.145411 10203 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:41.078279 15105 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:41.087783 14925 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:41.153160 10056 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:41.127266 14759 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:41.087495 14468 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:18:41.178030 10129 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 6.
I20260709 13:18:41.179335 10124 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:41.180234 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.151954 14841 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:41.186950 10055 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:41.189366 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.190963 10050 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:41.157888 14882 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:41.199743 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 57819.7us
I20260709 13:18:41.201846 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:41.205947 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:41.177539 14793 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:41.135373 15043 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:41.214216 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.184479 14841 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.198992 14833 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:18:41.214849 14458 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.164s
I20260709 13:18:41.215260 14942 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:41.215600 15076 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.305197 15076 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.305648 15076 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.216154 14793 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:41.308077 14793 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.222329 10049 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:41.311581 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.311975 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.312384 10055 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:41.312731 10049 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:41.313433 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.313613 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.314169 10049 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:41.314707 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.253528 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.189635 15019 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.257113 14841 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:41.302604  9949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 119) took 12355 ms (client timeout 15000 ms). Trace:
I20260709 13:18:41.303668 10892 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:41.304817 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.239406 10126 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:41.315021 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.326040 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.178537 14925 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.364576 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.365370 10049 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:41.366394 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.367070 10049 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:41.369411 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.370205 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.371222 10047 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:41.243597 14709 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.379956 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.380676 10051 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:18:41.383085 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.387516 10053 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:18:41.355589  9949 rpcz_store.cc:269] 0709 13:18:28.824560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.868870 (+ 44310us) service_pool.cc:224] Handling call
0709 13:18:41.175575 (+12306705us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:34.693621 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:34.803995 (+110374us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:34.804060 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:18:34.804103 (+    43us) tablet.cc:662] Decoding operations
0709 13:18:34.849910 (+ 45807us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:34.850005 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:34.850036 (+    31us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:18:34.851568 (+  1532us) tablet.cc:701] Row locks acquired
0709 13:18:34.851621 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:18:34.851835 (+   214us) write_op.cc:270] Start()
0709 13:18:34.852030 (+   195us) write_op.cc:276] Timestamp: P: 1783603114851764 usec, L: 0
0709 13:18:34.852055 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:18:35.156818 (+304763us) log.cc:844] Serialized 6668 byte log entry
0709 13:18:36.298289 (+1141471us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:18:37.105439 (+807150us) op_driver.cc:464] REPLICATION: finished
0709 13:18:37.261141 (+155702us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277fffe28 00007fa27ba3a581 00007fa27ba39d58 00007fa27ba3e9f1 00007fa27ba3e9a9 00007fa27ba3e939 00007fa27ba3e901 00007fa27ba3db2d 000000000038f354 000000000038f199
0709 13:18:37.365013 (+103872us) write_op.cc:301] APPLY: starting
0709 13:18:37.365133 (+   120us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:37.638033 (+272900us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:37.638067 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:37.884810 (+246743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:37.885164 (+   354us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:18:37.885230 (+    66us) write_op.cc:312] APPLY: finished
0709 13:18:37.889920 (+  4690us) log.cc:844] Serialized 156 byte log entry
0709 13:18:37.890138 (+   218us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.890354 (+   216us) write_op.cc:454] Released schema lock
0709 13:18:37.890423 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:37.890913 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.890950 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103194,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":256459,"apply.run_cpu_time_us":31367,"apply.run_wall_time_us":525868,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75051,"num_ops":17,"prepare.queue_time_us":246210,"prepare.run_cpu_time_us":81334,"prepare.run_wall_time_us":2609567,"raft.queue_time_us":224495,"raft.run_cpu_time_us":46694,"raft.run_wall_time_us":235158,"replication_time_us":2250097,"spinlock_wait_cycles":104449024,"thread_start_us":1927442,"threads_started":4,"wal-append.queue_time_us":1493066,"wal-append.run_cpu_time_us":31931,"wal-append.run_wall_time_us":2318439}]]}
I20260709 13:18:41.243173 15023 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.170s
I20260709 13:18:41.395052 15105 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.398124 14882 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:41.403331 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.414383 10126 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:41.423754 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:41.421969 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.407032 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.380210 14709 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.432657 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:41.437191 10125 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.016s
W20260709 13:18:41.439162 10125 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.005s	sys 0.016s
I20260709 13:18:41.438809 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:41.447618 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.452533 10053 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:18:41.367995 15019 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.456173 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.457537 15042 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:41.470052 10203 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:41.464540 10128 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.006s	sys 0.064s
I20260709 13:18:41.426482 14992 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:41.471817 10128 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.006s	sys 0.064s
I20260709 13:18:41.430991 14709 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.492978 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.494170 10049 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:41.495316 14709 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.498581 14709 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:41.500394 14759 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:41.501274 14759 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:41.508296 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:41.513850 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.515908 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.539108 10051 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:41.541175 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
W20260709 13:18:41.525719 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58124.9us
W20260709 13:18:41.544677 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 330826us
I20260709 13:18:41.552878 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.563550 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.564621 10055 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:41.543438 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:41.568818 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.570642 10049 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:41.593017 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.596463 14841 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.599196 14732 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:41.601502 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.602312 10199 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:18:41.589330 15107 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.604678 15107 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.581971 15042 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:41.572739 14797 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:41.607074 15107 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:41.628619 14841 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:41.633322 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.638505 15019 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:41.641403 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.641610 15076 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:41.664187 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.649761 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:41.678584 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.651327 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.651528 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.679505 10200 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:41.672533 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:41.680156 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:41.607786 10201 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:41.626411 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.618541 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.657353 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:41.646379 15080 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.699s	user 0.007s	sys 0.004s
I20260709 13:18:41.507016 14925 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:41.618955 10130 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.000s
I20260709 13:18:41.668592 10054 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:41.676043 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.681689 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:41.684563 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.593039 15043 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:41.685976 10899 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:41.707924 15043 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:41.708420 15043 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.708809 15043 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:41.710682 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 53482.5us
I20260709 13:18:41.716338 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.608349 15042 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.718209 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.612426 15105 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:41.688235 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:41.711830 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.719322 14709 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.720217 10894 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:41.720649 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:41.722868 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.725230 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:41.728547 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.729493 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:41.728324 14709 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.740526 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:41.727959 10095 connection.cc:256] RPC call timeout handler was delayed by -1.0509921289999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:18:41.743018 10095 connection.cc:256] RPC call timeout handler was delayed by -1.0126829000000726s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
I20260709 13:18:41.747529 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.749739 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.750728 10049 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:41.758634 10130 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:41.798647 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2248) took 1109 ms. Trace:
I20260709 13:18:41.773979 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:41.799821 10198 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 11: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:41.739032 10892 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:41.773108 15105 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:41.804214 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:41.732573 14925 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:41.808218 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.797987 10054 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:41.822425 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.765916 10204 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:41.784219 15108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.005s
I20260709 13:18:41.727327 15080 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:41.836407 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:41.811622 10124 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:41.796052 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:41.836515 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.801805 10130 rpcz_store.cc:276] 0709 13:18:40.689062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.702504 (+ 13442us) service_pool.cc:224] Handling call
0709 13:18:41.019502 (+316998us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:41.798557 (+779055us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12569472}
I20260709 13:18:41.843297 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:41.846815 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50898.3us
I20260709 13:18:41.794595 15019 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:41.848263 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:41.847605 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.851104 15080 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:41.846836 15072 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:41.855523 14709 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:41.859688 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.860852 10049 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:41.862113 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.830497 15040 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:41.868858 15040 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:41.869381 15040 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:41.871889 15019 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.862854 10049 raft_consensus.cc:2376] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14. Re-sending same reply.
I20260709 13:18:41.882740 10055 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:41.835518 15108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.005s	sys 0.005s
I20260709 13:18:41.874728 15019 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:41.887578 10053 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:41.890509 10130 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:18:41.894332 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.896886 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.848791 14904 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:41.903889 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:41.912102 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 384641us
I20260709 13:18:41.907015 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:41.908429 10204 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:41.916375 10125 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:18:41.914644 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.918401 10049 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:41.923580 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.833480 14942 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:18:41.933461 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:41.834755 14925 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:41.938537 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:41.935684 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.939919 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:41.991380 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52930.4us
I20260709 13:18:41.992691 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:41.904268 15106 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:41.996719 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.002501 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:41.997084 10054 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:41.998138 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:41.909765 14904 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.024503 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.024107 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.025394 10055 raft_consensus.cc:2376] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9. Re-sending same reply.
I20260709 13:18:42.030241 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:41.930250 15019 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:42.027956 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.040894 10052 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:41.901247 15093 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260709 13:18:41.930505 14709 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:41.938918 15080 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:18:42.037859 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 485009us
W20260709 13:18:42.053540 10020 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.809s	user 0.009s	sys 0.030s
I20260709 13:18:41.909709 15072 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 13:18:42.061021 10020 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.009s	sys 0.030s
W20260709 13:18:42.061520 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 816944us
I20260709 13:18:42.022964 15105 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.063814 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:41.913892 14759 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.090018 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.035112 14904 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.096045 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.029953 14742 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:42.097234 10125 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:42.097754 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:42.098693 10052 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.096094 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.094823 10054 raft_consensus.cc:2368] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:42.055838 14797 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:42.062598 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.109465 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.034849 14841 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 13:18:42.124017 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61570.2us
I20260709 13:18:42.125550 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.109714 14826 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.131318 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:42.107683 14797 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.066655 14709 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:42.165897 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.171672 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.116155 15076 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:42.173982 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.175143 10125 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:42.135226 14882 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:18:42.191466 10051 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.003s
I20260709 13:18:42.195534 10051 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:42.128388 14826 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.166177 14748 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:42.167941 14797 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:42.122802 15043 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.116911 15019 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.193660 14882 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:42.216203 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.221539 10054 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:42.226287 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.233050 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.211596 15084 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:42.237355 10127 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 9.
W20260709 13:18:42.261732 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 101681us
I20260709 13:18:42.211234 14871 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:42.271965 15122 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 13:18:42.272487 15121 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:18:42.273311 15125 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:18:42.229490 14942 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.277479 14826 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:42.280154 15133 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:42.280772 15133 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:42.281237 15133 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:42.284186 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.286314 10050 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:42.278192 14942 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.289173 14942 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.289976 14942 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:42.279387 15134 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:42.291209 15134 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:42.292387 15134 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.293994 15134 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:42.298103 15019 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.300132 10050 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:42.301783 10050 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:42.303948 14925 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.304716 14925 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:42.305132 14925 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:42.306133 14925 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.306533 15043 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.315279 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.315980 10127 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 6.
I20260709 13:18:42.314261 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.251920 14882 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:42.272508 15123 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:18:42.320904 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.325202 10052 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:42.284310 14709 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.298988 15019 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:42.332582 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:42.343444 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.345038 10127 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:18:42.349263 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 223820us
I20260709 13:18:42.350517 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:42.354736 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.309444 14871 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.322120 14955 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:18:42.361953 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.366650 10127 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:42.352449 14759 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.386385 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.389346 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.390439 10130 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.391628 10124 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:18:42.392757 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.393935 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.394717 10124 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:42.396765 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.397955 10131 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:42.379837 14841 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:42.405328 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.408290 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.409965 10124 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:42.353106 15019 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.405390 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.423719 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.424389 10131 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:42.429590 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.378703 14709 raft_consensus.cc:2764] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:42.413206 15135 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:42.407589 15087 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:42.444694 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.455386 15087 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:42.457346 15093 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:18:42.459353 15137 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:18:42.413111 14748 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.464952 14871 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.465526 10051 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:42.465197 14748 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.421474 14841 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:42.457515 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.467481 10124 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.467830 14841 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.476138 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.478262 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.480998 14861 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.481563 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.482344 10129 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:42.484953 14709 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:42.486011 14709 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.486800 14709 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:42.490639 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.536202 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:42.451864 10130 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:42.497097 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:42.497592 15084 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:42.497875 10892 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:18:42.501895 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.503238 15133 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.552619 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.554709 15133 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.454324 15135 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:42.510742 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:42.515511 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.521916 14904 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.540048 10124 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:42.541885 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:42.542593 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.491283 14882 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.572962 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.573696 10124 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.578415 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.579193 14904 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.579378 10199 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:18:42.581287 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:42.581583 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.584112 10196 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:42.585386 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.587435 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.587590 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.573745 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:42.555984 15040 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.003s
I20260709 13:18:42.582703 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.593461 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.546512 10049 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:42.536988 14709 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:42.556780 15084 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:42.555927 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:42.552656 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:42.554930 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.604357 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.604871 10130 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:42.562531 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 208382us
I20260709 13:18:42.566623 10197 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:42.606043 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:42.594611 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:42.607050 10201 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:42.607949 10202 raft_consensus.cc:2368] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:18:42.609351 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:42.505534 15019 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.607609 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:42.610796 10900 raft_consensus.cc:2368] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:42.568849 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:42.614208 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.575321 15135 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:42.619284 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:42.620731 10054 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:42.622354 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3164) took 1564 ms. Trace:
I20260709 13:18:42.586759 15133 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:42.634244 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:42.629014 10205 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:18:42.625053 10203 rpcz_store.cc:276] 0709 13:18:41.058200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.060843 (+  2643us) service_pool.cc:224] Handling call
0709 13:18:42.622152 (+1561309us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:42.622285 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70288896}
I20260709 13:18:42.610682 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:42.632212 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 371567us
I20260709 13:18:42.603312 14749 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:42.642328 10122 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 13:18:42.609892 15134 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:42.647476 10126 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:42.579211 15083 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:18:42.620107 14759 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:42.618768 14468 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:42.603017 15040 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.662786 10130 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:18:42.633118 14748 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:42.666712 10129 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:18:42.647846 15134 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.687986 10122 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:18:42.663333 14749 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:42.704039 10124 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 8.
I20260709 13:18:42.705941 10123 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:18:42.717656 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 202673us
I20260709 13:18:42.692834 15134 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:42.706808 14759 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:42.726589 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 149646us
I20260709 13:18:42.676759 15144 raft_consensus.cc:2764] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:18:42.721671 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83991.9us
W20260709 13:18:42.725653 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85480.1us
W20260709 13:18:42.731724 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 122408us
I20260709 13:18:42.734793 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:42.736361 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:42.749558 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 271397us
I20260709 13:18:42.677439 14468 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:42.761842 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.706693 14748 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:18:42.732381 14759 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:42.800032 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65334.4us
I20260709 13:18:42.801088 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:42.765836 14797 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.007s
W20260709 13:18:42.823614 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61898.3us
I20260709 13:18:42.825945 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:42.832631 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59195.8us
I20260709 13:18:42.835073 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.797062 14759 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:42.797008 15107 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:42.797572 15019 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.844401 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 110273us
I20260709 13:18:42.906826 10197 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
W20260709 13:18:42.946636 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 145633us
I20260709 13:18:42.947788 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:42.949481 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:42.823792 14749 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:42.951478 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 204985us
I20260709 13:18:42.900576 10129 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:42.842159 14882 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.938783 15040 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.871445 15161 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:42.925904 15160 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:42.970513 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 224777us
I20260709 13:18:42.973850 10095 leader_election.cc:400] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:18:42.982929 10095 leader_election.cc:403] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:42.832312 14797 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.837180 15140 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.303s
W20260709 13:18:42.978229 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 144379us
I20260709 13:18:42.954025 15073 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.936707 15164 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:42.849344 15019 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:42.901393 15152 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:42.953687 14468 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:42.982802 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:42.860545 14826 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 13:18:42.992343 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.994560 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.002763 10900 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:42.866837 15158 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:18:42.864665 15141 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.236s
I20260709 13:18:42.903875 15135 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:43.012629 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 175232us
I20260709 13:18:43.016112 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:42.987448 14882 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:43.016739 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:43.002230 15040 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:43.016819 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65961.1us
I20260709 13:18:43.025357 15140 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:18:43.039544 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.040552 10203 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:43.044026 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.041800 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.032581 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.033588 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.048533 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.050836 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.052347 15019 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:43.052505 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.053277 15042 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:43.031105 15175 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.055178 15167 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:42.991386 15160 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.056643 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:43.057705 15160 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:43.058307 10197 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.019666 15174 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:43.059515 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.060170 14826 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:43.060609 14826 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:43.061600 15152 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:43.026037 15040 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:43.062682 15040 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:43.022836 15165 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:43.021894 15105 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:43.069295 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48783.1us
I20260709 13:18:43.061741 14826 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.093981 10900 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:43.119659 14826 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.139379 15157 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:43.140413 15157 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.122020 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.126530 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:43.127867 10048 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:43.141664 10122 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:43.127678 15140 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:43.141624 15157 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.128887 15135 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:43.141696 10055 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:18:43.142719 15140 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.129664 15174 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:43.146502 15174 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.130640 15179 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:43.148697 15179 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:43.149221 15179 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.094026 15134 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.157047 15134 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:43.158390 15087 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.160171 15087 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.007176 15073 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.167950 14907 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:43.168955 14907 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.171068 14925 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:43.174232 14709 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.175647 14709 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:43.176388 15084 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.543s	user 0.013s	sys 0.000s
I20260709 13:18:43.176915 15084 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.179514 14797 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:43.181423 15073 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:43.075347 15167 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:43.183045 15167 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.186614 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2005) took 1506 ms. Trace:
I20260709 13:18:43.187288 10205 rpcz_store.cc:276] 0709 13:18:41.680547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.740028 (+ 59481us) service_pool.cc:224] Handling call
0709 13:18:42.622782 (+882754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:42.635797 (+ 13015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.186335 (+550538us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b540039f150. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:43.186525 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895460736}
I20260709 13:18:43.132616 10124 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:43.188925 14861 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:43.189554 14861 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:43.199106 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 245772us
W20260709 13:18:43.106458 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 123770us
W20260709 13:18:43.217332 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 183822us
I20260709 13:18:43.218798 10129 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:18:43.221524 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 201801us
I20260709 13:18:43.222755 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.224123 10048 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:43.224947 10203 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:43.224855 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.228818 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.231151 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.231472 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:43.235378 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 243137us
I20260709 13:18:43.235664 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.236207 10892 raft_consensus.cc:2368] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:43.237289 10095 leader_election.cc:400] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:43.237440 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:43.238502 10203 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:43.238636 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.238883 10095 leader_election.cc:403] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:43.244463 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.246191 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.246249 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.249964 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.257690 10095 leader_election.cc:400] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:43.258213 10095 leader_election.cc:403] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:18:43.258911 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:43.272781 10201 raft_consensus.cc:2368] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:43.273940 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.190701 14861 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.270624 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:43.261008 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.297005 10048 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:43.240067 15134 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:18:43.297731 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:43.300601 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.242322 15165 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:43.301584 10055 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:43.301632 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100715us
I20260709 13:18:43.300617 10131 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:43.304216 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85292.7us
I20260709 13:18:43.305347 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:43.306856 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66290.1us
W20260709 13:18:43.308382 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 50868.7us
W20260709 13:18:43.308532 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62380.6us
I20260709 13:18:43.309894 10095 leader_election.cc:400] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:43.233397 14871 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:18:43.271543 15133 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:43.309417 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.313409 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.313858 14942 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:43.322359 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.323297 15158 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:43.323787 14907 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:43.302742 15072 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:43.318580 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:43.319020 10896 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:18:43.318006 10898 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
W20260709 13:18:43.312875 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71942.8us
I20260709 13:18:43.314291 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.339304 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.292405 15084 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:43.258806 15170 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:43.350430 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:43.344045 10055 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:18:43.350735 10894 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 8: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:43.354887 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.275444 15141 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:18:43.310653 15169 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
W20260709 13:18:43.396517 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47932.3us
I20260709 13:18:43.358726 15140 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:43.363359 15152 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:43.374686 14942 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:43.374934 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52667.6us
I20260709 13:18:43.398303 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:43.346251 15144 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:43.379460 15171 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:18:43.379437 15174 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:43.315024 15160 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:43.415398 10125 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:43.417819 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.343081 10203 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:43.338201 15158 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:43.385088 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63733.6us
I20260709 13:18:43.302175 15105 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:18:43.386199 15072 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:43.387672 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.387861 10201 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:43.388325 15175 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:18:43.388537 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.388871 15170 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:43.389590 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:43.405069 15144 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:43.446915 10197 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:43.390635 10200 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:43.390795 15040 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.379411 15019 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:43.392817 10900 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:43.392635 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:43.396353 15135 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:18:43.393018 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50144.1us
I20260709 13:18:43.394909 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:43.358055 14759 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.001s
I20260709 13:18:43.425086 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:43.425781 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.426855 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:43.397043 14882 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:43.341648 15182 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:43.430300 15152 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:43.404974 15006 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:18:43.455881 15170 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.457528 14841 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:43.459661 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.460909 10197 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:18:43.464938 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.592031 10200 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:43.432008 14942 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:43.431663 15158 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.483707 15040 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:43.484603 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86381.1us
I20260709 13:18:43.485301 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:43.441663 15174 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:18:43.598161 15169 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:43.495347 15176 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:43.598845 15169 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:43.600222 15169 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:43.600536 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 205720us
I20260709 13:18:43.601508 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:43.595877 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 131042us
I20260709 13:18:43.510044 10131 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:43.518702 15157 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:43.522128 15072 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.499547 15144 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:43.536095 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:43.620262 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84286.8us
I20260709 13:18:43.536409 14861 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:43.537963 15148 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.774s	user 0.001s	sys 0.006s
I20260709 13:18:43.542418 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2287) took 1157 ms. Trace:
I20260709 13:18:43.622618 10123 rpcz_store.cc:276] 0709 13:18:42.384903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.385067 (+   164us) service_pool.cc:224] Handling call
0709 13:18:43.542165 (+1157098us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:43.542336 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689245312}
I20260709 13:18:43.555552 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2347) took 2120 ms. Trace:
I20260709 13:18:43.557835 15160 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:18:43.559219 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2284) took 1175 ms. Trace:
I20260709 13:18:43.560602 10054 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:18:43.638955 10054 pending_rounds.cc:85] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:18:43.639307 10054 pending_rounds.cc:107] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:18:43.639660 10054 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.561074 15179 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.562577  9953 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:18:43.589607 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 163959us
W20260709 13:18:43.591665 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 132103us
W20260709 13:18:43.643985 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.644757 10200 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.646757 10197 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.597714 15006 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:43.602854 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.584048 14759 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.616935 14833 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:43.677533 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.678620 10051 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:43.638255 10125 rpcz_store.cc:276] 0709 13:18:42.383484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.386041 (+  2557us) service_pool.cc:224] Handling call
0709 13:18:43.558863 (+1172822us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001275d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:43.559094 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363167488}
W20260709 13:18:43.666777 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64010.6us
I20260709 13:18:43.595055 15135 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:43.671720 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.598898 15174 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:43.688989 10049 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:43.677604 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.603523 15176 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.691877 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.612532 15157 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.589736 15087 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:43.681586 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.594748 14942 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:43.621485 14882 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:43.618686 15126 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.683876 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.697193 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:43.698113 10047 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:43.702077 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:43.639225 15040 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:43.622020 14964 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:18:43.635049 10053 rpcz_store.cc:276] 0709 13:18:41.435513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.133393 (+697880us) service_pool.cc:224] Handling call
0709 13:18:42.164029 (+ 30636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.208854 (+ 44825us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:42.255239 (+ 46385us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.432325 (+177086us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.432384 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:42.432436 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.552790 (+1120354us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b18003ad680. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419
0709 13:18:43.555198 (+  2408us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b54000da054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:43.555254 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.555461 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31380,"prepare.queue_time_us":53911,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":17590,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":26196,"spinlock_wait_cycles":108829696,"thread_start_us":159051,"threads_started":1,"wal-append.queue_time_us":155613,"wal-append.run_cpu_time_us":1470,"wal-append.run_wall_time_us":1004325}
I20260709 13:18:43.694674 15171 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:43.665926 15006 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:43.672441 14748 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:43.737318 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 142320us
I20260709 13:18:43.738461 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:43.738744 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57332.9us
I20260709 13:18:43.740147 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.743569 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.725991 14882 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.748185 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:43.705701 14942 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:43.664264 14826 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:18:43.701371 15174 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.715199 15140 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:43.752852 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:43.754325 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.683357 14709 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:18:43.768589 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.737938 15094 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.733392 15040 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:43.778412 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.780879 10131 raft_consensus.cc:2368] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:18:43.780663 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.779003 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:43.743312 15175 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.747857 15135 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:43.750154 15076 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.753600 15141 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:18:43.821302 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100879us
I20260709 13:18:43.798661 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.802979 15043 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:43.809828 10128 raft_consensus.cc:2368] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:18:43.809456 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.810917 10125 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.714998 15182 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:18:43.760876 15160 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:43.637246 15117 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.130s
I20260709 13:18:43.758142 15171 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:43.818231 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65461.3us
I20260709 13:18:43.909602 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:43.820712 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:43.821024 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.822170 14925 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:43.822435  9953 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 10 to 11, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 11 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.824110 15179 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:43.794935 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.832696 10047 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:43.842444 10051 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:43.843318 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 147266us
I20260709 13:18:43.845141 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.846261 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.849752 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.850100 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:43.874081 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.920763 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:43.900799 15140 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:43.904672 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:43.923038 10131 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:43.923677 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88104.7us
I20260709 13:18:43.925231 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.827450 14748 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:43.931594 10127 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:18:43.926268 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.933360 10126 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:43.926832 14709 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:43.936314 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.771103 15094 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.837219 15043 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:43.945019 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.936591 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:43.978086 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:43.979187 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
W20260709 13:18:43.981343 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 160674us
I20260709 13:18:43.921938 15087 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:43.995886 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65731.7us
I20260709 13:18:43.837554 15175 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:43.870285 15138 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:18:43.837137 15076 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.006557 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:43.778434 15157 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.018627 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:43.815665 15006 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.976001 15179 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:43.994326 15045 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:43.943708 15019 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
W20260709 13:18:44.114352 15152 consensus_queue.cc:1175] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 12 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.7-11.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:44.117822 10047 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:44.118276 10047 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:44.123955 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.124835 10197 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:44.082882  9953 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 9 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 10 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.053357 10051 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:18:44.129431 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:44.135311 10200 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:18:44.126463 10047 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.143841 10047 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:44.148793 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.153627 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:44.126387  9953 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:44.158645 10047 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:44.162621 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.163542 10200 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:44.051271 10048 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:44.085583 15176 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.088246 14759 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.090490 15116 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:44.090844 15182 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.093935 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87481.6us
I20260709 13:18:44.182368 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.164850 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:44.095510 15194 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:18:44.096510 14871 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:44.097021 15106 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.098304 15192 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:18:43.942620 15117 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:18:43.915941 15174 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:43.929912 14882 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.100564 15175 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.170837 14748 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:44.100932 15043 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.239356 15210 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:44.242990 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60759.3us
I20260709 13:18:44.103391 15140 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.046376 15179 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:44.105705 15094 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:44.111707 15157 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.048244 14797 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:44.094255 15211 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:44.168141 15135 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 13:18:44.215337 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65991.9us
I20260709 13:18:44.182010  9953 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 8 to 9, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:18:44.218871 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 298171us
I20260709 13:18:44.218163 14925 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:44.230715 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:44.100638 15076 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:44.274319 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.245214 15158 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.001s
W20260709 13:18:44.274422 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43858.8us
I20260709 13:18:44.183339 15116 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:44.233552 14749 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.102807 15163 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:44.246611 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:44.251048 10127 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:44.251327 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 314827us
I20260709 13:18:44.253062 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 773) took 1258 ms. Trace:
I20260709 13:18:44.011065 15040 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.261662 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2020) took 1038 ms. Trace:
I20260709 13:18:44.265606 10125 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:44.271044 10019 leader_election.cc:400] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:18:44.302239 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.241842 14871 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:44.256157 14795 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:44.295552 10899 rpcz_store.cc:276] 0709 13:18:42.981811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.981968 (+   157us) service_pool.cc:224] Handling call
0709 13:18:42.999677 (+ 17709us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.202024 (+202347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.202078 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:43.260295 (+ 58217us) log.cc:844] Serialized 52 byte log entry
0709 13:18:43.571462 (+311167us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:43.571510 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.583613 (+ 12103us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540053b454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:43.583696 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.240027 (+656331us) raft_consensus.cc:1685] finished
0709 13:18:44.240054 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.240305 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1160,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":12689,"spinlock_wait_cycles":383551232,"thread_start_us":232822,"threads_started":1,"wal-append.queue_time_us":440439}
I20260709 13:18:44.309537 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.257361 15167 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:44.226652 15106 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.312491 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:44.296075 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.256480 15179 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:44.275261 15126 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
I20260709 13:18:44.262913 15043 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:44.201088 15182 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:44.314378 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.299563 10199 rpcz_store.cc:276] 0709 13:18:43.223157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.224124 (+   967us) service_pool.cc:224] Handling call
0709 13:18:43.243984 (+ 19860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.296902 (+ 52918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.296937 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.309591 (+ 12654us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.714170 (+404579us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:43.714251 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.716889 (+  2638us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.261389 (+544500us) raft_consensus.cc:1685] finished
0709 13:18:44.261416 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.261585 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101829,"raft.queue_time_us":55767,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10432,"thread_start_us":342127,"threads_started":1,"wal-append.queue_time_us":532727}
I20260709 13:18:44.289736 14759 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:44.327849 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.330966 10019 leader_election.cc:400] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:18:44.333163 10019 leader_election.cc:403] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:18:44.334244 10205 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:44.311878 15167 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.316768 15211 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:44.340821 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 431316us
I20260709 13:18:44.347954 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:44.349457 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:44.324694 15126 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:44.261474 15210 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.290814 14925 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:44.360335 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 166882us
I20260709 13:18:44.361791 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.327853 14759 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:44.338373 14861 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.000s
I20260709 13:18:44.289808 15158 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.325955 15133 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:44.307740 15185 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:18:44.331346 15215 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:44.365957 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.329726 14797 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:44.369899 10202 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:44.308832 14749 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.357632 15167 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.385244 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:44.391152 15223 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:44.368953 14759 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:44.395150 15220 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:18:44.396103 15218 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:18:44.401466 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60556.4us
I20260709 13:18:44.402194 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:44.467020 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64911.8us
I20260709 13:18:44.362780 15199 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:18:44.468252 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:44.363921 14942 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.004s
W20260709 13:18:44.393229 15206 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:18:44.368237 15152 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.444700 10124 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:44.383613 15175 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:44.368973 15156 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:44.323355 14748 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:44.402319 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.380862 15133 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:44.472159 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.433069 14709 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:44.474421 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 125039us
W20260709 13:18:44.485380 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 159608us
I20260709 13:18:44.488248 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:44.470569 15215 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:44.492791 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.356700 15116 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.382063 14871 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.515591 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.488085 15223 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:44.529065 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 192031us
I20260709 13:18:44.476121 14942 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.485455 15232 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:44.505882 15199 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.504731 15211 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:44.550837 15231 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:44.513200 15185 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:18:44.487773 15156 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.518390 14797 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:44.554302 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66122.3us
I20260709 13:18:44.533974 15215 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:44.555425 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60195.7us
W20260709 13:18:44.561817 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59426.7us
I20260709 13:18:44.652333 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:44.700067 10205 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:44.586665 14925 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:44.657004 15242 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.002s
I20260709 13:18:44.666277 15200 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.712417 15045 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.001s
I20260709 13:18:44.712193 15200 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.713208 15045 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:44.666497 15210 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:44.666781 15228 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:18:44.668604 15241 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:18:44.668906 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:44.669559 15245 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
I20260709 13:18:44.675359 15188 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:18:44.684172 14942 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.689662 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:44.689471 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:44.634567 14749 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:44.616897 15144 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.952s	user 0.005s	sys 0.009s
I20260709 13:18:44.612813 15158 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:44.561991 15169 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.961s	user 0.012s	sys 0.005s
I20260709 13:18:44.652523 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.551628 15231 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:44.718143 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:44.592648 15182 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.552930 15106 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:44.721866 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:44.598135 15173 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.210s
I20260709 13:18:44.788277 15210 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:44.543037 15133 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.792330 15133 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:44.767161 15083 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.781955 10053 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:18:44.786608 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 134167us
I20260709 13:18:44.800320 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:44.554528 15199 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.774156 10128 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:44.794515 10126 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:44.802052 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.803352 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:44.809293 10053 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:44.820366 10130 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:44.823537 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:44.824743 14748 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:44.825321 14748 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:44.825755 14748 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:44.787088 15231 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:44.827886 10049 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:44.818642 15210 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:44.784632 14795 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:18:44.835203 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:44.813099 15101 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:44.790740 15169 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.815780 15179 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.780485 15144 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.809638 15083 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.841056 15101 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.868288 10047 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:44.868547 10130 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:44.845402 15236 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:44.848649 15223 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:44.869957 14759 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:44.875546 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.883957 15179 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.876510 10204 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:44.886150 15169 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:44.870573 15200 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:44.888453 14759 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:44.872097 15236 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:44.865299 15101 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:44.889863 15193 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.786s	user 0.000s	sys 0.004s
I20260709 13:18:44.896700 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:44.902280 10055 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:44.888538 15144 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:44.906034 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.916615 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:44.923520 15072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.006s	sys 0.004s
W20260709 13:18:44.924307 15072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.006s	sys 0.004s
I20260709 13:18:44.940033 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.948458 10205 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:44.950645 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.950940 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:44.956284 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.925984 15144 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:44.958436 15230 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:18:44.960057 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:44.965289 10055 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:44.958513 15083 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:44.889958 15236 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:44.963037 15199 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:44.964761 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.946334 15170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.021s	sys 0.021s
I20260709 13:18:44.977006 15156 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:44.981114 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:44.984104 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:44.990047 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2395) took 1217 ms. Trace:
I20260709 13:18:44.964653 15152 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:18:44.983603 15157 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.003s
W20260709 13:18:45.004093 15254 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:18:45.018466 10056 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:44.992051 15230 consensus_queue.cc:1243] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:18:45.006397 15167 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.002s
I20260709 13:18:45.020444 15167 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.021703 15199 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:45.022907 10899 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:45.024350 15167 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.025628 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:45.026391 10199 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.003578 15236 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:45.029158 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 607) took 1408 ms. Trace:
I20260709 13:18:45.029872 10204 rpcz_store.cc:276] 0709 13:18:43.620481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.620637 (+   156us) service_pool.cc:224] Handling call
0709 13:18:43.664496 (+ 43859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.885415 (+1220919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.028705 (+143290us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:45.029029 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776450432}
I20260709 13:18:45.033826 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:45.036285 15006 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:44.985047 15170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.022s	sys 0.022s
I20260709 13:18:44.992331 10050 rpcz_store.cc:276] 0709 13:18:43.773048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.774312 (+  1264us) service_pool.cc:224] Handling call
0709 13:18:43.822481 (+ 48169us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.858464 (+ 35983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.858496 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:43.895547 (+ 37051us) log.cc:844] Serialized 52 byte log entry
0709 13:18:44.314365 (+418818us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:44.322702 (+  8337us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.361073 (+ 38371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.361171 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.976588 (+615417us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b18001fd320. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:44.981270 (+  4682us) raft_consensus.cc:1685] finished
0709 13:18:44.989682 (+  8412us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.989954 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82050,"prepare.queue_time_us":15128,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6753,"raft.queue_time_us":232906,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":80710,"spinlock_wait_cycles":114194560,"thread_start_us":372873,"threads_started":2,"wal-append.queue_time_us":464708}
I20260709 13:18:45.037779 15215 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:45.004812 15249 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:18:45.066627 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:45.073707 10052 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:44.969832 15144 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:44.984027 15231 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.066195 15167 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:18:45.065231 15101 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.110378 10126 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:45.100148 15188 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:45.122800 15216 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:45.166536 10197 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:45.100986 14748 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.107072 15170 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:18:45.189308 10049 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:45.200736 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:45.205325 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.209661 15266 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:18:45.208372 15045 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:18:45.249190 15263 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:18:45.249275 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 613) took 1126 ms. Trace:
I20260709 13:18:45.250068 10202 rpcz_store.cc:276] 0709 13:18:44.123039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.125618 (+  2579us) service_pool.cc:224] Handling call
0709 13:18:44.125928 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.432085 (+306157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.248828 (+816743us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54002df0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:45.249179 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560541568}
I20260709 13:18:45.253042 10202 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:45.243566 15157 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.256117 15157 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.258621 15157 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.260933 14748 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:45.213893 15171 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.266404 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.270753 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:45.271041 15223 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:45.272346 15222 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.273083 15144 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:45.272986 15222 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.274286 15101 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:45.246727 15045 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.277486 15045 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:45.292863 15171 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.300590 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 605) took 1003 ms. Trace:
I20260709 13:18:45.301185 10047 rpcz_store.cc:276] 0709 13:18:44.297425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.298029 (+   604us) service_pool.cc:224] Handling call
0709 13:18:44.324180 (+ 26151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.868871 (+544691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.300233 (+431362us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54002364d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:45.300499 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250829184}
I20260709 13:18:45.319228 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2342) took 1065 ms. Trace:
I20260709 13:18:45.319761 10127 rpcz_store.cc:276] 0709 13:18:44.254129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.296649 (+ 42520us) service_pool.cc:224] Handling call
0709 13:18:44.296954 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.297165 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:44.387425 (+ 90260us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.318727 (+931302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.318806 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.318883 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.318966 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.319152 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":743220,"prepare.queue_time_us":227801,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":259037,"threads_started":2,"wal-append.queue_time_us":274827}
I20260709 13:18:45.321296 15160 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.321985 15160 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.324007 14907 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:45.325018 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:45.326407 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:45.327584 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:45.328055 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.323858 15200 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:45.330682 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2331) took 1083 ms. Trace:
I20260709 13:18:45.354180 14833 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:45.355409 15005 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:45.355985 14833 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:45.360786 15043 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:45.361454 15043 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:45.366003 15265 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.188s	user 0.004s	sys 0.000s
I20260709 13:18:45.369319 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:45.326968 14955 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:45.332779 15230 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:45.412833 10199 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:45.368404 14942 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.379863 15231 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.367511 14925 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.388634 15174 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:45.388190 15040 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.372820 15140 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.378228 15233 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:45.375588 15158 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.362684 15043 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.397624 14955 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:45.428573 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.428622 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.434665 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:45.435312 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.404510 15176 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:45.433826 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.455103 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:45.400490 15230 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:45.451390 10127 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.420145 15171 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.436309 15233 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:45.452174 10131 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:45.435539 14759 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.419866 15238 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:45.402333 15253 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:45.473707 10131 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:45.472230 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:45.470304 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:45.436631 15224 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:45.476215 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:45.481320 14955 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.476181 14759 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.488351 14955 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:45.489960 14759 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:45.495594 15268 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:18:45.448050 15140 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.425958 14942 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.471714 15043 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:45.507725 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:45.481498 15244 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:18:45.430881 14925 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.524887 10126 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:45.526196 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.534759 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.534541 15230 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.535772 10131 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:45.536105 15230 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.443243 15040 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.506729 15224 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.540172 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.541184 10196 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:45.522773 15141 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:45.542263 15233 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.543876 15233 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:45.545157 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65057.3us
I20260709 13:18:45.545747 15217 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.546407 15217 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.547422 15188 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:45.548961 15072 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:45.551601 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2340) took 1301 ms. Trace:
I20260709 13:18:45.552191 10123 rpcz_store.cc:276] 0709 13:18:44.250503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.277255 (+ 26752us) service_pool.cc:224] Handling call
0709 13:18:44.308408 (+ 31153us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:44.308869 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.308904 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:44.978486 (+669582us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b2c0011b9a8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:44.979035 (+   549us) log.cc:844] Serialized 52 byte log entry
0709 13:18:45.058983 (+ 79948us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.068276 (+  9293us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.365218 (+296942us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:45.416123 (+ 50905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.416253 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.551234 (+134981us) raft_consensus.cc:1685] finished
0709 13:18:45.551264 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.551513 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":347424,"prepare.queue_time_us":10712,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":16877,"raft.queue_time_us":350881,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7996,"spinlock_wait_cycles":19178496,"thread_start_us":78524,"threads_started":1,"wal-append.queue_time_us":98877}
W20260709 13:18:45.557818 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46990.5us
I20260709 13:18:45.563858 15171 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.564597 15171 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.331179 10122 rpcz_store.cc:276] 0709 13:18:44.247371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.251550 (+  4179us) service_pool.cc:224] Handling call
0709 13:18:44.251842 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.252096 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:44.465975 (+213879us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.538905 (+ 72930us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.009005 (+470100us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1
0709 13:18:45.009989 (+   984us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:45.310166 (+300177us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240
0709 13:18:45.312051 (+  1885us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1
0709 13:18:45.313589 (+  1538us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:45.314506 (+   917us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b18004070e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:18:45.314749 (+   243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.330348 (+ 15599us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:45.330414 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.330498 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.330617 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":807322,"prepare.queue_time_us":255305,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":103,"raft.queue_time_us":809700,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":10075,"spinlock_wait_cycles":95717504,"thread_start_us":223146,"threads_started":2,"wal-append.queue_time_us":65459}
I20260709 13:18:45.570536 15171 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.572185 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:45.572901 10123 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:45.574491 15271 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:45.575764 10124 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:45.579295 15141 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:45.476851 15253 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.527966 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.580117 10896 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:45.585402 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.587801 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.588636 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:45.589394 10053 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:45.590778 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:45.591665 10122 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.499771 15184 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:45.447259 15158 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.600318 10892 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:18:45.625720 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.602458 10128 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:18:45.628887 10128 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:45.609371 10056 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:18:45.597292 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:45.616185 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69047.1us
I20260709 13:18:45.621913 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:45.529011 10199 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:45.694983 10127 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:45.624864 15221 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:18:45.685633 15171 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:18:45.705493 15140 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.692011 15215 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.707535 15215 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:45.694025 15244 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:18:45.724248 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:45.743480 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 392774us
I20260709 13:18:45.583081 15253 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:45.719894 15184 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:45.774902 10050 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:45.775796 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.776073 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.774392 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.12208e+06us
W20260709 13:18:45.783581 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 138128us
I20260709 13:18:45.781379 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:45.788936 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 157336us
I20260709 13:18:45.786468 10123 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:45.687651 15141 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.775413 15246 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:45.775015 15278 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:45.840839 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56031.7us
I20260709 13:18:45.844630 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:45.841919 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60655us
I20260709 13:18:45.873513 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:45.817703 15133 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:45.847981 15045 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.004s
W20260709 13:18:45.849876 15281 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:45.840629 15238 raft_consensus.cc:2764] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:18:45.905421 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:45.900453 15222 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:45.819451 15076 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:18:45.885712 15083 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:18:45.854377 15246 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:45.875031 15278 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:45.894198 14793 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.909332 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:45.914469 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 69925.3us
W20260709 13:18:45.919564 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65710us
I20260709 13:18:46.055949 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:45.925966 15160 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:45.926941 15158 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:45.953824 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.954772 14797 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:46.059825 14797 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:46.060850 14797 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.062079 14797 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:45.957381 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:45.967693 15116 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:45.924311 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.064271 15158 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.988407 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.996758 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:45.999832 10126 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:46.004719 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:46.032073 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:46.071879 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.905s (SENT) [suppressed 34 similar messages]
I20260709 13:18:46.122337 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.127007 10199 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:46.002074 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:46.079154 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 121858us
I20260709 13:18:46.129729 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:45.913295 14942 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.134194 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:46.134780 15140 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.123420 15217 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.135334 15171 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:46.135759 15217 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:46.136170 15171 consensus_queue.cc:1243] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:46.136881 14795 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:46.137504 14795 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:46.140084 15019 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:18:46.140947 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85118.8us
I20260709 13:18:46.141209 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:45.975512 15278 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:45.902840 15045 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:45.990188 14955 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.002s
I20260709 13:18:45.966432 15224 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:45.983788 15163 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:45.989455 15215 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.136197 15217 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.899205 15133 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.050586 15246 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.017004 10053 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:18:46.064518 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 155322us
W20260709 13:18:46.144037 15263 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 13:18:46.146674 10122 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 13:18:45.916172 15188 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:18:45.987090 15176 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.002s
I20260709 13:18:46.150087 15291 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:46.140617 15019 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:46.151516 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:45.850370 15184 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.155349  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:45.968025 14925 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.020246 15072 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.968788 15040 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.180835 15215 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.138666 14795 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.214118 15188 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.176024 15133 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.129010 15226 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:46.235430 15152 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.973047 15169 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:46.154428 14942 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:46.256824 15188 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.185335 10122 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:45.978894 14793 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.173271 15163 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:46.177390 15217 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:46.181316 15230 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.003s
I20260709 13:18:46.276661 15152 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.296656 10051 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:46.296404 15163 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.281605 15284 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:46.253479 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102044us
I20260709 13:18:46.064178 15253 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.253852 15176 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.242163 15072 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.297286 14793 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.277796 15169 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:46.269562 15170 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:46.315934 10125 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
W20260709 13:18:46.301276 15233 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.006s
W20260709 13:18:46.145200 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 143285us
I20260709 13:18:46.326702 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.327349 10122 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:46.179320 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:46.330783 14942 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:46.330940 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 151839us
I20260709 13:18:46.332346 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:46.332743 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:46.333146 10055 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:46.315246 15163 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.349212 10129 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.273301 15019 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.317785 15140 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:46.345849 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:46.342617 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.340054 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.319188 15215 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.364037 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:46.369585 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.354327 10196 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:46.341418 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.375062 15215 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.377023 15253 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.377593 15215 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.378196 15215 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:46.379626 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.380589 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47947.3us
I20260709 13:18:46.381815 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:46.382601 10896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.014s	sys 0.055s
W20260709 13:18:46.383785 10896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.014s	sys 0.055s
I20260709 13:18:46.380733 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:46.390024 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.334618 15170 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:46.349292 15226 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.407786 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:46.393222 15133 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.325572 15176 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:46.419952 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.378732 15215 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.409992 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:46.491508 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 419652us
I20260709 13:18:46.394495 15233 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:46.394233 15253 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:46.399492 15040 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:46.547521 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 406390us
W20260709 13:18:46.492429 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.499763 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:46.503623 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:46.549000 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:46.424764 14925 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:46.399775 15045 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:46.485553 15140 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.516451 15170 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.491252 15169 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.495913 15233 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.524094 15072 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.425033 15226 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.411293 15019 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:46.626920 15116 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.006s
I20260709 13:18:46.496326 15141 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:46.533011 15176 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.631624 10899 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:18:46.631803 15176 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:46.632377 15279 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.632653 15176 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:46.633181 15279 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.634409 15176 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:46.535820 15215 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.635349 15169 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:46.635828 15215 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.636461 15215 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.550407 10123 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:46.641196 15215 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.550442 10126 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
W20260709 13:18:46.553459 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 143588us
I20260709 13:18:46.567626 15080 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:46.556938 15238 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:46.655696 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.653656 10096 leader_election.cc:400] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 7. Current term is 8.
I20260709 13:18:46.657045 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:46.570564 15233 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:46.664404 10863 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.718s (SENT)
I20260709 13:18:46.669108 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:46.683451 15226 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:46.686748 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.691984 15286 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.692674 15286 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.690698 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 582) took 1112 ms. Trace:
I20260709 13:18:46.695811 15152 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.566045 15140 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.699295 15213 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:46.699990 15213 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.701409 15140 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.702452 15213 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.695361 10899 rpcz_store.cc:276] 0709 13:18:45.578130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.578272 (+   142us) service_pool.cc:224] Handling call
0709 13:18:45.578738 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.632105 (+1053367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.690369 (+ 58264us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540063fbd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:46.690611 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420249984}
I20260709 13:18:46.557618 15040 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.707855 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.708630 10202 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:46.652887 15080 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:46.709218 15040 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.710075 15080 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.711302 15080 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:46.711719 14797 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.649s	user 0.015s	sys 0.002s
I20260709 13:18:46.713501 14793 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.714239 14793 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.711323 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.738821 15188 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.741299 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.742020 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.742736 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:46.746109 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2408) took 2598 ms (client timeout 2567 ms). Trace:
I20260709 13:18:46.747869 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:46.751230 10051 rpcz_store.cc:269] 0709 13:18:44.121319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.129140 (+  7821us) service_pool.cc:224] Handling call
0709 13:18:46.719432 (+2590292us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:46.719571 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4981641984}
I20260709 13:18:46.739986 15140 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:18:46.749985 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92803.2us
I20260709 13:18:46.750204 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.754819 10892 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:46.756845 10130 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:46.760527 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.762620 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.775156 10130 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.776973 10123 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.777192 10126 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.780488 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.787981 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:46.788546 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.670730 15217 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.757543 15040 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:46.790468 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.791301 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.791576 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.757445 15208 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:46.790994 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:46.790449 10199 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:46.776491 15152 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.662663 15238 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.802196 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2375) took 1321 ms. Trace:
I20260709 13:18:46.773819 15141 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:46.783318 14925 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.808003 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.776786 15215 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.807242 10126 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:46.810525 10129 rpcz_store.cc:276] 0709 13:18:45.480537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.480734 (+   197us) service_pool.cc:224] Handling call
0709 13:18:45.481170 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.353832 (+872662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.801830 (+447998us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540027d3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:46.802105 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995633536}
I20260709 13:18:46.809976 10123 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:46.824623 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:46.825647 15306 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:46.827838 15217 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:46.764878 15284 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:18:46.832089 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.832435 15233 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.813661 15208 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.833988 10899 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:46.835707 15208 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:46.837265 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.838231 15238 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.819447 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:46.840520 10202 raft_consensus.cc:606] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:46.840981 14793 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.813257 15152 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.844336 15152 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:46.853751 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.854985 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:46.856024 10051 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:46.853089 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 585) took 1142 ms. Trace:
I20260709 13:18:46.861052 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.865038 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:46.866609 10053 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:46.858527 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:46.803325 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 250752us
I20260709 13:18:46.816577 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.790815 15286 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:46.871666 10130 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:46.872534 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56073.3us
I20260709 13:18:46.870929 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:46.875012 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.861119 10896 rpcz_store.cc:276] 0709 13:18:45.710421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.710587 (+   166us) service_pool.cc:224] Handling call
0709 13:18:45.711239 (+   652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:45.749568 (+ 38329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.749630 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:45.763512 (+ 13882us) log.cc:844] Serialized 26 byte log entry
0709 13:18:45.976046 (+212534us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b48000b25d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:18:46.846944 (+870898us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b1800425200. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1
0709 13:18:46.848050 (+  1106us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:46.848116 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.848252 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:46.848321 (+    69us) raft_consensus.cc:1685] finished
0709 13:18:46.848339 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.853013 (+  4674us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21590,"raft.queue_time_us":38103,"raft.run_cpu_time_us":2742,"raft.run_wall_time_us":220807,"spinlock_wait_cycles":28395776,"thread_start_us":593942,"threads_started":1,"wal-append.queue_time_us":601513}
I20260709 13:18:46.877485 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:46.880812 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.881018 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.881727 10202 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:46.846033 15253 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:46.851209 15184 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.859472 15305 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.859951 15116 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.884582 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:18:46.895165 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.895325 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:46.900578 10127 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 8: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:46.879601 15171 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:46.880527 15040 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:46.871910 15233 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:46.903530 10125 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:18:46.904886 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:46.905458 10126 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:46.908104 10127 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:18:46.893015 15286 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.897565 15140 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:46.922430 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.852080 15215 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.924961 10199 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:46.919219 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:46.922606 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:46.927847 10203 raft_consensus.cc:606] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:46.928414 10203 raft_consensus.cc:614] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:46.923763 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.916489 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.908640 15217 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:46.947083 10197 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:46.884891 10051 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:46.940110 15152 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:46.940335 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:46.939177 15140 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:46.927214 15286 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.944078 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:46.937537 15174 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:46.957346 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:46.957821 15152 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.953894 15040 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:18:46.960116 10201 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:46.903291 15305 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:46.905885 15116 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:46.890235 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:46.946815 15217 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:46.969648 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.025013 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136858us
I20260709 13:18:47.026376 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:46.972891 15163 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.605s	user 0.015s	sys 0.000s
I20260709 13:18:46.966122 15188 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:47.033345 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:47.010023 15217 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:47.037596 15312 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.036787 15019 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:47.028717 15199 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.031198 15233 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:18:47.084741 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.093104 10201 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:47.095969 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:47.039433 15163 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:18:47.088972 14925 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:47.092705 15238 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:47.131343 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.049435 15312 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.144672 10131 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:47.047421 15140 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.095279 15199 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:47.160730 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.152037 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:47.165444 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3200) took 1177 ms. Trace:
I20260709 13:18:47.168522 10128 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:47.167088 10196 rpcz_store.cc:276] 0709 13:18:45.987882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.988064 (+   182us) service_pool.cc:224] Handling call
0709 13:18:47.165161 (+1177097us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54002df0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:47.165344 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522672384}
I20260709 13:18:47.189147 15305 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.192212 15305 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:47.192700 15305 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.193068 15305 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.178535 15215 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.178045 15313 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:47.152153 14925 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.137809 15238 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.227593 10200 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:47.270037 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.232713 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:47.297925 10053 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:47.302520 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.234097 15188 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:47.233783 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:47.309341 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:47.311574 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 276737us
I20260709 13:18:47.218341 15152 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.321651 15238 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.322341 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.220599 15116 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.220930 15313 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.245525 15184 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:18:47.311038 15176 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:47.325033 14925 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:47.327368 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.331393 15286 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:47.362494 10047 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:47.346762 10196 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:47.359539 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:47.297881 15215 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.235812 15279 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.407459 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.436905 10049 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:47.385005 10053 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:18:47.443805 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:47.407374 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 377369us
I20260709 13:18:47.446795 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:47.389321 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.426600 10123 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.274379 14793 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 13:18:47.419580  9949 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 17, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:47.435072 15215 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:47.455210 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 421959us
I20260709 13:18:47.448812 15176 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:47.462317 15315 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:18:47.462976 15176 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.477533 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.479918 10122 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:47.450006 15286 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:47.492975 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:47.493268 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46559.6us
I20260709 13:18:47.430262 15207 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:18:47.364521 15313 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:47.502913  9949 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.505568 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:47.352686 15188 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.496465 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.511076 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.499795 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:47.514218 10053 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:18:47.517773 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.522346 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.526381 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.528339 10122 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:47.528789 10050 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:47.531353 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.531661 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.503068 15279 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:47.534693 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.535480 10047 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 9.
I20260709 13:18:47.535657 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.537063 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.537431 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.532651 10129 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:47.499912 15233 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:47.501792 15152 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.538522 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.541126 10123 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:47.541329 10047 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:18:47.536867 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 440953us
I20260709 13:18:47.544430 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.545607 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.551987 10047 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:47.555541 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.536892 15279 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:47.511217 15221 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.559505 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.533278 15226 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.571367 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.549805 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:47.565428 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.572465 10129 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:47.573521 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.566905 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.503474 15286 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.571458 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.539028 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.575688 10050 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:18:47.530594 15163 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.537352 15312 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:47.584779 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.585273 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:47.585446 10050 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:47.585507 10129 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:47.584738 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.586796 10122 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:47.578001 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:47.588773 10047 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
W20260709 13:18:47.578810 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.001s	sys 0.025s
W20260709 13:18:47.589721 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.001s	sys 0.026s
I20260709 13:18:47.590197 10125 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:47.590374 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 633070us
I20260709 13:18:47.568779 15279 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:47.574231 15318 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.115s	user 0.001s	sys 0.001s
I20260709 13:18:47.589813 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:47.612367 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.618961 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.629906 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.642112 10053 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:47.564690 15152 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:47.666136 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.010s	sys 0.023s
I20260709 13:18:47.583312 15226 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.645031 15163 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:47.575001 15221 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:47.666946 15019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.036s
W20260709 13:18:47.677070 10128 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.001s
I20260709 13:18:47.677659 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.689711 10047 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:47.708060 10128 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:18:47.703504 10898 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.019s	sys 0.049s
W20260709 13:18:47.673704 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.010s	sys 0.023s
W20260709 13:18:47.728366 10898 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.023s	sys 0.052s
I20260709 13:18:47.715474 10049 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.728760 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.715767 15199 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.716979 15221 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.747857 10196 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:47.657591 15233 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:47.720878 15019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.008s	sys 0.037s
I20260709 13:18:47.721526 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.725975 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:47.759003 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:47.726871 15323 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.746893 10050 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:47.710927 15324 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:47.702869 15163 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.730136 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.732182 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:47.733206 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.771538 10049 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:47.771502 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.734303 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.773635 10128 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:47.775701 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.776968 10128 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:47.777544  9949 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 16, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.779160 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:47.718590 15226 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.676033 15312 raft_consensus.cc:2764] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:18:47.752588 15199 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.763705 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:47.727080 15332 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.755805 15221 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.763057 14925 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:47.734434 10124 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.016s	sys 0.017s
W20260709 13:18:47.805531 10124 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.017s	sys 0.018s
I20260709 13:18:47.779654  9949 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.781865 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:47.782053 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49959.4us
I20260709 13:18:47.757491 15019 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:47.793310 10096 leader_election.cc:400] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12.
I20260709 13:18:47.822093 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:47.794365 10051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.022s	sys 0.010s
I20260709 13:18:47.765676 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:47.823734 10051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.022s	sys 0.010s
W20260709 13:18:47.803093 10201 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.012s	sys 0.006s
W20260709 13:18:47.827859 10201 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.013s	sys 0.006s
I20260709 13:18:47.812223 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:47.766683 15215 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.769556 15324 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.776314 15163 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.830940  9949 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.789219 15300 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:18:47.833961 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.005s	sys 0.014s
W20260709 13:18:47.844753 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.006s	sys 0.014s
I20260709 13:18:47.699661 10052 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:47.843102 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.790830 15226 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.847193 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.856606 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.835093 15329 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:47.789538 15304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.019s
W20260709 13:18:47.850476 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 467801us
I20260709 13:18:47.851382 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:47.792359 15238 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:47.854063 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 617481us
I20260709 13:18:47.768857 15323 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:47.868228 10054 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.010s	sys 0.038s
W20260709 13:18:47.893172 10054 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.013s	sys 0.041s
I20260709 13:18:47.867249 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:47.894747  9949 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:47.857926 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.836884 15320 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:47.861898 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.844138 15324 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:47.856602 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.846822 15300 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:47.805007 15221 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.792657 15199 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.859584 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.924389 10052 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:47.859210 15304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.000s	sys 0.019s
I20260709 13:18:47.909130 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:47.914961 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47840.8us
I20260709 13:18:47.893087 15188 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:47.929786 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:47.857461 15286 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:47.931368 10053 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:47.854488 15328 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:47.892982 15312 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.940155 15221 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.920028 14925 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.941244 15329 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:47.941764 15329 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.942382 15329 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:47.934441 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:47.946465 15199 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.947319 10047 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:47.947425 15333 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:47.948154 15333 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.949388 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.920013 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.820353 15332 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.950256 10047 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:47.922097 15331 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:47.922636 15300 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:47.953387 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:47.907049 15320 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:47.950771 15333 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.894511 14795 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:18:47.952674 15230 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:47.976668 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:47.931574 15171 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.024s	user 0.015s	sys 0.002s
I20260709 13:18:47.963505 10053 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:47.965090  9949 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.988903  9949 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.993038 10052 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:47.994493 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.990643  9949 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.008765  9949 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:47.949808 15312 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:47.866459 15226 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:47.955603 15238 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.031100 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:47.993336 15174 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.004s
I20260709 13:18:47.975828 15230 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.978905 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:47.952203 15152 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:47.932742 15337 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:48.030135 15217 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.982s	user 0.013s	sys 0.003s
W20260709 13:18:47.968070 10127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.012s	sys 0.019s
I20260709 13:18:48.025235 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:47.960131 15300 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:47.974732 15215 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:48.023365 15171 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:48.044909 10049 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:18:48.049749 10127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.012s	sys 0.019s
I20260709 13:18:48.053457 10051 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:18:48.012843  9949 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:48.090832 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.934381 15019 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:48.025872 15176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.026s
W20260709 13:18:47.978219 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.013s
I20260709 13:18:48.042136 15174 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:48.043362 15230 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:48.177075 15140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.026s	sys 0.054s
W20260709 13:18:48.112944 15330 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
W20260709 13:18:48.181933 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.182075 10047 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:48.117298 15170 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:48.118237 15279 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.130909 10049 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:48.143668  9949 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 4 to 8, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:18:48.051756 15320 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.148633 15323 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.149377 15329 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:48.152381 15305 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.959s	user 0.015s	sys 0.000s
I20260709 13:18:48.047624 15337 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.096199 15215 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.159861 10052 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:18:48.039086 15238 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:48.167397 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.167069 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2048) took 1067 ms. Trace:
W20260709 13:18:48.268533 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.001s	sys 0.019s
W20260709 13:18:48.269069 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.001s	sys 0.019s
I20260709 13:18:48.268905 10201 rpcz_store.cc:276] 0709 13:18:47.099867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.100334 (+   467us) service_pool.cc:224] Handling call
0709 13:18:47.121646 (+ 21312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:47.136727 (+ 15081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.136769 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:47.165623 (+ 28854us) log.cc:844] Serialized 52 byte log entry
0709 13:18:47.828646 (+663023us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:47.828715 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.828834 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.166652 (+337818us) raft_consensus.cc:1685] finished
0709 13:18:48.166692 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.166962 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17359,"raft.queue_time_us":520,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":15775,"thread_start_us":621868,"threads_started":1,"wal-append.queue_time_us":975636}
W20260709 13:18:48.169592 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 67241.6us
W20260709 13:18:48.175408  9953 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 198) took 39308 ms (client timeout 15000 ms). Trace:
W20260709 13:18:48.176719 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.008s	sys 0.013s
I20260709 13:18:48.048364 15332 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.180286 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:48.273496 10895 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:48.241833 15253 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:48.288791 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.179706 15140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.026s	sys 0.055s
I20260709 13:18:48.059434 15217 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:48.216557 10129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.003s	sys 0.020s
W20260709 13:18:48.191273 10126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.004s	sys 0.017s
W20260709 13:18:48.294003 10129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.003s	sys 0.021s
W20260709 13:18:48.294309 10126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.006s	sys 0.018s
W20260709 13:18:48.224309 10125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.020s	sys 0.004s
W20260709 13:18:48.295171 10125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.021s	sys 0.004s
I20260709 13:18:48.184486 15170 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:48.241120 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2381) took 1932 ms. Trace:
W20260709 13:18:48.245745 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 423709us
I20260709 13:18:48.242679 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.295926 10124 rpcz_store.cc:276] 0709 13:18:46.308766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.310888 (+  2122us) service_pool.cc:224] Handling call
0709 13:18:46.713286 (+402398us) spinlock_profiling.cc:243] Waited 903 us on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfeee 0000000000375939 00007fa27cf531e0 00007fa27d01c0c4 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:46.713483 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.713744 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:47.135466 (+421722us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.240519 (+1105053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.240586 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:48.240650 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.240882 (+   232us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.241033 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":772844,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":72,"raft.queue_time_us":77924,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6481,"spinlock_wait_cycles":2101376,"thread_start_us":1525363,"threads_started":3,"wal-append.queue_time_us":899969}
I20260709 13:18:48.296980 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:48.253474 10056 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:48.256780 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.266067 10052 pending_rounds.cc:85] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 5
I20260709 13:18:48.266850 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2470) took 1002 ms. Trace:
I20260709 13:18:48.240300 14925 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:48.310021 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.179584 15340 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.284668 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.298928 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:48.285624 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.328192 10052 pending_rounds.cc:107] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:48.315212 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.328629 10052 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:48.286967 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.252828 15313 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.002s
W20260709 13:18:48.176617 15176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.010s	sys 0.026s
I20260709 13:18:48.324304 10048 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:48.280985 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.332828 10196 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:18:48.330240 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.331960 10203 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:48.334559 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.333616 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.335563 10123 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:18:48.354779 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.356647 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:48.361465 10131 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:48.362027 10123 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:48.364916 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.353438 14925 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.369015 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.370343 10200 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:48.268266 15215 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:48.373494 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.375183 10203 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:48.326167 15320 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:18:48.356814 15347 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:18:48.390750 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.391290 15348 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.391558 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.390875 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.323916 15217 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.353497 15340 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.393041 10198 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:48.394210 15188 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:48.394850 15188 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.397488 15188 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.398173 15188 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:48.399329 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:48.400785 14925 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.401662 15322 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:48.353477 15313 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.391696 10205 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:18:48.405069 15313 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.391968 15348 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.332860 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.401804 14925 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.384948 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.251219 15163 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.384712 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.409660 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.040s
I20260709 13:18:48.265638 15305 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.298110 15323 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:18:48.416185 10130 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:48.417876 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:48.420061 10203 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
W20260709 13:18:48.422453 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.013s	sys 0.041s
W20260709 13:18:48.423563 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 518843us
I20260709 13:18:48.382702 15332 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.444284 10892 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:48.398769 15188 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.446973 15327 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:48.448172 15226 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.445784 15170 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.460662 15356 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:48.480326 15353 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:48.481205 15188 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.482501 15305 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.483282 15348 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:48.484236 15352 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:18:48.446717 15332 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.486282 15279 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.241497  9949 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.606065 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2138) took 1050 ms. Trace:
I20260709 13:18:48.329630 10054 rpcz_store.cc:276] 0709 13:18:47.264545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.270384 (+  5839us) service_pool.cc:224] Handling call
0709 13:18:47.270760 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.271058 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:47.897040 (+625982us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.266306 (+369266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.266391 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:48.266447 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.266547 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.266756 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4037,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":34,"raft.queue_time_us":1402,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"thread_start_us":994177,"threads_started":2}
I20260709 13:18:48.567286 10055 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:18:48.365839 10124 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:48.660315 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.661636 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2468) took 1398 ms. Trace:
I20260709 13:18:48.570981 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2391) took 1472 ms. Trace:
I20260709 13:18:48.662352 10055 rpcz_store.cc:276] 0709 13:18:47.263725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.268776 (+  5051us) service_pool.cc:224] Handling call
0709 13:18:47.269366 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.650619 (+1381253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.661082 (+ 10463us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400136354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:48.661519 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982720256}
I20260709 13:18:48.575306 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3253) took 1104 ms. Trace:
I20260709 13:18:48.580613 14925 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:48.669127 14925 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:48.510439 15324 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 13:18:48.672583 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3261) took 1175 ms. Trace:
I20260709 13:18:48.672241 15324 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.469878 15327 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.675639 15080 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:48.678305 15348 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:48.678807 15348 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:48.679615 15355 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.680529 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.681322 15367 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.499313 15340 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.503410 15337 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.494899 15329 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.648604 14795 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:18:48.593393 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 195117us
W20260709 13:18:48.610330 15300 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 13:18:48.612725 15238 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.616046 15163 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.619427 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2483) took 1212 ms. Trace:
I20260709 13:18:48.619920 15361 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.694015 15361 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.621078 15226 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.680794 15355 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.698463 15324 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.699016 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.687929 10048 rpcz_store.cc:276] 0709 13:18:47.406950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.407141 (+   191us) service_pool.cc:224] Handling call
0709 13:18:48.618850 (+1211709us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54001b8854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:48.619178 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040858496}
I20260709 13:18:48.700155 10200 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:48.639215 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.473910 15170 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:48.271601  9953 rpcz_store.cc:269] 0709 13:18:08.790735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.791356 (+   621us) service_pool.cc:224] Handling call
0709 13:18:09.193899 (+402543us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:18:48.099062 (+38905163us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:44.848976 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:44.849842 (+   866us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:44.849901 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:18:44.849920 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:44.850736 (+   816us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:44.850846 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:44.850887 (+    41us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:18:44.894204 (+ 43317us) tablet.cc:701] Row locks acquired
0709 13:18:44.894263 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:18:44.894485 (+   222us) write_op.cc:270] Start()
0709 13:18:44.894659 (+   174us) write_op.cc:276] Timestamp: P: 1783603124894408 usec, L: 0
0709 13:18:44.894683 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:18:44.895594 (+   911us) log.cc:844] Serialized 2079 byte log entry
0709 13:18:45.201537 (+305943us) op_driver.cc:464] REPLICATION: finished
0709 13:18:45.780812 (+579275us) write_op.cc:301] APPLY: starting
0709 13:18:45.791707 (+ 10895us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:46.124692 (+332985us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:46.124852 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:46.171355 (+ 46503us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:46.173749 (+  2394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:18:46.173831 (+    82us) write_op.cc:312] APPLY: finished
0709 13:18:46.174248 (+   417us) log.cc:844] Serialized 57 byte log entry
0709 13:18:46.653948 (+479700us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:46.730983 (+ 77035us) write_op.cc:454] Released schema lock
0709 13:18:46.765802 (+ 34819us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:46.829796 (+ 63994us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:46.829835 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6872,"reported_tablets":52,"spinlock_wait_cycles":13478144,"child_traces":[["op",{"apply.queue_time_us":539860,"apply.run_cpu_time_us":63914,"apply.run_wall_time_us":1041469,"cfile_cache_hit":22,"cfile_cache_hit_bytes":24987,"num_ops":5,"prepare.queue_time_us":242643,"prepare.run_cpu_time_us":11153,"prepare.run_wall_time_us":75786,"raft.queue_time_us":230670,"raft.run_cpu_time_us":54144,"raft.run_wall_time_us":257085,"replication_time_us":289661,"spinlock_wait_cycles":313600,"thread_start_us":957740,"threads_started":5,"wal-append.queue_time_us":510127,"wal-append.run_cpu_time_us":43932,"wal-append.run_wall_time_us":2353978}]]}
I20260709 13:18:48.639315 10898 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:48.637538 15357 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.177s	user 0.000s	sys 0.002s
W20260709 13:18:48.571920 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 144978us
I20260709 13:18:48.680737 15348 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.512148 15332 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.690974 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.698257 15226 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.692019 14925 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.668329 10129 rpcz_store.cc:276] 0709 13:18:47.470628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.588674 (+118046us) service_pool.cc:224] Handling call
0709 13:18:47.588984 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.589379 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.589420 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.629663 (+ 40243us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.294490 (+664827us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:48.294539 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.294611 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.574595 (+279984us) raft_consensus.cc:1685] finished
0709 13:18:48.574637 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.575013 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":25891,"thread_start_us":603286,"threads_started":1,"wal-append.queue_time_us":706441}
I20260709 13:18:48.696740 15327 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.696355 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:48.662879 10126 rpcz_store.cc:276] 0709 13:18:47.098482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.098677 (+   195us) service_pool.cc:224] Handling call
0709 13:18:47.112272 (+ 13595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.112716 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:47.354704 (+241988us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.294678 (+939974us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.357525 (+ 62847us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.357688 (+   163us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:48.357736 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.357814 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.570222 (+212408us) raft_consensus.cc:1685] finished
0709 13:18:48.570293 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.570686 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":393357,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4861,"raft.queue_time_us":108427,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":16812,"thread_start_us":1106926,"threads_started":2,"wal-append.queue_time_us":1090653}
I20260709 13:18:48.696514 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2390) took 1604 ms. Trace:
I20260709 13:18:48.682420 15367 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.649602 10056 rpcz_store.cc:276] 0709 13:18:47.555075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.555237 (+   162us) service_pool.cc:224] Handling call
0709 13:18:48.605438 (+1050201us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b5400087354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:48.605743 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405441408}
I20260709 13:18:48.764122 15348 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:48.762873 10127 rpcz_store.cc:276] 0709 13:18:47.091764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.092005 (+   241us) service_pool.cc:224] Handling call
0709 13:18:47.095928 (+  3923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.323596 (+227668us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:48.100173 (+776577us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.694682 (+594509us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.694852 (+   170us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:48.694991 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.695875 (+   884us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:48.695940 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.696263 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":604,"prepare.queue_time_us":263965,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":87089,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":4540,"spinlock_wait_cycles":522998144,"thread_start_us":1368280,"threads_started":3,"wal-append.queue_time_us":352713}
I20260709 13:18:48.765026 15367 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.769738 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 579) took 1194 ms. Trace:
I20260709 13:18:48.772104 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:48.770359 10122 rpcz_store.cc:276] 0709 13:18:47.574947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.648772 (+ 73825us) service_pool.cc:224] Handling call
0709 13:18:47.669729 (+ 20957us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.670248 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.670293 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.690778 (+ 20485us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.248156 (+557378us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:18:48.325333 (+ 77177us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.325604 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.325705 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.636101 (+310396us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:48.769038 (+132937us) raft_consensus.cc:1685] finished
0709 13:18:48.769241 (+   203us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.769514 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20811,"prepare.queue_time_us":83554,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"raft.queue_time_us":104713,"raft.run_cpu_time_us":483,"raft.run_wall_time_us":47305,"spinlock_wait_cycles":4715264,"thread_start_us":461429,"threads_started":1,"wal-append.queue_time_us":657615}
I20260709 13:18:48.696899  9949 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.776108 15199 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:48.776736 15199 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.731956 15361 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.719725 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.673285 10125 rpcz_store.cc:276] 0709 13:18:47.497348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.599938 (+102590us) service_pool.cc:224] Handling call
0709 13:18:47.600374 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.600803 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.600831 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.602740 (+  1909us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.295574 (+692834us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:48.373598 (+ 78024us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.671890 (+298292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.672066 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.672138 (+    72us) raft_consensus.cc:1685] finished
0709 13:18:48.672156 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.672392 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36544,"prepare.queue_time_us":110919,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":80734,"raft.run_cpu_time_us":396,"raft.run_wall_time_us":20506,"thread_start_us":929086,"threads_started":2,"wal-append.queue_time_us":736467}
W20260709 13:18:48.743454 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.684968 14795 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:48.723394 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.777215 15199 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.776347  9949 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:48.758546 10051 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:48.756939 15355 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.730957 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:48.793100 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53337.9us
W20260709 13:18:48.793929 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63443.1us
I20260709 13:18:48.762581 15226 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:48.739295 15364 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.174s	user 0.006s	sys 0.000s
I20260709 13:18:48.809619 15361 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.810626 15361 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.794989 15346 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:48.820371 15324 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.820600 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:48.821345 15324 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.823892 15361 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.824138 15323 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:48.824885 15361 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.826081 15361 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.826784 15305 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.828127 15305 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.832330 15305 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.835603 15226 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.837023 15226 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.838018 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.838505 15324 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.839468 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:18:48.840247 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.797921 14795 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:48.840265 10892 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148"
dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:34750
I20260709 13:18:48.843412 15324 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.844000 10892 raft_consensus.cc:606] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:48.844646 10892 raft_consensus.cc:614] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:48.844419 15324 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.848544 15361 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.850155 15324 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.852228 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:48.844379 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.862035 10898 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:48.765668 15215 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.866714 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:48.884526 15226 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.887544 15355 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.887702 15367 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.888616 15355 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.889253 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:48.888909 15367 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.890332 15152 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.891215 15152 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.891990 15367 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:48.890130 10125 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:48.870160 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:48.899582 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:48.901499 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.913755 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45341.7us
I20260709 13:18:48.911695 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.915094 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.916635 10053 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:48.916972 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:48.922034 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62945.7us
I20260709 13:18:48.926908 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:48.932206 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:48.933248 10127 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:48.933959 15226 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.934823 15226 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:48.941164 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.942340 10122 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:48.944669 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:48.953531 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 597879us
I20260709 13:18:48.965179 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:48.972232 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:48.979241 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:48.976346 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:48.983959 10126 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:48.980791 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:48.991609 15324 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.991922 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:48.992478 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:48.992851 15361 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:48.992529 15324 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.994738 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:48.995918 15305 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:48.993764 15361 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.002550 14795 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:49.003391 14795 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:49.003860 14795 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:49.007350 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.007601 15152 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.008507 10122 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:48.996804 15324 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:48.997918 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:48.998590 15376 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:18:49.011832 15324 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:49.013031 15230 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.014s
I20260709 13:18:49.013898 15230 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:48.968573 15358 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.018246 15358 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:49.020640 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72041us
I20260709 13:18:49.021775 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.024325 15080 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:49.024948 15080 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:49.026422 15080 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.031883 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.022781 15188 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:48.985889 15346 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:49.044529 15320 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.637s	user 0.025s	sys 0.000s
I20260709 13:18:49.047514 15215 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:49.053478 15226 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:49.060039 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58847.4us
I20260709 13:18:48.991030 10125 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:49.070741 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:49.075011 15358 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.072966 15199 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:49.070725 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73085us
I20260709 13:18:49.074965 15361 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.076030 10122 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:18:49.076725 15355 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:49.077442 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 122343us
I20260709 13:18:49.086412 15188 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.086546 10201 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:49.087150 15346 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:49.097004 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65375.1us
I20260709 13:18:49.100971 10127 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:18:49.105629 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.102478 15346 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:49.107873 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.109010 15358 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:49.109817 15355 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:49.110488 15355 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:49.111747 15355 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.096203 15207 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:49.119935 10895 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:18:49.120908 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.099795 15188 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.119966 10048 raft_consensus.cc:2368] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:49.132623 15358 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.133769 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.134327 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:49.134698 10122 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:49.133579 15358 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.140852 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.145893 15374 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:18:49.146689 15358 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.149240 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.150099 15207 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.129926 15230 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.122710 15201 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:49.161398 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.161806 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.167984 10129 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:49.162300 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.170532 10124 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:49.172413 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:49.173110 15080 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:49.173455 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.173601 15080 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:49.174082 15080 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:49.174456 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66788.1us
I20260709 13:18:49.175102 10125 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:49.175578 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.176486 15358 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.177698 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:49.179050 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.180073 15170 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:49.188162 15170 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:49.188803 15170 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:49.191470 15358 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:49.192198 15358 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:49.193434 15358 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.188283 10122 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:49.197033 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.199066 10199 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:49.164661 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.165066 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.165405 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.150796 14964 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.150858 15207 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.164245 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.156708 15201 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:49.162513 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.164006 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.150458 14795 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.202493 15170 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.215561 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.229156 10131 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:49.244972 10126 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:49.247099 10127 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:49.250998 14925 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:49.250366 14964 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:49.251250 15215 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:49.251814 14925 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:49.252337 14925 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.253651 15207 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:49.247215 15367 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:49.255970 15201 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.247557 10130 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:49.233104 15346 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.255934 15170 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:49.265298 14964 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:49.282481 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:49.297442 10200 raft_consensus.cc:2368] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11.
I20260709 13:18:49.298924 15346 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:49.305431 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.308243 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111412us
W20260709 13:18:49.312181 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 175904us
I20260709 13:18:49.313740 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.318833 10895 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:18:49.325773 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 203463us
W20260709 13:18:49.335791 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120368us
W20260709 13:18:49.336264 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 172169us
I20260709 13:18:49.329649 15374 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.328325 15199 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.348400 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:49.336062 15215 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:49.359457 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.353718 15348 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:49.349658 15374 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.365675 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:49.378140 10895 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:49.380336 15188 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.382009 15348 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:49.384878 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.364240 14795 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:49.388725 15355 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.401014 10895 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:18:49.392176 15188 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:49.407212 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:49.383299 15348 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.403355 15215 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.409104 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.396701 15367 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:18:49.416282 14795 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:18:49.428337 14795 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:49.405131 15383 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:18:49.429549 14795 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:49.431644 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76635.6us
I20260709 13:18:49.430976 15367 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:49.429044 15374 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:49.412838 14925 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:49.467082 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74059.2us
I20260709 13:18:49.426819 15080 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.472749 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.474947 10130 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:49.478169 15080 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:49.478437 15374 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.479391 15374 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.483464 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.448813 15367 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:49.485016 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:49.486954 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89265us
I20260709 13:18:49.492429 15199 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:49.493036 15389 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 13:18:49.488107 15080 raft_consensus.cc:2764] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:18:49.505962 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.513377 10130 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:49.518992 15367 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.521167 15215 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.522209 15215 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.526535 15215 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:49.527621 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55957.1us
I20260709 13:18:49.529295 10170 leader_election.cc:400] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:49.529835 10170 leader_election.cc:403] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:49.530804 15080 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.532225 14925 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:49.536001 15375 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.537181 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.536762 15375 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.539196 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.539492 14925 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.540727 14925 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.541939 15374 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.543686 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.545573 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.549463 10131 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:49.550982 15215 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.551643 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.552628 10122 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:49.550973 10128 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:49.556694 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.557874 15199 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:49.558327 15199 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.558722 15199 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.559437 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.561264 10128 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:49.564018 15388 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:18:49.568639 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.569797 15226 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:49.570253 15226 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.570693 15226 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:49.571576 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.574580 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:49.543727 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 99421.2us
I20260709 13:18:49.551982 15215 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.577746 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.579310 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.589879 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.592785 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.576439 15369 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.599789 15369 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.602290 15369 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:49.612053 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.613232 10124 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:49.613658 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.614475 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.614532 10131 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:49.602735 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.618100 10129 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:49.619390 15215 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:49.589386 15387 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.085s	user 0.001s	sys 0.001s
I20260709 13:18:49.622669 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.623545 10123 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.633744 15226 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.635692 15375 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.636384 15226 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.639999 10123 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:49.640631 10129 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:49.596824 15361 raft_consensus.cc:2764] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:49.617093 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.658051 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.673198 15361 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:49.673308 15358 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:49.674788 15358 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:49.616559 15391 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.671861 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95050us
W20260709 13:18:49.682346 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79731.2us
I20260709 13:18:49.667629 15383 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:49.676362 15226 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:49.684496 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.693861 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:49.693789 15215 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:49.695668 15215 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.696105 15215 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.699751 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.709663 15080 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12.
I20260709 13:18:49.668998 15199 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.686519 15383 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:49.738503 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 158351us
I20260709 13:18:49.686324 15361 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:18:49.683279 15391 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:49.748126 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:49.775411 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.775480 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:49.782722 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 91926.9us
I20260709 13:18:49.786144 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.787467 10128 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
W20260709 13:18:49.783484 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90681.5us
I20260709 13:18:49.791723 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:49.796444 10051 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:49.806856 10124 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:49.819777 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.811674 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:49.782671 15405 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:49.835686 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:49.835513 15383 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.836683 10054 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.821339 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:49.823693 15384 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:18:49.837349 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.810837 15215 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.831352 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:49.832233 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45489.7us
W20260709 13:18:49.852085 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 262317us
I20260709 13:18:49.853411 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.854161 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:49.845641 15367 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:49.865347 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51618.7us
I20260709 13:18:49.856837 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.860963 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:49.860728 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:49.866519 10052 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:49.867951 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:49.868429 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:49.868831 10052 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:49.870060 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:49.832249 15199 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.871660 15199 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:49.872092 15199 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.872480 15199 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:49.872958 10200 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:49.874454 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:49.875412 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:49.876978 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.877269 10047 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:18:49.847774 15405 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:49.878600 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:49.882045 10124 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.879434 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:49.862514 10050 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:49.844411 15226 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:49.866680 10051 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:49.862221 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.884963 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:49.889885 15409 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:49.856992 15355 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:49.863338 15383 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:49.877148 15391 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:49.863785 15367 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.891008 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:49.896327 14925 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:49.914098 15409 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.884534 15358 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:49.884281 15405 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:49.922333 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106663us
I20260709 13:18:49.928046 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.891727 15403 raft_consensus.cc:2764] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:18:49.926714 15367 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:49.944047 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:49.949026 10049 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 12 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 13:18:49.944001 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:49.954953 15408 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.088s	user 0.003s	sys 0.000s
I20260709 13:18:49.930732 14925 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.921778 15404 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:49.920114 15226 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:49.938238 15405 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.951531 15361 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:49.965343 14925 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.963550 15215 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:49.964542 15404 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:49.967931 15358 raft_consensus.cc:2764] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:18:49.936131 15409 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.005069 15409 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:50.015973 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84161.5us
I20260709 13:18:49.999301 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:50.002468 10056 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:50.040602 10054 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:50.044823 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:49.979035 15361 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:50.048381 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104436us
I20260709 13:18:50.060560 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:50.048105 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 147099us
I20260709 13:18:50.048924 15417 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:50.051445 10056 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:50.055835 10203 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:18:49.970881 15188 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:18:50.064283 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111364us
I20260709 13:18:50.060671 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.068475 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:50.074697 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:50.000054 15404 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:50.079396 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.080432 10894 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:50.081725 15188 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:50.044260 15409 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:50.063082 15361 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:49.999300 15391 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:18:50.100178 10894 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:50.103726 15397 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.104427 15397 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:50.114576 15412 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:50.063846 15417 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:50.115764 15417 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:50.117267 15201 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:18:50.135895 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52865us
W20260709 13:18:50.138367 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89001.2us
I20260709 13:18:50.082261 15188 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:50.148283 10051 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
W20260709 13:18:50.150898 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55880.3us
I20260709 13:18:50.148907 10124 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:50.146139 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:50.153492 15406 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:50.154129 15406 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:50.154681 15406 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:50.155050 15409 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:50.161725 10894 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:50.160666 10052 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:50.177929 15361 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.184933 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:50.185369 10899 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:50.207870 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:50.215263 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:50.214001 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87da05694091470284f66a8181207e2f"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:50.216982 10200 raft_consensus.cc:606] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:50.191984 15080 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:50.162290 14925 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.223541 15080 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.229931 15080 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:50.216732 15367 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.239677 14964 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:50.241711 15188 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.243487 15188 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:50.223483 15201 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:50.206892 15412 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:18:50.248934 15397 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:50.276955 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.277954 10899 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:18:50.278414 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 214268us
I20260709 13:18:50.225735 15226 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:50.269410 14964 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:50.292374 10056 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:50.277916 15367 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.310496 10052 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:50.268181 15383 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:18:50.259665 15404 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:50.407555 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 114261us
I20260709 13:18:50.300657 15201 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:50.411852 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 446635us
I20260709 13:18:50.408215 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:50.425359 15199 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.552s	user 0.015s	sys 0.004s
I20260709 13:18:50.421744 15361 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:50.386826 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 319042us
W20260709 13:18:50.425554 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 158968us
I20260709 13:18:50.285691 15226 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:50.442209 15201 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.456025 15361 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:50.456784 15361 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:50.459000 15361 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:50.413602 14964 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.426347 15409 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:50.483596 10053 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:50.486340 15080 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.489177 15080 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:50.495400 15383 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:50.496415 15383 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:50.500042 14925 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.505395 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:18:50.515590 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 107595us
I20260709 13:18:50.521903 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:50.477046 15226 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:50.530711 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:50.544075 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 89979.4us
I20260709 13:18:50.526652 15367 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:50.508488 15425 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:50.485016 15199 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:50.558956 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 247405us
I20260709 13:18:50.549413 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:50.565781 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58554.8us
I20260709 13:18:50.570055 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:50.571703 10020 leader_election.cc:400] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11.
I20260709 13:18:50.572410 10020 leader_election.cc:403] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:18:50.571625 15425 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:50.577632 10125 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:50.526798 15409 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:50.515499 15405 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.542027 15427 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:50.591977 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.591539 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:50.599922 10048 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:50.568468 15383 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:50.559280 15426 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:50.610217 10130 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:18:50.611763 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:50.612794 10127 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:50.542501 15080 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:18:50.576768 15425 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:50.587028 15403 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:50.604564 10892 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:50.600759 15409 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.625638 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:50.636150 10050 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:50.627564 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:50.643528 10127 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:50.636616 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:50.628262 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56758.9us
I20260709 13:18:50.604218 15427 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.658929 10123 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:18:50.588752 15417 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.695240 10894 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:18:50.683703 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92267.1us
I20260709 13:18:50.691851 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:50.701146 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.638276 15425 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:50.636059 15405 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.715338 10055 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:18:50.628034 15404 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.626996 15409 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.644353 15403 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:50.634244 15199 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.634439 14925 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:50.740202 10127 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:50.754711 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:50.757697 10054 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:50.657610 15383 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.758472 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52533.1us
I20260709 13:18:50.738462 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:50.695290 15406 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.540s	user 0.014s	sys 0.001s
W20260709 13:18:50.759954 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 458529us
I20260709 13:18:50.664932 15427 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:50.672935 15432 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 10. Current term is 11.
I20260709 13:18:50.777045 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:50.780565 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:50.786262 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.803987 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.805286 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:50.809926 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:50.818223 10894 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:50.814723 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.787406 14925 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:50.822613 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:50.823589 10198 raft_consensus.cc:2368] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:50.793481 15426 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:50.824888 15426 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:50.827766 15444 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:50.828435 15444 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:50.828845 15444 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:50.833055 10123 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:50.842340 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:50.844623 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60746.4us
I20260709 13:18:50.790122 15409 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:18:50.850947 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.805337 15438 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.780293 15406 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.851612 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:50.862462 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63773.1us
I20260709 13:18:50.863557 14925 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.866598 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.867746 10055 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:18:50.853365 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 148448us
I20260709 13:18:50.833097 15230 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:50.836545 15358 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:50.880507 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.846915 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:50.858693 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.875754 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.868233 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.883038 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.874840 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.806780 15383 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.889580 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2517) took 1063 ms. Trace:
I20260709 13:18:50.891790 10053 rpcz_store.cc:276] 0709 13:18:49.826081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.941321 (+115240us) service_pool.cc:224] Handling call
0709 13:18:49.966734 (+ 25413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.484353 (+517619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.889270 (+404917us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54000d9b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:50.889502 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187880704}
I20260709 13:18:50.893357 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:50.923557 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.924602 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.925813 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.930233 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.932020 10051 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15. Ops: []
W20260709 13:18:50.944674 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51426.1us
I20260709 13:18:50.946466 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.886468 15445 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:50.956091 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.957480 10056 raft_consensus.cc:2410] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:18:50.955261 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:50.922791 15432 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:50.920019 15426 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:50.967407 10892 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:50.968423 15437 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:18:50.971539 15417 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:50.927323 15215 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.879083 15226 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.922602 15080 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:18:50.974045 15440 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:50.981410 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.976487 10047 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:50.986214 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:50.991711 10894 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:50.930878 15425 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:50.984509 15404 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:50.858522 15403 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:50.953496 15421 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.002141 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:50.884824 15438 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.042910 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2525) took 1152 ms. Trace:
I20260709 13:18:51.052098 10048 rpcz_store.cc:276] 0709 13:18:49.890417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.013443 (+123026us) service_pool.cc:224] Handling call
0709 13:18:50.013752 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.610244 (+596492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.042391 (+432147us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b5400135e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:51.042778 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347314432}
W20260709 13:18:51.056533 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 136217us
I20260709 13:18:51.056814 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:51.058894 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113062us
I20260709 13:18:50.996569 15440 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:18:51.047832 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56367.6us
I20260709 13:18:51.052630 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:51.069638 10047 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:18:51.076835 15080 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:51.078428 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:50.999830 15215 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.086460 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:51.087458 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.088346 10047 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:51.089321 15404 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:51.102210 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.063922 15440 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:51.114322 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45991.4us
I20260709 13:18:51.064481 15450 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:51.119038 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59207.6us
I20260709 13:18:51.077441 15080 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.036660 15425 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.123759 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.125181 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.126224 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.054162 15405 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:18:51.029808 15432 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.129938 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:51.131556 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.083571 15439 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:51.148106 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.149730 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.133864 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2515) took 1308 ms. Trace:
I20260709 13:18:51.014950 15421 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:51.154666 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.613s (SENT) [suppressed 15 similar messages]
I20260709 13:18:51.151286 10050 rpcz_store.cc:276] 0709 13:18:49.825486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.888530 (+ 63044us) service_pool.cc:224] Handling call
0709 13:18:49.899323 (+ 10793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.652066 (+752743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.133595 (+481529us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:51.133787 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1661720960}
I20260709 13:18:51.070233 15445 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:18:51.156641 15441 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
W20260709 13:18:51.228410 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66389.9us
I20260709 13:18:51.157487 15406 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.174839 15450 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.290848 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:51.295192 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.192960 15188 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:18:51.324600 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.239799 15080 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.289716 15409 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:51.348809 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 194474us
I20260709 13:18:51.335806 10127 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:51.350181 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:51.231833 15383 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:51.331873 15215 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.357788 15459 raft_consensus.cc:476] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Not starting pre-election -- already a leader
I20260709 13:18:51.358640 15459 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.359223 15459 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.359505 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.360247 15384 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:51.361204 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.361917 10126 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.360430 10125 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:51.357892 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.316902 15226 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.363869 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.312340 15445 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:51.371181 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.190840 15439 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.361447 15384 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:51.378084 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.335033 15460 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.385715 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.336530 15458 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.389784 10125 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.389662 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.387414 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:51.394384 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 260650us
I20260709 13:18:51.394914 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.410460 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:51.411760 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.372071 15445 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.368966 15404 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:51.412006 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:51.409586 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.421895 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.427630 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.386382 15384 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.434976 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.377328 15409 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.409479 15226 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.425951 15080 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:51.452371 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102277us
I20260709 13:18:51.452378 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.456934 15448 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:51.457973 15456 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:51.430260 15437 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:18:51.467610 10127 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:51.463343 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:51.460465 15425 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:51.488186 10126 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:51.411584 15458 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.494375 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.560019 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.492666 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.561192 10131 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.552260 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.398224 15460 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.465159 15448 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:51.466835 15464 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:51.475675 15457 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:18:51.560360 15358 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.496469 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:51.480007 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52003.6us
W20260709 13:18:51.527817 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.485853 10125 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:51.572167 10130 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.572784 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.476820 15409 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:51.477059 15439 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:51.581811 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162338us
I20260709 13:18:51.495085 15432 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.466009 15469 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:51.559317 15456 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.568538 15448 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:51.586199 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.468214 15450 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:51.587809 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91456.5us
I20260709 13:18:51.589478 15471 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.589576 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.592053 15472 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:51.592327 15361 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.595345 15438 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.596459 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2456) took 1056 ms. Trace:
I20260709 13:18:51.597147 15428 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:18:51.581249 15425 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.475374 15348 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 13:18:51.475878 15384 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.641116 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:51.611901 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:51.572506 15421 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.632493 10129 rpcz_store.cc:276] 0709 13:18:50.539665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.548120 (+  8455us) service_pool.cc:224] Handling call
0709 13:18:50.585706 (+ 37586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.640889 (+ 55183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.640928 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:50.653451 (+ 12523us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:50.653505 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.596177 (+942672us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b54002ed1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:51.596225 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.596391 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126254336}
I20260709 13:18:51.484181 15226 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.647176 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.625039 15409 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:51.622135 15427 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.828s	user 0.010s	sys 0.005s
I20260709 13:18:51.643379 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:51.648468 10865 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:51.651849 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.656360 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.582964 15358 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.659929 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:51.662781 15427 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.669173 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.670058 10202 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:51.586337 15464 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.681104 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.686964 10199 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: []
W20260709 13:18:51.689491 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46862.6us
I20260709 13:18:51.653113 15369 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:51.692940 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.666265 15472 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:51.670835 15427 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:51.671744 15460 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:51.696543 15444 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.867s	user 0.018s	sys 0.000s
I20260709 13:18:51.726878 10125 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:51.601814 15404 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.679412 15478 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.685477 15421 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.715205 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.647917 15425 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.694936 15384 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.663254 15361 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:51.789885 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64371.6us
I20260709 13:18:51.714439 15468 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:18:51.715612 15437 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:18:51.775504 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.776233 10896 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:51.731945 15369 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.691262 15439 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.833334 15439 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.835983 15439 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.675295 15416 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:18:51.836519 15437 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:51.837018 15437 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.837350 15416 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:51.837777 15416 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.800571 15421 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.838047 10130 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:51.722081 15432 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.840993 15421 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.664158 15409 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.841257 15432 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.843427 15409 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.845170 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.846702 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.801367 10201 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:51.648783 15471 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.802977 10131 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:51.737628 15444 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.811430 15478 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:51.817988 15440 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.693s	user 0.007s	sys 0.005s
I20260709 13:18:51.863430 15440 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.818815 15403 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.819754 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.820166 10202 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:51.820694 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:51.821084 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.821755 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:51.818017 15226 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.824261 15425 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.874297 15440 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:51.824539 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.717741 15456 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.875077 15478 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.827518 15481 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:51.878460 15481 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.881283 15481 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:51.829159 15458 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.831281 15361 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.832012 15215 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.882266 15445 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.882519 15215 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.884080 15445 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.885188 15215 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.888705 15475 raft_consensus.cc:993] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:51.889359 15475 raft_consensus.cc:1081] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:51.892094 15444 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:51.893211 10200 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:18:51.833365 15369 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.897019 15358 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.898139 15432 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.899818 15188 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.900449 15188 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.901569 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 599) took 1207 ms. Trace:
I20260709 13:18:51.902045 10892 rpcz_store.cc:276] 0709 13:18:50.694229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.695442 (+  1213us) service_pool.cc:224] Handling call
0709 13:18:50.967170 (+271728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.977761 (+ 10591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.901305 (+923544us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540063fbd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:51.901460 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623873792}
I20260709 13:18:51.902822 15188 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:51.904011 15443 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:51.905614 15443 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:51.906065 15443 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:51.838766 15416 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.907021 15443 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:51.913228 10198 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:18:51.914595 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113129us
W20260709 13:18:51.916939 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 185263us
I20260709 13:18:51.908298 15416 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:51.924350 15432 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:51.926595 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:51.801399 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68622.6us
I20260709 13:18:51.850322 15383 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:18:51.857141 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.935135 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:51.818228 15426 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:51.864018 15199 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:51.869412 10196 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:18:51.868788 15403 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:51.838058 15437 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:51.929220 10865 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.743s (SENT)
I20260709 13:18:51.908260 15443 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:51.940910 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.941874 10020 leader_election.cc:400] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
I20260709 13:18:51.957743 10020 leader_election.cc:403] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:18:51.960170 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44805.8us
W20260709 13:18:51.964247 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46218.3us
I20260709 13:18:51.966710 10201 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:51.980310 10051 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:51.982129 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:51.991673 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:51.994129 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:51.940920 15199 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:51.996709 10051 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:51.935775 15471 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:51.952261 15215 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:18:52.001490 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:51.940809 15426 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:52.080951 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 278361us
W20260709 13:18:52.007330 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.018574 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:52.017760 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.033622 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:52.010715 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.021302 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.027292 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.007330 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:52.003079 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.004180 10048 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:52.085793 10050 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:52.003312 10131 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:52.007936 10124 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:52.019975 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.080883 10051 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:52.022205 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.032779 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:52.086289 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:18:52.081677 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87621.9us
W20260709 13:18:52.085521 10863 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.743s (SENT)
I20260709 13:18:52.088125 10122 raft_consensus.cc:2376] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12. Re-sending same reply.
I20260709 13:18:52.093344 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.084229 10128 raft_consensus.cc:2376] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14. Re-sending same reply.
I20260709 13:18:52.096323 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2532) took 1805 ms. Trace:
I20260709 13:18:52.096786 10054 rpcz_store.cc:276] 0709 13:18:50.290736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.290881 (+   145us) service_pool.cc:224] Handling call
0709 13:18:50.306690 (+ 15809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.770362 (+463672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.096052 (+1325690us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b5400199954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:52.096253 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036682624}
I20260709 13:18:52.095225 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.105937 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.090325 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.095857 10049 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:52.002308 15432 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.009579 15456 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:52.027640 15437 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:52.113272 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:52.118120 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.102067 14795 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:52.183298 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65079.4us
I20260709 13:18:52.184719 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.022729 15201 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:18:52.131978 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:52.190878 10892 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.138912 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.139935 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.142174 15403 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:52.091138 14925 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:52.091572 15405 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:52.151338 15473 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.163475 15207 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:18:52.164899 15448 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.003s
I20260709 13:18:52.016494 15429 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:52.178469 10123 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.176513 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.228356 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.229802 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.179667 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.117419 15460 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.115237 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.015898 15464 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:52.188119 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.235677 15460 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.237322 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.238088 10203 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:52.239044 15460 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:52.239459 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.117734 15425 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.192919 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:52.240780 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61276.1us
I20260709 13:18:52.240589 15425 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.241838 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.242933 15461 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.243870 15425 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.129703 15367 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:52.245266 15367 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.248575 15367 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.249486 15367 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.130977 15478 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.190878 10020 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:52.195186 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.251268 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60480.5us
I20260709 13:18:52.252101 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.252791 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.253903 10205 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:52.130123 15458 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:52.255738 15458 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.257681 10123 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:52.258105 15458 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.134085 15450 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.259137 15450 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.262203 15450 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.262542 15461 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.186501 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:52.224112 15447 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:18:52.118417 15471 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:52.285615 15471 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.286285 15471 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.289126 15471 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.291790 15369 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:52.292261 15199 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.292577 15369 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.293900 15199 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:52.193665 15215 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:52.295897 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 210398us
I20260709 13:18:52.193888 15417 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.296257 15369 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:52.296612 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.304518 10892 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:52.304724 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.305414 10054 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:52.194410 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:52.311442 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.313804 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.319963 15426 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.263372 15478 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:52.321920 15478 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.322521 15478 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:52.193477 14795 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.325155 10053 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:52.325870 14795 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:52.264882 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:52.327414 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62645.7us
I20260709 13:18:52.266229 15425 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:52.331300 15425 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.332464 15425 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:52.251307 15429 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:52.276125 15464 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:52.336313 15464 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.276988 15226 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.339564 15226 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.172667 15456 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:52.211519 14925 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.344085 15456 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.345286 14925 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.278801 15448 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.346005 15456 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.348026 15448 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:52.348124 10203 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:52.249297 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.349677 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.204545 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.352093 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102993us
I20260709 13:18:52.353104 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.356730 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.364768 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.365797 10131 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:52.357497 10899 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:52.358665 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.371412 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.374080 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.342729 15226 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.386279 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.385738 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:52.384253 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.334611 15425 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:18:52.398191 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:52.362531 14925 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.357204 15201 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:52.405193 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:52.377897 15461 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.359833 10123 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87da05694091470284f66a8181207e2f"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:18:52.378422 15358 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.382369 15450 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.392843 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:52.408754 10123 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:52.411284 10123 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:52.384159 15458 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:52.422329 10050 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:52.425354 10204 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:52.425987 10198 raft_consensus.cc:606] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.426640 10198 raft_consensus.cc:614] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:52.432636 15475 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:52.432983 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.433501 15475 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.434141 15475 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:52.435906 15444 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.438046 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.442315 15465 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:18:52.454262 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72821us
I20260709 13:18:52.395925 15226 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:52.456547 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.627s (SENT)
I20260709 13:18:52.382123 15367 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:18:52.474488 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.473065 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.480548 10201 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:52.401567 15416 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:52.405017 14964 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:18:52.406101 15447 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:52.416230 15464 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.485211 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:52.489557 10863 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.694s (SENT)
I20260709 13:18:52.481884 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:52.506683 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.415501 15322 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:18:52.507385 10053 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:52.390424 15188 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:18:52.451390 15425 consensus_queue.cc:1243] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:18:52.478721 15461 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:52.516415 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.517757 10050 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:52.418452 15201 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.559489 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.560572 10050 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
W20260709 13:18:52.562078 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80961.2us
I20260709 13:18:52.563482 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.550370 15448 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:52.564881 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.484336 15470 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
W20260709 13:18:52.557214 15443 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.591s	user 0.013s	sys 0.000s
W20260709 13:18:52.566943 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.506290 15226 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.566601 10123 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.575443 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:52.580024 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.586067 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.562032 15482 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.587074 10050 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:52.592442 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:52.582383 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 162370us
I20260709 13:18:52.587667 10199 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:52.589840 10123 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:52.567209 15425 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:52.556926 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164175us
I20260709 13:18:52.599213 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.603049 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 121454us
I20260709 13:18:52.603680 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.605490 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.609836 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.581730 15322 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:52.619171 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.586730 15439 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.607076 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.623585 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48244us
I20260709 13:18:52.599213 15438 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:18:52.612054 15506 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:18:52.626184 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136688us
I20260709 13:18:52.627049 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:52.612780 15495 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 13:18:52.699956 15504 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:18:52.700493 15505 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:52.628157 15444 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:52.701085 15444 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.609447 15417 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.753293 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.754055 10050 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
W20260709 13:18:52.698314 15437 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.509s	user 0.000s	sys 0.009s
I20260709 13:18:52.698109 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.699339 15494 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.597s	user 0.005s	sys 0.000s
W20260709 13:18:52.613559 15501 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:52.609346 15416 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:52.708576 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.528s (SENT)
W20260709 13:18:52.774614 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66139.6us
W20260709 13:18:52.775357 10863 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.658s (SENT)
I20260709 13:18:52.775756 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.751821 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.622354 15322 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:18:52.773689 15507 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:18:52.701588 15444 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:52.761070 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 136226us
I20260709 13:18:52.776969 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:52.756677 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129705us
I20260709 13:18:52.801637 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.802400 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.785555 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.685371 15426 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:52.793031 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:52.794073 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.622840 15418 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.339s
W20260709 13:18:52.819768 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44445.2us
I20260709 13:18:52.820164 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:52.820964 10863 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.924s (SENT)
I20260709 13:18:52.821543 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:52.821976 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:52.798178 10130 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:18:52.811046 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:52.824321 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:52.800578 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:52.813290 15416 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:52.813603 15322 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:52.814399 15499 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:18:52.815765 15510 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:18:52.816208 15438 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.818065 15508 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:52.864725 15508 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.865500 15508 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.826138 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:52.862599 15322 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:52.869954 15384 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:52.819732 15426 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:52.871183 15426 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:52.829501 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:52.875346 15502 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
I20260709 13:18:52.851820 15188 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:52.878593 15188 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:52.846477 15471 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:18:52.891081 10197 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:52.861356 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:52.862489 15430 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:18:52.862296 15438 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:52.875185 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47595.9us
I20260709 13:18:52.847425 15417 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:52.846767 15432 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:52.879920 15201 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:52.848093 10123 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:18:52.882782 10050 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:52.857481 15448 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:52.854162 10863 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.101s (SENT)
I20260709 13:18:52.902212 10053 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:53.000631 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:53.002681 10050 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:52.995298 15188 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.001538 15471 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:53.021193 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:52.901381 15475 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:52.999873 15432 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:53.023864 15512 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:53.024749 15515 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:18:53.023824 15517 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:53.037730 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 183608us
W20260709 13:18:53.040221 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 138915us
W20260709 13:18:53.049052 15199 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.755s	user 0.008s	sys 0.006s
W20260709 13:18:53.037639 15478 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.707s	user 0.006s	sys 0.006s
I20260709 13:18:53.060767 15226 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.076759 10053 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:53.052124 15417 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:53.090408 15461 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.008s
I20260709 13:18:53.071398 15478 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:53.110365 10863 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.981s (SENT)
I20260709 13:18:53.110770 15516 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:53.111274 15471 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:53.123935 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3255) took 1500 ms. Trace:
I20260709 13:18:53.124611 10200 rpcz_store.cc:276] 0709 13:18:51.622548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.622848 (+   300us) service_pool.cc:224] Handling call
0709 13:18:51.623433 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.893713 (+270280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.122257 (+1228544us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:53.122626 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619781504}
I20260709 13:18:53.123983 15516 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:53.127545 15516 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.125026 15471 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.113658 15461 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:53.156370 10055 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:18:53.157020 10055 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:53.160717 10095 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.034s	sys 0.051s
W20260709 13:18:53.165704 10095 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.034s	sys 0.052s
W20260709 13:18:53.166218 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.17105e+06us
W20260709 13:18:53.158964 15473 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.003s
I20260709 13:18:53.110867 15432 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.175967 10055 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.170369 15417 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:53.191148 15456 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.844s	user 0.011s	sys 0.006s
I20260709 13:18:53.192891 15481 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:53.193545 15481 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:53.196480 15459 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.197577 15475 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.200263 15478 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.202946 14925 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:18:53.203585 14925 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:53.204509 14925 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.206264 14925 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:53.207857 15438 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.175901 15473 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13.
W20260709 13:18:53.213801 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 611413us
I20260709 13:18:53.215277 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.216203 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2073) took 1395 ms. Trace:
I20260709 13:18:53.216325 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.216836 10204 rpcz_store.cc:276] 0709 13:18:51.820425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.821318 (+   893us) service_pool.cc:224] Handling call
0709 13:18:53.215979 (+1394661us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54001aa4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.216108 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742516992}
W20260709 13:18:53.218413 10863 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.613s (SENT)
I20260709 13:18:53.219406 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.231279 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.232091 10125 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:53.234736 10125 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:53.226908 15426 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.248008 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.242478 15463 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:53.251847 15463 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.252794 15426 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.253518 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:53.254520 15463 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.261343 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.262511 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.263643 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.265826 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.260802 15367 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:53.267366 15367 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.268075 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.268980 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.269493 10047 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:53.269609 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50275.8us
I20260709 13:18:53.276433 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:53.243036 15459 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:53.273604 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.274729 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.239616 15461 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.288435 15461 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.288985 15461 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.227728 15458 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:18:53.279829 15459 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.291743 15459 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.292940 15458 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:53.271914 15513 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:53.294865 15513 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.297489 15513 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.298214 15513 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:53.298612 15513 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:53.301200 15444 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.302651 10196 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:53.194775 15481 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.273268 15358 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:53.303530 15358 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.304090 15481 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.305114 15461 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.283857 15463 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:53.308141 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.265928 10125 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:53.319734 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:53.310814 15500 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.744s	user 0.000s	sys 0.007s
I20260709 13:18:53.328261 10126 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:53.329986 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.331548 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.324528 15514 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.334771 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:53.335244 10900 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:53.334857 15514 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.335621 10898 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:53.336987 15513 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.341177 10056 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:53.349007 10126 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
W20260709 13:18:53.349833 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71479.7us
I20260709 13:18:53.304121 15358 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.344655 15367 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.353420 15463 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:18:53.357161 15429 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.358645 15429 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:53.361816 15508 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.368295 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.370501 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.371412 10129 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:53.375165 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.400558 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.404367 10124 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:53.404667 14964 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.405403 14964 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.406227 10128 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:18:53.413082 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2536) took 1437 ms (client timeout 1827 ms). Trace:
W20260709 13:18:53.413630 10047 rpcz_store.cc:269] 0709 13:18:51.976017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.995748 (+ 19731us) service_pool.cc:224] Handling call
0709 13:18:53.412868 (+1417120us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002350d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.413012 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595457408}
I20260709 13:18:53.413950 15459 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.415503 10125 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:53.419888 10129 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:53.420116 10050 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:53.422245 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:53.398631 15500 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:18:53.435815 15444 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:53.452157 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.456938 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.457263 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.457557 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.453469 15367 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.459053 15367 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.461694 15367 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.466555 15514 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.467324 15463 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:53.467942 15463 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.470268 15463 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:53.473054 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.476574 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.477970 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:53.489635 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:53.443295 15456 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.495102 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 608) took 1880 ms. Trace:
I20260709 13:18:53.490999 15348 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.503327 15456 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.506047 10895 rpcz_store.cc:276] 0709 13:18:51.614613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.625618 (+ 11005us) service_pool.cc:224] Handling call
0709 13:18:53.339196 (+1713578us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:18:53.339527 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.339901 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.339943 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.352320 (+ 12377us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.453868 (+101548us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:53.460476 (+  6608us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.461043 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.461129 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.494627 (+ 33498us) raft_consensus.cc:1685] finished
0709 13:18:53.494717 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.494871 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6400,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":114,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":6655872,"thread_start_us":46353,"threads_started":1,"wal-append.queue_time_us":117336}
I20260709 13:18:53.504287 15428 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.510048 14964 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.511678 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.518296 10126 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:53.513177 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:53.513844 15358 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.523536 15459 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.523912 15358 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.524401 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 675) took 1174 ms. Trace:
I20260709 13:18:53.525413 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 34) took 1490 ms. Trace:
I20260709 13:18:53.514245 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.515101 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.515300 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.527714 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.520767 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.521507 15460 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:53.521183 15469 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:53.528354 10129 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:53.528797 10126 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:18:53.525919 10198 rpcz_store.cc:276] 0709 13:18:52.034833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.086130 (+ 51297us) service_pool.cc:224] Handling call
0709 13:18:53.525156 (+1439026us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54001aa4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.525302 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775572224}
I20260709 13:18:53.532805 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.524871 10205 rpcz_store.cc:276] 0709 13:18:52.349907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.362412 (+ 12505us) service_pool.cc:224] Handling call
0709 13:18:52.397703 (+ 35291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.567509 (+169806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.567544 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:52.567587 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:18:52.567620 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.524114 (+956494us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400354654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:53.524175 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.524329 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389968256}
I20260709 13:18:53.537189 15346 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:53.537771 15346 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:53.541215 15456 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.541884 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 377) took 1243 ms. Trace:
I20260709 13:18:53.518397 15348 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.542286 10899 rpcz_store.cc:276] 0709 13:18:52.298080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.310195 (+ 12115us) service_pool.cc:224] Handling call
0709 13:18:52.310636 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.378875 (+ 68239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.541629 (+1162754us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b5400604854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:53.541817 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96291712}
W20260709 13:18:53.543956 10052 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 650) took 2437 ms (client timeout 1743 ms). Trace:
I20260709 13:18:53.544049 15348 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.547432 15460 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.548116 15460 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:53.549299 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.550923 15460 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.554149 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:53.554462 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:53.555370 10047 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:53.555490 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:53.556281 10049 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:53.557490 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:53.558079 10049 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:53.558943 15513 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.555022 10051 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:53.559664 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:53.560729 10053 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.560627 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:53.561214 10051 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:53.555641 10054 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:53.567947 15444 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:53.568423 15170 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:53.568814 15170 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:53.568662 15444 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.569796 15170 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.570590 15383 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:53.572628 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.573388 15215 consensus_queue.cc:765] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 14 is ahead of the local log (next sequential op: 14). Destination peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 1.622s
I20260709 13:18:53.574440 15429 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:53.575666 15468 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:53.571100 15444 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.584715 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2084) took 1408 ms. Trace:
W20260709 13:18:53.544430 10052 rpcz_store.cc:269] 0709 13:18:51.106105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.106253 (+   148us) service_pool.cc:224] Handling call
0709 13:18:53.543702 (+2437449us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002e95d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.543840 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447958784}
I20260709 13:18:53.592676 10051 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:53.562659 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:53.601508 10129 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
W20260709 13:18:53.608078 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45501.5us
W20260709 13:18:53.593163 15416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.026s	sys 0.022s
W20260709 13:18:53.609057 15416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.027s	sys 0.023s
I20260709 13:18:53.609407 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:53.606238 10203 rpcz_store.cc:276] 0709 13:18:52.176411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.239048 (+ 62637us) service_pool.cc:224] Handling call
0709 13:18:53.584414 (+1345366us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b54002ddcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.584607 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188781952}
I20260709 13:18:53.570844 15170 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.616621 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:53.617071 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44511.5us
I20260709 13:18:53.622500 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:53.627671 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:53.629616 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:53.611486 15459 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:53.605573 15468 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:53.606597 15230 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:53.623399 15458 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:53.624975 15358 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:53.623049 15461 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:53.605518 15383 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.538810 15346 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.623234 15412 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:53.646983 15456 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:53.660303 10049 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:53.643778 15348 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:53.652792 15230 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.667560 15461 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:53.664513 15456 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:53.676193 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:53.670480 10047 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:53.672452 10053 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:53.672364 15346 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.693994 15521 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.695011 15405 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:53.668128 14964 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:18:53.691954 15456 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:53.702644 15383 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.691843 15429 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:53.704319 15429 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.706795 15429 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.675933 15468 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.708650 15468 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.684756 15461 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:53.727792 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.758952 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2483) took 1629 ms. Trace:
I20260709 13:18:53.759639 10127 rpcz_store.cc:276] 0709 13:18:52.135617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.135794 (+   177us) service_pool.cc:224] Handling call
0709 13:18:52.136174 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.352271 (+216097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.352304 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:52.352351 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:52.352385 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.758559 (+1406174us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400125cd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:53.758634 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.758859 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496340736}
I20260709 13:18:53.764448 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3274) took 1390 ms. Trace:
I20260709 13:18:53.766928 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2567) took 1185 ms. Trace:
I20260709 13:18:53.770406 10050 rpcz_store.cc:276] 0709 13:18:52.582019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.037568 (+455549us) service_pool.cc:224] Handling call
0709 13:18:53.419764 (+382196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.421403 (+  1639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.766584 (+345181us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54002350d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.766825 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865242880}
I20260709 13:18:53.765151 10196 rpcz_store.cc:276] 0709 13:18:52.374441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.697705 (+323264us) service_pool.cc:224] Handling call
0709 13:18:53.764139 (+1066434us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54002184d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.764349 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258733312}
W20260709 13:18:53.775677 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 144715us
W20260709 13:18:53.788364 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 152940us
W20260709 13:18:53.778851 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129013us
I20260709 13:18:53.756578 15437 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:53.773875 15450 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.788431 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 133698us
I20260709 13:18:53.800058 15437 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:53.800746 15437 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:53.802551 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.732115 15461 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.795375 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.818199 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:53.815795 15543 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.834523 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.835724 10125 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:53.836040 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.836843 10124 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:53.842444 10123 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:18:53.845832 15437 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.851364 15461 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:53.852015 15437 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.853422 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:53.853852 15486 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:18:53.851496 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.855248 15486 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:18:53.856290 15455 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:53.856913 15455 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:53.857897 15455 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.861580 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.862497 10125 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:53.862672 15429 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:53.863423 15429 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.864207 10130 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:53.862416 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.865767 15429 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.866236 10124 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
W20260709 13:18:53.866469 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49244.2us
I20260709 13:18:53.869576 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:53.869901 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 80911.4us
I20260709 13:18:53.794220 15450 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.823171 15521 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:53.872761 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:53.876582 10125 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:53.878966 10122 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:18:53.879077 10095 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:53.871068 15513 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:53.881445 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.874186 15412 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:18:53.885586 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.886075 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.886281 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:53.886487 10198 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:53.890472 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.876812 15358 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:53.897902 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:53.877868 15455 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:53.900128 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.900482 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.884083 15416 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:53.905287 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.906018 10205 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:53.907341 15439 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:53.907855 15439 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:53.912798 15458 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:53.913398 15458 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.913816 15458 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:53.915549 15412 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:53.916105 15412 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:53.917927 15450 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.922544 10055 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:53.904585 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.927066 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.898015 10198 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:53.928895 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.931200 10198 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:53.932950 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:53.933563 10198 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.936403 10124 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:53.883558 15513 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.940290 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:53.942242 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:53.869800 15543 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:53.943292 15369 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:53.963649 15369 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.964214 15369 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:53.943580 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.944794 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:53.915298 15430 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:53.968019 10205 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:53.953720 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.874483 15521 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.958037 15543 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:53.973351 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:53.973621 10128 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:18:53.979701 10128 pending_rounds.cc:85] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 11
I20260709 13:18:53.981132 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:53.981648 15456 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.983265 15437 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:53.907385 15416 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:53.984560 15437 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:53.985049 15437 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:53.986640 15416 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:53.954535 15513 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:53.948278 15461 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:53.987557 15530 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:53.986184 15437 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:54.052649 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.993633 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:53.975626 15430 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.053624 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3277) took 1678 ms. Trace:
I20260709 13:18:54.044899 10055 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:18:53.980041 10128 pending_rounds.cc:107] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 14.12
W20260709 13:18:54.047703 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2072) took 2179 ms (client timeout 1725 ms). Trace:
I20260709 13:18:54.061097 10128 raft_consensus.cc:2889] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:54.062747 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2538) took 2085 ms. Trace:
I20260709 13:18:54.063467 10055 rpcz_store.cc:276] 0709 13:18:51.976979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.977206 (+   227us) service_pool.cc:224] Handling call
0709 13:18:51.995893 (+ 18687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.156249 (+1160356us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003497a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa2780061c0 00007fa278007231 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c
0709 13:18:53.921870 (+765621us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5800568ad0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277ffafe7 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4
0709 13:18:54.062336 (+140466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.062656 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228608512}
W20260709 13:18:54.061200 10197 rpcz_store.cc:269] 0709 13:18:51.819998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.820650 (+   652us) service_pool.cc:224] Handling call
0709 13:18:53.999237 (+2178587us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b540012a554. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:53.999377 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049325824}
I20260709 13:18:54.057891 10201 rpcz_store.cc:276] 0709 13:18:52.375296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.814568 (+439272us) service_pool.cc:224] Handling call
0709 13:18:53.588390 (+773822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.588988 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.589031 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.589836 (+   805us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.735441 (+145605us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:53.735509 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.735631 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.845389 (+109758us) raft_consensus.cc:1685] finished
0709 13:18:53.845418 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.008923 (+163505us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54002ddcd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:54.053513 (+ 44590us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":155,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":1778214272,"thread_start_us":145030,"threads_started":1,"wal-append.queue_time_us":204724}
I20260709 13:18:53.917202 15412 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.000973 15207 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:54.078920 15207 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.079499 15207 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.072942 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.083688 10197 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:54.084838 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.085785 10197 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:18:53.991096 15521 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.086680 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.075778 15459 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:53.983330 15429 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:18:54.091688 15459 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.092307 15459 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:54.076500 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.058022 15460 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.059406 15461 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.096040 15460 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.098250 15461 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:54.072489 15361 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:18:54.099684 10197 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:54.020054 15450 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:54.088171 15428 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.094883 15459 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.065562 15430 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.109361 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.116019 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:54.113370 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:54.117401 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:54.129686 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.131738 10197 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:54.103746 15437 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.101739 15450 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.158453 10202 raft_consensus.cc:2376] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12. Re-sending same reply.
I20260709 13:18:54.159312 15450 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:54.162498 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:54.166415 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 216745us
I20260709 13:18:54.167407 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.173076 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.137683 10122 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:54.102912 15428 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.232811 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2492) took 1663 ms. Trace:
I20260709 13:18:54.133755 15528 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:54.107604 15460 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:54.232451 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 238878us
I20260709 13:18:54.235307 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.238315 10127 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:54.239264 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:54.240793 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74132.3us
W20260709 13:18:54.241703 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68734.2us
I20260709 13:18:54.242686 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.233436 10131 rpcz_store.cc:276] 0709 13:18:52.569859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.570025 (+   166us) service_pool.cc:224] Handling call
0709 13:18:52.570497 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.704592 (+1134095us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:53.704958 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.946224 (+241266us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.947330 (+  1106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:53.947390 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:53.947450 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.176168 (+228718us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:54.232581 (+ 56413us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.232739 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36324,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":6123,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":2485,"spinlock_wait_cycles":2607463296,"thread_start_us":227152,"threads_started":2,"wal-append.queue_time_us":208521}
I20260709 13:18:54.248860 10201 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:54.234277 15528 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.180961 15550 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:54.235358 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.258272 15528 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.260882 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.230614 15521 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.262607 10200 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:18:54.247462 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.237957 15188 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:18:54.265466 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.263381 15460 raft_consensus.cc:2764] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:18:54.284523 10020 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:54.237097 15215 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:54.267046 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.270944 15428 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.255888 15526 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.286084 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:54.263422 15550 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.291949 15550 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.284556 15188 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.294379 15188 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.295641 15560 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:54.291934 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:18:54.296265 15560 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.296661 15560 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:54.257166 15468 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.002s
I20260709 13:18:54.297614 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:54.298957 15561 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:54.299430 15561 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.299798 15561 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:54.300104 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.302363 15358 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:54.302930 15358 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:54.303524 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.304033 15358 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.306126 15412 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:54.307950 15188 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.307817 15550 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.310854 15550 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.311180 15230 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.311363 15482 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:54.315384 10205 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:54.322278 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.324643 10894 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:18:54.324707 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77352.4us
I20260709 13:18:54.325994 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.327384 15432 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:54.267944 15459 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.328272 15432 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:54.328194 15459 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.328648 15482 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.329135 15482 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:54.329703 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45314.7us
I20260709 13:18:54.330623 15459 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.330842 15530 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:54.268640 15456 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.332353 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.337198 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:54.337703 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50334.2us
I20260709 13:18:54.340067 10895 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:18:54.340580 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:54.323098 10198 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:54.343043 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:54.345819 10196 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:54.349476 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.349896 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.350559 10898 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:54.354524 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.356320 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:54.358084 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.331382 15530 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:54.364881 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:54.314888 15458 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.361915 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:54.368996 10128 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.352366 15170 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:18:54.331815 15369 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:54.373504 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.373608 15369 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.356325 15464 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:18:54.327600 15207 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.356714 15361 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:54.384265 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.384538 15207 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.387249 10203 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:54.388471 15550 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:54.388968 15550 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:54.389390 15550 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.391278 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:54.393623 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 479705us
I20260709 13:18:54.397367 15450 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.398473 15170 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:54.399715 15530 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.403548 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:54.403265 15439 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.404174 10131 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:54.404723 15439 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.405714 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.406623 10204 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:54.409122 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.415706 10197 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:18:54.418375 10205 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:54.421128 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60739.5us
I20260709 13:18:54.420802 10131 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:54.422353 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.353322 15215 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:54.365770 15521 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:54.362864 15526 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.437973 10125 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:54.430756 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.436395 10203 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:54.438977 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.352753 15418 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:18:54.457463 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.437687 15432 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:54.442086 15513 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.434820 15464 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:18:54.477264 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 179714us
I20260709 13:18:54.466989 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:54.500661 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.483035 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.505987 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:54.506779 10124 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.510454 10129 raft_consensus.cc:2368] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:54.471745 15521 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:54.474181 15558 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:54.477396 15464 consensus_queue.cc:1243] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:54.456254 15384 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:18:54.535804 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:54.536522 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53636.5us
I20260709 13:18:54.538101 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.508900 15450 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.508728 15369 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:18:54.566982  9949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 120) took 11504 ms (client timeout 15000 ms). Trace:
I20260709 13:18:54.506990 15461 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.569010 10095 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:54.570657 10095 leader_election.cc:403] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:54.472211 15430 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.575634 15475 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:54.577045 15513 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.577783 15475 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:54.578332 15475 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.580523 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.568929 15361 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:18:54.569010 15416 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:54.582798 15416 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.583988 15207 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:54.584760 15207 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.585036 15526 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:54.584220 15416 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.585444 15526 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:54.585788 15207 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.586418 15526 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.587643 15548 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:18:54.581902 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.593660 10168 leader_election.cc:400] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:54.594950 15383 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:54.595592 15383 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.596614 15383 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.598268 15188 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:54.598630 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:18:54.599411 10203 raft_consensus.cc:606] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.599426 15469 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.600934 15482 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.601230 15469 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.602083  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:54.571885 15558 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.603818 15558 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.605429  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:54.605952 10898 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:18:54.606241 15558 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.608112 15469 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.608394 15528 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.609478 15475 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.610745 15528 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.480319 14964 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:18:54.535229 15432 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.611222 15475 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:54.611734 15432 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.626530 15432 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.509693 15459 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:18:54.623519 15430 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.528499 15348 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.628026 15348 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.628695 15560 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.631070 15348 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.636909 15432 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.637585 15432 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.640275 15432 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.643395 15369 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:54.632819  9950 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:54.645066 15369 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.645670  9950 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 10, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:54.563272 15405 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.647857 15405 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.670413 15450 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:54.671386 15450 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:54.672246 15459 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:54.672693 15450 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.673786 15460 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.677251 15460 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.679788 10126 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:18:54.681429 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:54.685230 15455 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:54.686131 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:54.687219 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:54.687517 10048 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:54.688053 10051 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:54.687371 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:54.689559 10055 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:54.691526 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:54.693976 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 125154us
I20260709 13:18:54.697525 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.697659 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.698004 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:54.698843 15428 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.698982 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.576828 15461 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.654147  9950 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 5 to 7, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:54.710278 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.710393 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.710691 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:54.722873 10894 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:54.724861 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.725737 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.724870 10204 raft_consensus.cc:2368] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:54.727888 10199 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:54.680007 15460 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.717461 10124 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:54.708967 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.737408 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:54.739259 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 302999us
I20260709 13:18:54.740531 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.742376 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.743268 10205 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:54.744397 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.745589 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.756734 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.631644 15458 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.730165 15428 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:54.759848 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.756588 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:54.759980 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.672739 15459 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:54.718556 15455 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:54.768626  9950 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 6, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:18:54.727245 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.731832 15528 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.771251 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:54.784494 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:54.773842 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46788.9us
I20260709 13:18:54.788429 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:54.789516 10124 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:54.792546 10894 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
W20260709 13:18:54.569657  9949 rpcz_store.cc:269] 0709 13:18:43.042607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.042811 (+   204us) service_pool.cc:224] Handling call
0709 13:18:54.546985 (+11504174us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:49.165293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:49.167261 (+  1968us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:49.167321 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:18:49.167342 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:49.169048 (+  1706us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:49.211248 (+ 42200us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:49.211299 (+    51us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:18:49.213332 (+  2033us) tablet.cc:701] Row locks acquired
0709 13:18:49.213406 (+    74us) write_op.cc:260] PREPARE: finished
0709 13:18:49.213724 (+   318us) write_op.cc:270] Start()
0709 13:18:49.213967 (+   243us) write_op.cc:276] Timestamp: P: 1783603129213616 usec, L: 0
0709 13:18:49.214013 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:18:49.232970 (+ 18957us) log.cc:844] Serialized 5974 byte log entry
0709 13:18:49.581715 (+348745us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277fffe28 00007fa27ba3a581 00007fa27ba39d58 00007fa27ba3e9f1 00007fa27ba3e9a9 00007fa27ba3e939 00007fa27ba3e901 00007fa27ba3db2d 000000000038f354 000000000038f199
0709 13:18:49.675320 (+ 93605us) op_driver.cc:464] REPLICATION: finished
0709 13:18:49.815658 (+140338us) write_op.cc:301] APPLY: starting
0709 13:18:49.821830 (+  6172us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:50.503334 (+681504us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:50.506619 (+  3285us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:51.378150 (+871531us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:51.458556 (+ 80406us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:18:51.458615 (+    59us) write_op.cc:312] APPLY: finished
0709 13:18:51.459167 (+   552us) log.cc:844] Serialized 139 byte log entry
0709 13:18:51.647491 (+188324us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:51.665770 (+ 18279us) write_op.cc:454] Released schema lock
0709 13:18:51.691455 (+ 25685us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:51.777996 (+ 86541us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:51.778045 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":599954,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":89245,"apply.run_cpu_time_us":42481,"apply.run_wall_time_us":1936929,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66707,"num_ops":15,"prepare.queue_time_us":116640,"prepare.run_cpu_time_us":38312,"prepare.run_wall_time_us":439313,"raft.queue_time_us":180159,"raft.run_cpu_time_us":28020,"raft.run_wall_time_us":47633,"replication_time_us":460984,"spinlock_wait_cycles":14232320,"thread_start_us":480982,"threads_started":5,"wal-append.queue_time_us":418309,"wal-append.run_cpu_time_us":66772,"wal-append.run_wall_time_us":2227668}]]}
W20260709 13:18:54.825739 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50029.2us
I20260709 13:18:54.721941 15516 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:18:54.774041 15482 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:54.810585 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.827713 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:54.837250 10124 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
W20260709 13:18:54.850287 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 392883us
I20260709 13:18:54.857978 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:54.855980 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.767102 15412 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:54.763378 15361 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.885082 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:54.888864 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:54.890445 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.905859 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47998.1us
I20260709 13:18:54.784112 15458 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:54.944940 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:54.842945 15405 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:54.882561 15417 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:54.808336 15528 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:54.886159 15560 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:54.847424 15464 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:18:54.909999 15358 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:54.998495 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:54.947515 15188 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.945868 15458 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.887866 15432 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
W20260709 13:18:55.009519 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 198950us
I20260709 13:18:54.842825 15561 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.024226  9950 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.908695 15558 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:54.984288 15230 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:54.941227 15361 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.036621 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:55.009990 15417 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:55.014963 15447 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:55.033239 15526 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.029512 15458 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:54.944571 15459 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.032352 15358 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:55.001597 15405 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:18:55.070215 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.009950 15567 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:18:55.073947 10095 leader_election.cc:400] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.053745 15560 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:55.078403 10095 leader_election.cc:403] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:55.080221 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.042604 15230 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.087983 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:55.092872 10124 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.032084 15188 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.097813 10204 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:55.097723 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:55.101212 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 216241us
I20260709 13:18:55.102227 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.067147 15447 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:55.103648 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.104794 10053 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.082062 15530 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:55.114804 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:55.104292 15439 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.699s	user 0.016s	sys 0.000s
I20260709 13:18:55.097271 15461 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.099829 15459 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:55.100510 15558 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:55.128813 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:55.133009 15464 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:55.142858 15432 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:55.141273 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.146458 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.147372 10049 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.116811 15560 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:55.144620 15432 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.114281 15530 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:55.117375 15486 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.152367 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.152384 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.181860 10124 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:55.182528 10056 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.152710 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.128649 15574 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:55.150024 15565 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.266s	user 0.002s	sys 0.000s
W20260709 13:18:55.165855 15577 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:18:55.166261 15568 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:18:55.136555 15558 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:55.154202 15432 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:55.165930 15361 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.168718 15585 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:55.171929 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:55.152788 15550 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.763s	user 0.015s	sys 0.001s
I20260709 13:18:55.145745 15428 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.177312 10205 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
W20260709 13:18:55.225903 15571 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:18:55.188704 10053 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:55.214953 15461 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.233099 10049 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:18:55.184252 15564 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
W20260709 13:18:55.298740 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88210.4us
I20260709 13:18:55.317965 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.319290 15429 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:55.250713 10051 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:18:55.180521 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:55.164265 15486 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:55.329114 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 119703us
I20260709 13:18:55.333423 10049 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:55.234947 15574 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.376327 15574 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.379092 15574 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.234265 15528 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.386495 10049 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:18:55.389976 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.395251 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 203426us
I20260709 13:18:55.396795 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:18:55.375104 15521 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.845s	user 0.013s	sys 0.006s
I20260709 13:18:55.172386 15358 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:55.404553 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.406769 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.395306 15207 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:55.341132 15561 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:55.423009 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.423312 15570 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
W20260709 13:18:55.423800 15588 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:18:55.432116 15361 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:55.432693 15361 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.433115 15361 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:55.433311 15188 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.309433 15550 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.221818 15447 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.436522 15550 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.397902 15384 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:55.450568 15604 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.463433 15384 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.463598 15604 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.466171 15384 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.247411 15530 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.451170 15475 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.467468 15530 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.467371 15475 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.389008 15558 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.469372 15558 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.470312 15475 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.458006 15486 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.411324 15521 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:18:55.471215 15586 consensus_queue.cc:1175] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 15 status { last_received { term: 13 index: 11 } last_committed_idx: 11 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:18:55.461628 15587 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:18:55.472676 10055 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:55.374720 15461 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.401585 15516 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.476289 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2093) took 1588 ms. Trace:
I20260709 13:18:55.401233 15598 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:55.479243 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:55.485323 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.486407 15598 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.487449 15584 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.488951 15482 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:55.489884 15482 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:55.494611 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3291) took 1079 ms. Trace:
I20260709 13:18:55.494809 15405 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:18:55.489182 15598 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.494436 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.495231 10202 rpcz_store.cc:276] 0709 13:18:54.414861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.415226 (+   365us) service_pool.cc:224] Handling call
0709 13:18:54.434450 (+ 19224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.440651 (+  6201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.440705 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.444342 (+  3637us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.772643 (+328301us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:54.772688 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.773278 (+   590us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.493561 (+720283us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b24003c28b8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274d9d9c4 00007fa274d9bacb 00007fa274f56dc9 00007fa274f56a55 00007fa274f55fb0 00007fa277bcc488 00007fa277bccbe8 00007fa277ba4398 00007fa277b9f76a 00007fa277b9ebb6
0709 13:18:55.494218 (+   657us) raft_consensus.cc:1685] finished
0709 13:18:55.494241 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.494530 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5694,"raft.queue_time_us":20573,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":3047168,"thread_start_us":327879,"threads_started":1,"wal-append.queue_time_us":533927}
I20260709 13:18:55.497051 10047 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.497800 15461 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.498713 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:55.498951 10198 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:55.501433 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:55.503648 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.503345 15516 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.504534 10198 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:18:55.505995 15595 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:18:55.506310 15516 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.507894 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.495424 15482 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.538033 10124 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:55.480338 10199 rpcz_store.cc:276] 0709 13:18:53.887841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.888063 (+   222us) service_pool.cc:224] Handling call
0709 13:18:54.175068 (+287005us) spinlock_profiling.cc:243] Waited 9.67 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:55.476033 (+1300965us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540012b454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:55.476186 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484978816}
I20260709 13:18:55.540511 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.541258 10129 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:55.542340 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.510210 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.552508 10056 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:55.554953 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.555663 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.556387 10197 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:55.560034 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.564352 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.565457 10128 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:55.569921 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.571477 10047 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:55.589498 10204 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:55.589865 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:55.596407 10124 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:18:55.543057 10127 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:55.534778 15458 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.611400 10900 raft_consensus.cc:2368] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
W20260709 13:18:55.605762 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 425341us
I20260709 13:18:55.606211 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.613052 10204 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:55.554363 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.614681 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.615644 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.615901 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.616627 10047 raft_consensus.cc:2368] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.617498 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.618230 10047 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:18:55.618603 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.620184 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.621575 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:55.510208 15230 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.495569 15405 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:55.617117 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.629505 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.626577 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.629182 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.630924 10056 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 10.
I20260709 13:18:55.473202 15604 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.634202 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.590713 15429 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:55.650005 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.649796 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.658574  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:55.635946 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.599174 15516 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.632712 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.661888 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.661800 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.663329 10056 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.662308 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.664117 15513 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:55.674140 15429 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.679618 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.657583 15461 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.682338 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:55.725747 10199 raft_consensus.cc:2376] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16. Re-sending same reply.
I20260709 13:18:55.727362 10050 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:55.729977 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.683504 15405 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.684944 10052 raft_consensus.cc:1217] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.10->[13.11-13.11]   Dedup: 13.11->[]
I20260709 13:18:55.730360 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.671000 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.676872 15460 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:55.687758 15513 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:55.660571 15608 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:18:55.701797 15526 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.634s	user 0.012s	sys 0.000s
W20260709 13:18:55.715320 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.703063 10054 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:55.743235 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.713667 15564 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:55.703020 10055 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:55.712141 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.748775 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.749303 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.730913 10053 raft_consensus.cc:2368] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.749652 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.622996 15361 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.725252 15461 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.734186 15460 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:55.750331 10124 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:55.744004 15603 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.668699 15516 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:55.711068 15459 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.565s	user 0.002s	sys 0.006s
W20260709 13:18:55.710211 15569 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.717s	user 0.000s	sys 0.003s
I20260709 13:18:55.739704 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.731112 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.743388 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.744172 10054 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:55.711760 10122 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:18:55.757400 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.749408 10049 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:55.770744 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.762272 10129 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:55.772136 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.772264 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.763168 10128 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:55.751027 10048 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:18:55.752522 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.743337 14964 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:55.731561 10052 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:18:55.775964 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.765242 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.776906 10130 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:18:55.776835 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.770942 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.752522 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.778510 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.779013 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:55.764097 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:55.766988 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.779989 10898 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.764335 10123 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:18:55.777420 10130 pending_rounds.cc:85] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
I20260709 13:18:55.779167 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.781066 10130 pending_rounds.cc:107] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:18:55.781430 10130 raft_consensus.cc:2889] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:55.781821 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.774583 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.724282 10056 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:55.782810 10047 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:18:55.774253 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.785948 10050 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:18:55.786909 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.772006 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.787683 10050 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.787428 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.788777 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:55.788946 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:55.789872 14964 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:55.790331 15598 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.790809 15598 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.791275 15598 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.791827 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.793604 10122 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:18:55.753888 15461 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.755173 15361 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.795434 15428 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.757144 15460 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:55.797210 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:55.754859 15437 raft_consensus.cc:993] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:55.799916 15437 raft_consensus.cc:1081] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:18:55.753691 15469 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:18:55.775645 15475 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:55.801460 15475 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.801870 15475 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:55.790932 14964 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.754978 15170 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:55.807538 15447 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:55.774253 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.808943 15348 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:55.809443 15547 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.809824 15464 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:55.807538 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.810614 15464 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.813323 15464 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.814261 10898 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:55.814497 15170 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:55.815037 15170 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:55.760037 15358 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:55.815531 15170 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:55.815891 15358 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.808614 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:55.817754 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45851.6us
W20260709 13:18:55.817984 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 43823us
I20260709 13:18:55.820407 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.820511 10050 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:55.821717 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.824824 10055 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:55.726787 15432 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.826345 10049 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:55.826102 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:55.823305 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:55.822504 10204 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:18:55.869378 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60892.6us
W20260709 13:18:55.869766 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81067.6us
I20260709 13:18:55.827396 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:55.845067 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:55.871354 10197 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:55.752597 15614 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.809674 15348 consensus_queue.cc:1243] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:55.871399 10096 leader_election.cc:400] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:55.887928 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:55.888741 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 69829us
I20260709 13:18:55.888890 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:55.889663 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44884.8us
I20260709 13:18:55.875753 10200 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:18:55.890985 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:55.810078 15547 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.889750 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.753309 15405 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:55.759415 15584 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.891423 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:55.873463 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 47459.7us
I20260709 13:18:55.893980 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:55.756668 15458 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.827478 15567 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.759963 15603 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.898211 15560 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:55.897112 15607 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:55.868512 15464 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:18:55.898020 15547 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.921984 15416 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:55.763610 15513 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.923615 15358 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.894987 15361 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:55.926481 15603 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.927358 15567 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.930022 15567 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.868498 15526 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:18:55.932382 15439 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:18:55.932883 15439 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:55.933838 15439 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.936794 10169 leader_election.cc:400] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:55.937299 10169 leader_election.cc:403] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:18:55.905807 15613 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:18:55.752257 15564 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:55.937778 15412 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:18:55.868597 15447 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:55.910203 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.939843 15564 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:55.940868 15526 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:55.885867 15608 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:55.775591 15611 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:55.767554 15516 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:55.871753 15615 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:55.875377 15604 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:55.875605 15583 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:55.918849 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:55.918309 15558 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:18:55.981137 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:55.989276 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99611.7us
I20260709 13:18:55.910763 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:55.987401 15615 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:55.990682 15412 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:55.991580 15615 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.994374 10130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cebbf225b866442ab895577b0496de1b"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:18:55.995059 10130 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:55.995472 10130 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:55.978066 15604 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:55.997363 15604 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:56.002791 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.003191 15543 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:18:56.003561 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.004671 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.007206 15207 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:55.950718 15447 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:56.009192 15461 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:55.943938 15475 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.012272 15475 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:55.920395 10050 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:55.954902 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:55.954990 10131 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:56.014181 15461 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:55.989818 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71097.4us
I20260709 13:18:56.031759 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:55.953990 15439 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.953650 15611 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:56.020025 15558 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:56.020566 15215 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:18:56.021677 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 111010us
I20260709 13:18:56.049010 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:56.022037 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:56.023020 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.887040 15561 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:56.050741 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:56.026562 15526 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:56.061255 10020 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:56.025170 10130 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.064519 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:56.067047 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:56.042443 15611 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:56.042865 15412 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:55.983866 15608 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:18:56.029178 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52224.7us
W20260709 13:18:56.033001 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.009907 15447 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.036545 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:56.127462 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91087us
I20260709 13:18:55.896236 15614 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.132130 15614 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:56.133093 15614 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:18:56.134945 10127 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:18:56.137503 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.059170 15464 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:56.141533 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:56.142745 10050 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:56.146631 10096 leader_election.cc:400] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:56.066674 15607 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:56.151932 10130 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:56.152771 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:56.163455 10050 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:56.068885 15530 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:56.162622 15607 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.063441 15561 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.127727 15444 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:18:56.059090 15215 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.189036 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:56.152668 15464 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.150188 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:56.194155 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44076.6us
I20260709 13:18:56.168514 15412 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.212909 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:56.218464 10894 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:56.168731 15530 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.126044 15460 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.185834 15567 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.188289 15513 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:56.204283 15464 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:56.186571 15607 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:56.220737 15620 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:18:56.210682 15583 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:56.209290 15461 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:56.243152 10129 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:56.236514 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:56.223770 15615 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:56.211346 15598 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.259383 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2599) took 1415 ms. Trace:
I20260709 13:18:56.269169 10048 rpcz_store.cc:276] 0709 13:18:54.844460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.845629 (+  1169us) service_pool.cc:224] Handling call
0709 13:18:56.245552 (+1399923us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54002369d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:56.259277 (+ 13725us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931246976}
I20260709 13:18:56.221064 15530 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:56.184407 15558 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.221187 15432 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:56.249518 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:56.275460 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 216849us
I20260709 13:18:56.276518 15583 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:18:56.248068 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 226086us
I20260709 13:18:56.266314 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:56.264743 15614 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:18:56.273965 10127 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:18:56.275252 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:56.279644 10891 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:56.225409 15412 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.249286 15417 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:56.237255 15513 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.234197 15567 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.284823 15361 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:56.248641 15461 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.333441 15361 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.386842 15621 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:18:56.275403 15615 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.379467 15361 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:56.524151 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.696085 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.011s
W20260709 13:18:56.700881 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.015s	sys 0.017s
W20260709 13:18:56.868331 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.015s	sys 0.017s
W20260709 13:18:56.868325 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.000s	sys 0.012s
I20260709 13:18:56.245348 15608 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:56.897161 15460 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:56.930266 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 652738us
I20260709 13:18:56.872303 15574 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:56.937023 10900 raft_consensus.cc:2368] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
W20260709 13:18:56.939219 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 601423us
I20260709 13:18:56.947803 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:56.951421 10056 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:18:56.952065 10056 pending_rounds.cc:85] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:18:56.952340 10056 pending_rounds.cc:107] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:18:56.952620 10056 raft_consensus.cc:2889] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:56.959059 15625 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:18:56.967319 15634 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.968088 15634 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:56.970727 15634 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:56.930967 15614 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:56.956384 15543 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.953s	user 0.022s	sys 0.005s
I20260709 13:18:56.999356 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2123) took 1233 ms. Trace:
I20260709 13:18:56.999850 10205 rpcz_store.cc:276] 0709 13:18:55.765770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.765908 (+   138us) service_pool.cc:224] Handling call
0709 13:18:55.766237 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.766652 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.766681 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.022727 (+256046us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:18:56.023394 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.254652 (+231258us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:56.254693 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.254777 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.973605 (+718828us) raft_consensus.cc:1685] finished
0709 13:18:56.973645 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.999264 (+ 25619us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":588,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":21659,"spinlock_wait_cycles":14750464,"thread_start_us":190415,"threads_started":1,"wal-append.queue_time_us":248819}
W20260709 13:18:56.972653 10891 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.006s
I20260709 13:18:57.002290 10891 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:18:56.956058 15170 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.140s	user 0.010s	sys 0.016s
W20260709 13:18:56.982203 15604 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.984s	user 0.027s	sys 0.001s
I20260709 13:18:56.947314 15417 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.014166 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2583) took 1094 ms. Trace:
I20260709 13:18:57.034369 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.066576 10126 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:57.065624 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:57.073939 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:57.076855 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:56.979705 15584 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.118351 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3347) took 1297 ms. Trace:
I20260709 13:18:57.064348 10124 rpcz_store.cc:276] 0709 13:18:55.919895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.997774 (+ 77879us) service_pool.cc:224] Handling call
0709 13:18:55.998182 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.998614 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:55.998647 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.007632 (+  8985us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.210730 (+203098us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:56.210798 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.210918 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.013868 (+802950us) raft_consensus.cc:1685] finished
0709 13:18:57.013894 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.014080 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2709,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":202701,"threads_started":1,"wal-append.queue_time_us":257674}
W20260709 13:18:57.121169 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47355.4us
I20260709 13:18:57.119059 10129 rpcz_store.cc:276] 0709 13:18:55.771397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.774323 (+  2926us) service_pool.cc:224] Handling call
0709 13:18:57.068364 (+1294041us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54002d1254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:57.068498 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058746112}
I20260709 13:18:56.996306 15428 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.128306 10048 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:18:56.992532 15460 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.137637 10048 pending_rounds.cc:85] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 11
I20260709 13:18:57.140651 10048 pending_rounds.cc:107] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:18:57.141104 10048 raft_consensus.cc:2889] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:56.976188 15574 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.045276 15450 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.050460 15611 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.898s	user 0.020s	sys 0.001s
W20260709 13:18:57.009972 15405 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.094s	user 0.015s	sys 0.007s
W20260709 13:18:57.050710 15632 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
W20260709 13:18:57.032483 15609 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.428s	user 0.003s	sys 0.006s
I20260709 13:18:57.083674 15629 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:57.062839 15432 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.187212 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:57.208657 10124 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:18:57.197429 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.210845 10129 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.012315 15543 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16.
I20260709 13:18:57.136837 15580 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:57.156745 15215 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.205444 10126 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:18:57.009332 15614 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.138155 15584 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.246524 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:57.218040 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 273036us
I20260709 13:18:57.195797 15561 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.151206 15460 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.050554 15361 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.213663 15637 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.010224 15170 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.105010 15604 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.143456 15428 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.384136 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.209723 15384 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.272228 10129 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:18:57.219182 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3351) took 1395 ms. Trace:
I20260709 13:18:57.261111 15215 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:57.254985 15464 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.978s	user 0.015s	sys 0.005s
I20260709 13:18:57.206640 15598 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:57.351127 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47010.7us
I20260709 13:18:57.350528 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:57.307971 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.304729 15207 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:57.308296 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 44625us
I20260709 13:18:57.424053 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:57.426959 10203 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:57.355302 15629 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:18:57.429131 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 295115us
I20260709 13:18:57.381081 15561 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:18:57.260352 15633 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.186s
I20260709 13:18:57.156450 15450 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.437345 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.392457 15615 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.391429 10126 rpcz_store.cc:276] 0709 13:18:55.824185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.895746 (+ 71561us) service_pool.cc:224] Handling call
0709 13:18:57.034097 (+1138351us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:18:57.219072 (+184975us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4798336}
I20260709 13:18:57.212030 15611 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.271242 15558 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.409017 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.220552 15405 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.419804 10198 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:18:57.420338 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3354) took 1398 ms. Trace:
W20260709 13:18:57.389108 15607 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.139s	user 0.009s	sys 0.009s
I20260709 13:18:57.423071 15567 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:57.404960 15547 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.478s	user 0.018s	sys 0.008s
I20260709 13:18:57.354802 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:57.255261 10204 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:18:57.215915 15580 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:57.217265 15630 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:57.377952 15639 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.475726 10128 rpcz_store.cc:276] 0709 13:18:56.022395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.024598 (+  2203us) service_pool.cc:224] Handling call
0709 13:18:56.040330 (+ 15732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.044826 (+  4496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.044868 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.055277 (+ 10409us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.246890 (+191613us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:56.246954 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.247066 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.402818 (+1155752us) raft_consensus.cc:1685] finished
0709 13:18:57.402859 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.420253 (+ 17394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3732,"raft.queue_time_us":3992,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":11171,"thread_start_us":86249,"threads_started":1,"wal-append.queue_time_us":202707}
I20260709 13:18:57.500783 10198 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:57.419399 15513 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:57.405434 15439 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.358s	user 0.010s	sys 0.013s
I20260709 13:18:57.429880 15461 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.433785 15643 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.473575 15615 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.402268 15384 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.432231 15207 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.482204 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:57.573920 10201 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:57.574445 10201 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:57.520349 15639 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.575284 15460 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.574301 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.575522 10201 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.576474 10198 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:57.576797 10201 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.419350 15464 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.511044 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:57.435787 15215 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:57.469707 15530 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.183s	user 0.012s	sys 0.013s
I20260709 13:18:57.388744 15637 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:57.547592 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 361041us
I20260709 13:18:57.479516 15561 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.466445 15636 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:18:57.497961 15417 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.449820 15629 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.590607 15215 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.593719 15604 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.595508 15654 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.595971 15384 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.518055 15580 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:57.213476 15432 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.596757 15614 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.597164 15464 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.517264 15630 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.489374 15558 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.597868 15170 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:57.598223 15638 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:18:57.483301 15405 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.585170 15615 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.583948 15652 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:57.599541 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.021s
I20260709 13:18:57.599795 15428 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.605818 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 861) took 1621 ms. Trace:
I20260709 13:18:57.508610 15640 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.304368 15543 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.590317 15207 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.508149 15567 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.559463 15439 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.611732 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 156740us
I20260709 13:18:57.692309 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:57.638562 15574 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.563304 15461 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.616864 15417 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:18:57.617365 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.003s	sys 0.003s
W20260709 13:18:57.702071 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.004s	sys 0.003s
W20260709 13:18:57.703038 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 755557us
I20260709 13:18:57.621009 15450 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.494011 15547 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.492807 15607 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.632813 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.633688 10197 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:57.659852 15464 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.605669 15598 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.645823 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.612201 15655 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.649731 15629 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.649943 15405 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.650635 15361 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:57.651029 15647 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:18:57.652689 15637 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.653743 15639 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.561343 15643 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.729460 10200 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:18:57.655870 15432 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.635344 15654 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.604346 15530 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.657881 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:57.659104 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.004s	sys 0.022s
I20260709 13:18:57.663269 10898 rpcz_store.cc:276] 0709 13:18:55.984377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.984601 (+   224us) service_pool.cc:224] Handling call
0709 13:18:55.989466 (+  4865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.989787 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:56.358141 (+368354us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.127690 (+769549us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.604954 (+477264us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.605029 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:57.605111 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.605210 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.605737 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6666,"prepare.queue_time_us":981516,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":4116,"raft.queue_time_us":172645,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":26537,"thread_start_us":1067923,"threads_started":3,"wal-append.queue_time_us":591643}
W20260709 13:18:57.668170 15622 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.434s	user 0.003s	sys 0.000s
I20260709 13:18:57.667776 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:57.668620 15660 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
W20260709 13:18:57.610569 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 113666us
I20260709 13:18:57.610379 15561 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.710852 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.740207 10197 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:18:57.636792 10201 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:57.711336 10096 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:57.712577 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:57.725057 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2621) took 2110 ms. Trace:
I20260709 13:18:57.741595 10056 rpcz_store.cc:276] 0709 13:18:55.614353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.783276 (+168923us) service_pool.cc:224] Handling call
0709 13:18:56.245753 (+462477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.953170 (+707417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.188543 (+235373us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54002369d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:57.724866 (+536323us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54002369d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:57.724995 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2671420416}
I20260709 13:18:57.742163 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:18:57.721349 15638 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:18:57.706276 15644 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:57.734174 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:57.767614 10203 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.768633 15662 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:57.769061 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:57.770165 15661 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:57.770685 15661 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:57.771085 15661 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:57.741613 10204 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:57.751245 15460 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.773237 15460 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.744611 15637 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.776348 15637 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.780838 10203 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:18:57.786208 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:57.746145 10122 pending_rounds.cc:85] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 5
I20260709 13:18:57.746637 10126 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:57.787333 10122 pending_rounds.cc:107] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:18:57.724057 15655 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.787660 10122 raft_consensus.cc:2889] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.788458 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.694146 15543 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.789654 15655 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.789923 10049 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.767522 15626 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:18:57.791992 15543 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.727605 15207 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:18:57.727481 15215 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:57.792948 15215 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.794292 10196 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:57.720980 15614 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:18:57.724308 15464 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.789924 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:57.726943 15405 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:57.796253 15405 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.796911 15464 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.798547 15405 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.756377 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.799347 10130 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:57.800280 15547 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.801966 15607 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.802511 15637 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.713235 15417 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.670943 15652 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.699523 15567 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.699280 15439 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.761408 15608 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:57.823150 15668 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.761467 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:57.823836 15668 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:57.824404 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63015.5us
I20260709 13:18:57.824859 15652 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.825937 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:18:57.826382 15668 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.751418 15450 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.825980 15654 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.827075 15450 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.824924 15558 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.828456 15608 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:57.830416 15450 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.763731 15658 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.831707 15417 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.693009 15640 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.832016 15658 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.834420 15640 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.743682 15639 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.745640 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:57.728952 15598 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.835604 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:57.835389 15639 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.836512 15615 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.783715 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.766121 15432 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.794143 15643 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.755851 15653 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.796473 10126 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:57.757988 15561 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.850040 15660 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.808737 10204 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:57.734629 15584 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.815341 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:57.861793 15660 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:57.862738 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47509us
I20260709 13:18:57.817718 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.818542 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:57.865167 15660 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.712105 15580 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.710289 15461 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.910859 15637 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.914573 15637 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.915715 15580 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.827661 15651 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:57.834511 15658 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.837467 10048 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:57.922870 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.923966 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:57.925362 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.926582 10048 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:57.786841 15666 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.766634 15361 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.762856 15530 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.838136 10055 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:18:57.838581 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.838856 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:57.836905 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:57.937335 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.907768 15640 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.845818 10056 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:18:57.847142 15639 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.847795 15432 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.828080 15439 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.849331 15668 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.857468 15662 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.957120 15668 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.857059 15653 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.920745 15215 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.959949 15653 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.754992 10131 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:57.862870 15486 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:57.821206 10047 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:18:57.962610 15486 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.963367 15486 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.963512 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.964454 10047 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:18:57.965415 15482 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.965502 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.818681 10130 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:57.967607 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3344) took 2197 ms (client timeout 2820 ms). Trace:
W20260709 13:18:57.968045 10127 rpcz_store.cc:269] 0709 13:18:55.770568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.778776 (+  8208us) service_pool.cc:224] Handling call
0709 13:18:57.967429 (+2188653us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540023b9d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:57.967525 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816409984}
I20260709 13:18:57.969394 10127 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:57.891342 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.900669 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.901536 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:57.973066 10126 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:57.973829 10125 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:18:57.850178 15661 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.976787 15661 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.866300 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.977706 10054 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:57.966828 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.865171 10053 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:57.869060 15447 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.979859 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.980631 10053 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:57.981565 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.876070 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:57.896970 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:57.983704 10050 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:57.983946 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108009us
I20260709 13:18:57.986553 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:57.894275 15567 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.987367 14964 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:57.915628 15461 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.826086 15626 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:57.988207 15412 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.988498 15626 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:57.989003 15626 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:57.989379 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.989902 15626 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.928839 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:18:57.929950 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.991096 15661 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:57.991557 10203 raft_consensus.cc:2368] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17.
I20260709 13:18:57.991605 15661 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:57.939232 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100485us
I20260709 13:18:57.995476 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:57.996483 10864 leader_election.cc:400] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17.
I20260709 13:18:57.997002 10864 leader_election.cc:403] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:18:57.998092 15580 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.842566 10049 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:57.999799 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.000558 10049 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:18:58.000787 15416 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:58.001646 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.002504 10049 raft_consensus.cc:2368] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:57.939443 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.918602 15636 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.004271 15636 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.004736 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.005030 15661 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:57.913015 15615 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.931444 15614 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.918852 15637 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:57.955396 15460 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:57.956065 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.957199 15662 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:57.857978 15170 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.858235 15644 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:57.974701 10126 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:18:57.978722 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.034579 10054 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:58.034873 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.938385 10129 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.035449 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:57.982859 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.036406 10053 raft_consensus.cc:2368] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:58.006703 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.036610 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.008145 10047 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:58.037117 10049 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:58.038172 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.038341 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.039446 10047 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.040872 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.041419 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.040885 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.007573 10050 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:18:58.043474 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.016289 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:58.043936 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:57.934619 15450 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.013604 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.017469 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.012748 15626 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:57.933720 15643 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:57.936617 15651 raft_consensus.cc:476] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 13:18:58.029872 10056 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:58.031817 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:57.886065 15611 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:57.950170 15604 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:58.003656 10048 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:58.044632 10050 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:58.086755 10050 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:18:58.043500 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.002055 15416 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:58.051719 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.048815 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.058821 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.060148 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.061187 10129 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:18:58.067016 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49670.3us
I20260709 13:18:57.934185 15666 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:57.935861 15361 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.076150 10203 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:18:58.082856 10047 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:18:58.084043 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.083587 10054 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:18:58.040408 15600 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.086791 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.099475 10894 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:18:57.948168 15640 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.092765 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.108609 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.109648 10895 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:18:58.083678 15430 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:58.112424 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:58.114620 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.029996 15615 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.120553 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.072319 15637 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:58.084787 15604 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.048127 15170 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.137975 10055 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.142140 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.045434 15614 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.093966 15416 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:58.152210 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.111167 15657 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:18:58.169530 10202 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:18:58.171380 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.128233 15369 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:58.077704 15450 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.129024 15668 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.078401 15643 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.137563 15604 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.176435 15369 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.205984 10129 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:18:58.176990 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 683) took 2075 ms. Trace:
I20260709 13:18:58.128854 15633 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:18:58.189054 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:58.194574 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50705.9us
I20260709 13:18:58.165390 10055 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:58.134654 15430 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:58.212523 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.216687 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:58.207451 10051 rpcz_store.cc:276] 0709 13:18:56.101451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.102880 (+  1429us) service_pool.cc:224] Handling call
0709 13:18:56.104265 (+  1385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.223673 (+119408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.223711 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.223759 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:18:56.223795 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.176672 (+1952877us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:58.176728 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.176909 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274009856}
I20260709 13:18:58.227674 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.225354 10051 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:58.216343 15369 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:58.119817 15600 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.193153 15361 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.225158 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.228608 10056 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:18:58.209754 15596 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.227576 15486 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.238122 15450 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.208989 15637 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:58.240674 15598 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:58.241623 10900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 390) took 2287 ms (client timeout 1722 ms). Trace:
I20260709 13:18:58.212263 15658 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:18:58.244135 15615 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.247023 15430 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.248528 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:58.280138 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:58.249178 15668 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:58.249565 15654 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:18:58.228964 15645 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:18:58.250134 15170 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:58.210021 15547 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:18:58.223770 15641 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:58.262912 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2217) took 2516 ms. Trace:
I20260709 13:18:58.300709 10052 rpcz_store.cc:276] 0709 13:18:55.737083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.798362 (+ 61279us) service_pool.cc:224] Handling call
0709 13:18:55.798678 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.493550 (+1694872us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:57.493603 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.493673 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:57.493751 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.243092 (+749341us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400135e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:58.252683 (+  9591us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.252879 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":916,"raft.queue_time_us":283551,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":16321,"spinlock_wait_cycles":3250710912,"thread_start_us":280841,"threads_started":1}
I20260709 13:18:58.228137 15636 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:58.219340 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.230804 15662 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:58.272820 10900 rpcz_store.cc:269] 0709 13:18:55.954252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.954494 (+   242us) service_pool.cc:224] Handling call
0709 13:18:58.241372 (+2286878us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54001e64d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:58.241522 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590417664}
I20260709 13:18:58.254925 15657 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:18:58.312184 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.242935 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.248512 15643 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.258039 15596 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:18:58.318336 10200 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2125) took 2268 ms (client timeout 2208 ms). Trace:
I20260709 13:18:58.296165 15567 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:18:58.320233 10200 rpcz_store.cc:269] 0709 13:18:56.050297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.050952 (+   655us) service_pool.cc:224] Handling call
0709 13:18:58.318070 (+2267118us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001bfb54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:58.318247 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3826814720}
I20260709 13:18:58.264019 15580 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17.
I20260709 13:18:58.229010 15416 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.306246 10129 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:58.338282 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.299834 15432 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.307547 15464 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:18:58.312018 15215 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.352917 15215 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.323764 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 695) took 1095 ms. Trace:
I20260709 13:18:58.307780 15645 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:58.354831 10205 rpcz_store.cc:276] 0709 13:18:57.228433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.238412 (+  9979us) service_pool.cc:224] Handling call
0709 13:18:57.238944 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.649540 (+410596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.649579 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.649628 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:57.649665 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.323494 (+673829us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:58.323551 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.323685 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943707904}
I20260709 13:18:58.356065 15645 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.339249 10129 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:18:58.344563 15486 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.346089 15416 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:58.360173 15600 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:58.363807 15614 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.364516 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:58.365326 10123 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.367516 15637 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.369062 15215 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.372972 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:58.374531 15666 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.307101 15668 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:58.347533 15596 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:58.348464 15567 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:18:58.403443 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 160551us
I20260709 13:18:58.405275 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.415467 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.417019 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.417230 15678 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:18:58.419517 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.423847 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:58.428570 10895 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:58.423727 15384 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:58.428645 10129 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:18:58.429978 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.424973 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.425025 15666 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.431077 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.433920 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.434911 10129 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 8.
I20260709 13:18:58.424904 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.435998 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.419035 15369 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.375903 15668 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.437844 10096 leader_election.cc:400] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:58.438131 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.449113 10096 leader_election.cc:403] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:58.449677 10894 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:18:58.442507 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.467757 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.465731 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.478381 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:18:58.481462 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 43797.4us
I20260709 13:18:58.481667 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.482432 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.482981 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.485253 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.436751 15678 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.495672 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.497000 10096 leader_election.cc:400] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:58.497342 10096 leader_election.cc:403] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:18:58.445739 15614 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:18:58.462550 15600 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:58.461453 15215 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:58.462352 15547 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:58.509663 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.494117 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:58.492864 15444 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:18:58.513949 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.443146 15615 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:58.484094 15652 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:58.523595 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81164us
I20260709 13:18:58.528596 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:58.441936 15666 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.526957 10123 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:58.501308 15643 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.507529 15412 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.531355 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.532392 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.501679 15678 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.531844 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.516186 15600 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:58.506542 15614 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:18:58.571856 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49665.4us
I20260709 13:18:58.572264 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:58.574506 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:58.575594 10123 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:58.519083 15547 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:58.577913 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46625.5us
I20260709 13:18:58.578298 10049 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:18:58.526422 15615 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.564652 15444 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:18:58.580597 15564 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:18:58.557348 15482 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.003s
I20260709 13:18:58.563474 15672 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:18:58.529467 15680 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.592092 15450 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.578135 15600 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.594208 10895 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:58.594810 10899 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:18:58.526316 15432 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:58.671262 15432 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.671898 15432 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:58.530886 15633 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.674078 15633 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:58.674566 15633 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.649662 15626 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:58.675488 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.677119 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.677819 10200 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:58.678740 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.528777 15604 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.600464 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:58.602471 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.603837 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.604562 15651 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.610554 15671 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.420s	user 0.004s	sys 0.009s
W20260709 13:18:58.697631 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93909.6us
I20260709 13:18:58.698202 10894 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:18:58.621443 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.627445 10052 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.619284 15547 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:58.650764 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:58.708933 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87645us
I20260709 13:18:58.709352 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.528646 15567 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.657882 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.659092 10891 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:18:58.613998 15678 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.662263 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.662783 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.716882 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:58.717101 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.648439 15416 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:58.521255 15215 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.667985 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.565183 15369 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:58.661700 10123 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:58.576790 15412 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.576735 15614 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:18:58.597829 15666 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:58.679706 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.728473 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2124) took 2900 ms. Trace:
I20260709 13:18:58.728778 10203 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:58.654280 15564 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.658897 15672 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:58.686262 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:18:58.686539 15661 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.681s	user 0.004s	sys 0.008s
I20260709 13:18:58.649729 15615 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.693298 10200 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:18:58.700096 10122 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.016s	sys 0.030s
W20260709 13:18:58.744585 10122 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.016s	sys 0.030s
I20260709 13:18:58.746402 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:58.705324 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:18:58.748237 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:18:58.749565 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:18:58.707077 10047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.035s	sys 0.012s
W20260709 13:18:58.750296 10047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.035s	sys 0.012s
I20260709 13:18:58.660723 10201 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:18:58.713491 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.714093 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.717779 10197 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.700876 10900 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:18:58.535542 15652 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.724581 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.723048 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.729019 10199 rpcz_store.cc:276] 0709 13:18:55.819699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.819887 (+   188us) service_pool.cc:224] Handling call
0709 13:18:55.891068 (+ 71181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.891367 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.891395 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.908772 (+ 17377us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.031215 (+122443us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:56.031255 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.031339 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.667477 (+1636138us) spinlock_profiling.cc:243] Waited 8.17 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:57.760679 (+ 93202us) raft_consensus.cc:1685] finished
0709 13:18:57.760702 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.719780 (+959078us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b54002fbcd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:58.719919 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26421888,"thread_start_us":89142,"threads_started":1,"wal-append.queue_time_us":278756}
I20260709 13:18:58.731835 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.699141 15596 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.735664 10050 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:58.765206 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.766148 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:18:58.740934 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.741497 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.741873 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.775581 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:58.776855 10050 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:18:58.765792 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.780753 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.767800 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.782612 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.785727 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.790782 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.791312 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.743736 15564 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:58.746533 10052 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:18:58.717105 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:58.797910 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84540.7us
I20260709 13:18:58.798561 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:58.798952 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.799284 10201 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.803406 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.745138 15672 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:58.733014 15412 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.712214 15455 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.718477 15645 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:58.715931 15651 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.796432 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.801546 10051 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 8.
I20260709 13:18:58.785256 15636 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.784881 15486 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:58.785089 15482 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:58.812494 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.751940 10049 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:18:58.791796 15654 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:18:58.815949 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93022.4us
I20260709 13:18:58.816885 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.761946 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:58.758806 15661 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.828011 10197 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:58.797237 15416 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:58.835352 10201 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:18:58.805704 15444 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:58.847954 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2640) took 1029 ms. Trace:
I20260709 13:18:58.853255 10048 rpcz_store.cc:276] 0709 13:18:57.818863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.085216 (+266353us) service_pool.cc:224] Handling call
0709 13:18:58.090698 (+  5482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.090890 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.090915 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.090964 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:58.090998 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.091050 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.847675 (+756625us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b540010a9d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:58.847859 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11773312}
I20260709 13:18:58.855343 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.839731 15369 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:18:58.784793 15596 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.864750 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2611) took 1083 ms. Trace:
I20260709 13:18:58.866004 10127 rpcz_store.cc:276] 0709 13:18:57.781039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.047838 (+266799us) service_pool.cc:224] Handling call
0709 13:18:58.048309 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.048592 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:58.313519 (+264927us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.863884 (+550365us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.864113 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.864351 (+   238us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:58.864422 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.864510 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.864684 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3931,"prepare.queue_time_us":266668,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":35344,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":13765,"thread_start_us":780324,"threads_started":3}
I20260709 13:18:58.846050 15645 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:18:58.869285 15688 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:18:58.870025 15645 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.840562 15486 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:58.874658 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.841163 15636 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:58.882859 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.840960 15482 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:58.876813 10196 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:58.877420 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.845525 15461 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:58.880144 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.885115 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:58.884711 15486 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:58.860422 15450 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:58.822937 15455 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.902386 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.897567 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.906934 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.909512 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:58.898968 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:58.911690 10197 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:58.913694 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:58.874053 15369 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:18:58.887367 15636 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:58.886705 15482 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:58.894335 15680 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:58.930418 15369 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.939870 10197 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:18:58.944819 15215 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:58.947693 15630 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:58.954560 15547 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.963534 15416 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:58.964061 15482 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:58.965690 15547 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:58.913439 15580 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:18:58.918721 15662 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:58.972669 15383 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:18:58.987804 15665 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:59.000622 15678 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.000142 15652 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.021173 15643 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13.
I20260709 13:18:59.055766 15640 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.017706 15662 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.075207 15705 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:18:59.080919 15432 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.081722 15564 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.083467 15432 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.071014 15654 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:59.088322 15651 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.088145 15654 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.073859 15596 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.100244 15596 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.100817 15596 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.074498 15600 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.056133 15633 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.119342 15636 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.102970 15596 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.128506 15672 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.133245 15672 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.174888 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2240) took 1232 ms. Trace:
I20260709 13:18:59.177096 10204 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:18:59.179718 10050 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:59.185847 10123 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:18:59.190353 10125 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:59.190697 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2589) took 1445 ms. Trace:
I20260709 13:18:59.176275 10056 rpcz_store.cc:276] 0709 13:18:57.942542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.257101 (+314559us) service_pool.cc:224] Handling call
0709 13:18:58.279533 (+ 22432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.292302 (+ 12769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.292338 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.300560 (+  8222us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.698833 (+398273us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.698898 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.698984 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.173910 (+474926us) raft_consensus.cc:1685] finished
0709 13:18:59.174010 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.174526 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5845,"raft.queue_time_us":17049,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":10626,"thread_start_us":388735,"threads_started":1,"wal-append.queue_time_us":581140}
I20260709 13:18:59.039054 15383 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.219719 10127 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:18:59.222713 15450 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:59.225394 15450 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.228281 15615 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.056692 15665 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.236850 15564 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.237205 15665 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.246137 10128 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:59.086966 15486 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.263226 15486 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.200973 10124 rpcz_store.cc:276] 0709 13:18:57.745137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.746221 (+  1084us) service_pool.cc:224] Handling call
0709 13:18:57.746576 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.746797 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:18:58.100787 (+353990us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.599915 (+499128us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b180049e280. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:18:58.819597 (+219682us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.189586 (+369989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:59.189738 (+   152us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:59.189940 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.190145 (+   205us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.190456 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4833,"prepare.queue_time_us":530742,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":3869,"raft.queue_time_us":264004,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":6137,"spinlock_wait_cycles":12643968,"thread_start_us":1086946,"threads_started":4,"wal-append.queue_time_us":400259}
I20260709 13:18:59.297776 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2652) took 1027 ms. Trace:
I20260709 13:18:59.239218 15708 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:18:59.321045 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2649) took 1459 ms. Trace:
I20260709 13:18:59.321426 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2597) took 1567 ms. Trace:
I20260709 13:18:59.323220 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2598) took 1568 ms. Trace:
I20260709 13:18:59.330559 10051 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:18:59.209642 15680 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:18:59.331812 15707 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.332536 15486 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:59.330559 10053 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:59.343722 10130 rpcz_store.cc:276] 0709 13:18:57.754318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.967633 (+213315us) service_pool.cc:224] Handling call
0709 13:18:57.967964 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.224425 (+256461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.224462 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.224512 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.224543 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.301576 (+1077033us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001270d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.322490 (+ 20914us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.322953 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":589226752}
I20260709 13:18:59.343592 10131 rpcz_store.cc:276] 0709 13:18:57.754132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.961606 (+207474us) service_pool.cc:224] Handling call
0709 13:18:57.962111 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.167013 (+204902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.167075 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.167159 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:58.167218 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.302212 (+1134994us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002ed1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.320731 (+ 18519us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.321146 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470447488}
I20260709 13:18:59.341382 15461 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.350721 15455 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.343863 10055 rpcz_store.cc:276] 0709 13:18:57.860931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.004445 (+143514us) service_pool.cc:224] Handling call
0709 13:18:59.292985 (+1288540us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002e8e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:59.320701 (+ 27716us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304577792}
I20260709 13:18:59.336266 10050 rpcz_store.cc:276] 0709 13:18:58.270900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.777891 (+506991us) service_pool.cc:224] Handling call
0709 13:18:58.778380 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.180653 (+402273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.297336 (+116683us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001b8854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.297674 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921022336}
I20260709 13:18:59.334995 15636 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:18:59.208154 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 295767us
I20260709 13:18:59.374163 15461 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.338963 15615 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:59.384752 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.355559 15708 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.388438 15708 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.389597 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:59.391832 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:59.392884 10894 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:59.397544 15707 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.400755 15707 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:18:59.374327 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 470600us
W20260709 13:18:59.407073 15614 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.004s
I20260709 13:18:59.413378 15215 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:18:59.444818 15667 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:59.416145 15383 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:59.446988 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.658s (SENT) [suppressed 8 similar messages]
I20260709 13:18:59.420801 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:18:59.455641 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59271.7us
I20260709 13:18:59.380643 15486 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.480022 15667 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:59.483068 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:59.483458 15215 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.464762 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:59.465164 15543 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.485682 15543 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.487627 15564 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.488919 15543 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.488725 15564 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.457574 15615 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:18:59.485682 10197 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:59.492400 15706 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:59.493430 15706 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.494172 15706 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.466475 15662 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.497044 15705 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:18:59.497975 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:18:59.498499 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2136) took 1463 ms (client timeout 1725 ms). Trace:
I20260709 13:18:59.499653 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 691) took 1154 ms. Trace:
I20260709 13:18:59.500715 10205 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:18:59.500571 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:59.470674 15633 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.478997 15600 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.479753 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:18:59.498917 10124 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:18:59.491880 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 71356.1us
I20260709 13:18:59.507373 10130 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:18:59.500349 10052 rpcz_store.cc:276] 0709 13:18:58.344904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.516749 (+171845us) service_pool.cc:224] Handling call
0709 13:18:59.499242 (+982493us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b5400235ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:59.499446 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225041280}
W20260709 13:18:59.499217 10202 rpcz_store.cc:269] 0709 13:18:58.035398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.036304 (+   906us) service_pool.cc:224] Handling call
0709 13:18:59.498042 (+1461738us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54002175d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:59.498254 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304079616}
I20260709 13:18:59.516834 15707 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:59.517616 15707 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:59.520104 10895 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:59.520208 15680 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.521376 15680 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:18:59.524227 15665 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:59.531610 15667 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.534374 15667 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:59.547472 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:18:59.549593 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 166015us
I20260709 13:18:59.525350 15614 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.466089 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:59.557840 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:59.558995 15455 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:59.559689 15455 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.560169 15455 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.566162 10095 leader_election.cc:400] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:18:59.566679 10095 leader_election.cc:403] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:18:59.567653 15694 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.639s	user 0.003s	sys 0.000s
I20260709 13:18:59.567893 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:18:59.568518 15687 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.655s	user 0.002s	sys 0.000s
I20260709 13:18:59.569128 10202 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:18:59.529443 15665 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.575392 15665 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.578105 15604 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.553133 15707 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.576177 15705 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:59.483027 15486 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.586316 15614 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.587193 10122 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:18:59.588722 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2647) took 1728 ms. Trace:
I20260709 13:18:59.595317 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2663) took 1171 ms. Trace:
I20260709 13:18:59.597447 10056 rpcz_store.cc:276] 0709 13:18:58.423599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.326894 (+903295us) service_pool.cc:224] Handling call
0709 13:18:59.428095 (+101201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.584354 (+156259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.584384 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:59.594964 (+ 10580us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:59.595012 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.595082 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.595250 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:59.562868 15412 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.735s	user 0.006s	sys 0.015s
I20260709 13:18:59.610899 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2133) took 1585 ms. Trace:
I20260709 13:18:59.612198 15651 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:59.612888 15651 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.611629 10130 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:59.613462 15651 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:59.613590 10130 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:59.614430 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2629) took 1123 ms. Trace:
I20260709 13:18:59.614916 10128 rpcz_store.cc:276] 0709 13:18:58.491034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.587454 (+ 96420us) service_pool.cc:224] Handling call
0709 13:18:58.587916 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.247082 (+659166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.614027 (+366945us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54001261d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.614341 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480146176}
I20260709 13:18:59.615643 10130 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.617226 10130 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:59.617520 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:59.618695 10125 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:59.620314 15698 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:18:59.621750 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:59.622725 10123 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:18:59.630270 15564 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.632453 15707 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.637353 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:59.638057 15708 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.641157 10130 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:59.611553 10196 rpcz_store.cc:276] 0709 13:18:58.025522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.025768 (+   246us) service_pool.cc:224] Handling call
0709 13:18:58.026229 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.886893 (+860664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.610206 (+723313us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b540012b454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.610649 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955576704}
I20260709 13:18:59.648890 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:59.652544 10124 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:18:59.658383 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60842.3us
I20260709 13:18:59.654517 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:18:59.595156 10047 rpcz_store.cc:276] 0709 13:18:57.860347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.116607 (+256260us) service_pool.cc:224] Handling call
0709 13:18:58.117178 (+   571us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:58.117765 (+   587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.117818 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:58.118689 (+   871us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.750735 (+632046us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:58.750782 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.750888 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.567316 (+816428us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:18:59.576690 (+  9374us) raft_consensus.cc:1685] finished
0709 13:18:59.576720 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.588479 (+ 11759us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74665,"raft.queue_time_us":144,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7693,"spinlock_wait_cycles":23326592,"thread_start_us":574005,"threads_started":1,"wal-append.queue_time_us":718024}
I20260709 13:18:59.652323 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:18:59.673645 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 394) took 1590 ms (client timeout 1778 ms). Trace:
W20260709 13:18:59.674288 10891 rpcz_store.cc:269] 0709 13:18:58.043004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.043195 (+   191us) service_pool.cc:224] Handling call
0709 13:18:59.632641 (+1589446us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b540063ef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:18:59.632862 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413868928}
I20260709 13:18:59.675087 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:59.671784 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:59.685269 15608 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:59.686111 15608 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.686546 15608 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:59.688238 15482 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.685436 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 693) took 1300 ms. Trace:
I20260709 13:18:59.682308 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:18:59.680605 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75471.1us
I20260709 13:18:59.696560 10123 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:18:59.679024 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:18:59.649811 15215 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.691246 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.690683 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:18:59.692838 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:59.703940 10125 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:18:59.704416 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:59.711347 10050 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:18:59.692957 10049 rpcz_store.cc:276] 0709 13:18:58.345529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.851053 (+505524us) service_pool.cc:224] Handling call
0709 13:18:58.851485 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.351247 (+499762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.351321 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.352345 (+  1024us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.381523 (+ 29178us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:18:59.381581 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.429650 (+ 48069us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.639662 (+210012us) raft_consensus.cc:1685] finished
0709 13:18:59.639688 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.645836 (+  6148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":498912,"raft.queue_time_us":562113,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"thread_start_us":28430,"threads_started":1,"wal-append.queue_time_us":142889}
I20260709 13:18:59.712872 15666 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:59.713578 15666 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.713922 15666 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:18:59.695166 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57527.3us
I20260709 13:18:59.716181 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2618) took 1705 ms. Trace:
I20260709 13:18:59.716723 10126 rpcz_store.cc:276] 0709 13:18:58.013406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.070642 (+ 57236us) service_pool.cc:224] Handling call
0709 13:18:58.070944 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.230163 (+159219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.230198 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:58.230246 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:58.230278 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.682125 (+1451847us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:18:59.696780 (+ 14655us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.715996 (+ 19216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365662976}
I20260709 13:18:59.714761 15461 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:18:59.718526 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 163612us
I20260709 13:18:59.697230 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:18:59.720010 15450 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:18:59.721232 15450 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:59.721927 15580 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.722267 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.722872 15450 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:59.723347 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2648) took 1838 ms. Trace:
I20260709 13:18:59.723675 15450 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:59.723945 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:18:59.725268 15450 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.725919 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.723469 10125 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:18:59.727269 15691 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.779s	user 0.001s	sys 0.004s
I20260709 13:18:59.727823 15654 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:18:59.729629 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:18:59.722707 15580 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.643009 15412 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.737270 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.734982 10128 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:59.752003 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.739533 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.682096 15659 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:18:59.747278 15713 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:18:59.724023 10054 rpcz_store.cc:276] 0709 13:18:57.860663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.118819 (+258156us) service_pool.cc:224] Handling call
0709 13:18:58.119301 (+   482us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:58.130681 (+ 11380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.130739 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:58.140017 (+  9278us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.477059 (+337042us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:58.515421 (+ 38362us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.583416 (+ 67995us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b18004abc00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:18:58.583733 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.069860 (+486127us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b18004abc00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:18:59.346917 (+277057us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b18004abc00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa27bed858f 00007fa274f17571 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd
0709 13:18:59.657626 (+310709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.677556 (+ 19930us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.682765 (+  5209us) raft_consensus.cc:1685] finished
0709 13:18:59.682809 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.698536 (+ 15727us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":905880,"prepare.queue_time_us":600233,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":3306,"raft.queue_time_us":846976,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":22150,"spinlock_wait_cycles":20124160,"thread_start_us":458654,"threads_started":2,"wal-append.queue_time_us":399715}
I20260709 13:18:59.759385 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.760015 15450 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.761207 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.761510 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:18:59.747486 15716 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.765162 15705 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:18:59.765715 15705 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.744782 15654 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:18:59.766083 15444 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.748852 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:18:59.749408 15717 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:59.748968 15455 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.759457 15461 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.778479 15455 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.767884 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.772502 15580 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.770134 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2656) took 1349 ms. Trace:
I20260709 13:18:59.768098 15654 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.784866 15412 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.774780 15716 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:59.785745 10129 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:18:59.773686 15661 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.781759 10048 rpcz_store.cc:276] 0709 13:18:58.420916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.865859 (+444943us) service_pool.cc:224] Handling call
0709 13:18:59.172568 (+306709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.348969 (+176401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.349016 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.349887 (+   871us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.370683 (+ 20796us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:59.370719 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.374632 (+  3913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.767330 (+392698us) raft_consensus.cc:1685] finished
0709 13:18:59.767351 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.769909 (+  2558us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175183,"raft.queue_time_us":239967,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":237,"thread_start_us":20246,"threads_started":1,"wal-append.queue_time_us":91412}
I20260709 13:18:59.766088 15705 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:18:59.795356 10124 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:18:59.798357 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46870.3us
I20260709 13:18:59.766976 15444 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.801527 15564 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.802371 15564 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.805171 15564 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:18:59.808804 15444 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:18:59.809942 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:59.771303 15680 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.817941 15680 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.799474 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:18:59.824846 15412 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.825526 15412 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:18:59.826023 15412 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.797812 15706 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.837474 15706 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:18:59.838569 15719 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:18:59.845714 10127 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:18:59.864185 15450 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:18:59.865207 15450 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:18:59.868880 15450 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:18:59.873245 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:59.874249 10048 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:18:59.875313 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:18:59.879177 10050 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:18:59.853376 15412 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.838598 15706 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:18:59.892659 15450 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:18:59.896005 15450 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:18:59.897686 15450 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:18:59.891362 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:59.901121 15667 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:18:59.902209 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 123121us
W20260709 13:18:59.903498 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88276us
I20260709 13:18:59.905359 10020 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:18:59.908893 15667 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:18:59.909636 15667 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:18:59.910837 15667 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:00.006479 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.018784 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.025974 10056 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:18:59.887029 10048 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:18:59.914412 10124 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:18:59.940840 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:18:59.915076 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:18:59.920557 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 176310us
I20260709 13:19:00.147025 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.160596 10052 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:18:59.926821 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:00.130426 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.113895 10201 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:00.103104 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.939836 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 493025us
I20260709 13:18:59.910974 15665 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.168327 15645 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:59.973157 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 255779us
W20260709 13:19:00.177202 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.179914 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:18:59.924353 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.054983 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2630) took 1563 ms. Trace:
I20260709 13:19:00.180751 10131 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:00.197197 10125 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:00.181653 10122 rpcz_store.cc:276] 0709 13:18:58.491446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.805944 (+314498us) service_pool.cc:224] Handling call
0709 13:18:58.806406 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.606450 (+800044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.054491 (+448041us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54002d1754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:00.054866 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768481536}
I20260709 13:19:00.056332 15636 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:00.166646 15665 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.208966 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.210182 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.210476 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.211483 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.212090 10051 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:00.215564 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.217592 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.221764 10051 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:00.222426 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:00.224049 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.207540 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.228508 10053 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:00.229064 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.225852 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.199658 15450 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:00.229601 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.244868 15716 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:00.245126 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.247175 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:19:00.247406 10053 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:00.246606 10131 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.250332 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:00.253142 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 347851us
I20260709 13:19:00.255306 10129 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:00.257508 10048 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:00.252552 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.262070 10051 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:19:00.245541 10050 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:00.261898 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.111291 15608 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.263738 15564 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:00.264240 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.265197 10124 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.265842 15708 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:00.266088 15608 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.266335 15708 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:00.013939 15651 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.266130 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.268987 15651 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.266973 10051 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:00.271509 10051 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:00.224455 15636 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:00.238183 15666 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.516s	user 0.014s	sys 0.000s
I20260709 13:19:00.276598 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:00.323504 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 382803us
I20260709 13:19:00.279191 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.323433 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.324959 15666 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.284642 15412 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.288553 10054 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:00.326784 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.351140 10127 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:00.352360 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:00.280594 10056 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:00.353194 10127 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.204576 15705 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.277191 15708 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.329629 10051 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.337329 10131 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:00.336061 10052 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:00.346958 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:00.346000 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:00.348758 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.345151 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90999.5us
I20260709 13:19:00.283358 10125 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.320546 10128 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.311158 10123 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:00.360005 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.364363 10050 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:00.375309 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.376515 10123 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:00.371415 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3336) took 1510 ms. Trace:
I20260709 13:19:00.379351 10049 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:19:00.349360 15604 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.381628 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.382623 10048 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:19:00.378597 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.383630 10053 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.377509 10201 rpcz_store.cc:276] 0709 13:18:58.860581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.860882 (+   301us) service_pool.cc:224] Handling call
0709 13:18:58.861288 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.163340 (+1302052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.349166 (+185826us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:00.371341 (+ 22175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2879389056}
I20260709 13:19:00.387768 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.388263 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.388806 10129 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:00.389761 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:00.380527 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 184575us
I20260709 13:19:00.388064 15636 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.387609 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.363624 15444 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:00.363029 15705 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.395211 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:00.387784 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64970.4us
I20260709 13:19:00.397385 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:00.404909 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:00.408279 10128 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.399744 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:00.408567 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.409776 10123 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.411185 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:00.411561 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.413080 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:00.414108 15564 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:00.415071 15564 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:00.415767 10130 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:00.415517 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.416494 10129 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:00.417160 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.411582 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.422483 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.424973 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.426151 10123 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:00.426638 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:00.427062 10051 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:00.431499 10131 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 9: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:00.432722 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.401813 15444 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:00.404206 15708 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.435379 10123 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:00.433823 10052 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:00.432173 10122 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:00.442662 10048 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:00.451077 10049 pending_rounds.cc:85] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:19:00.403230 15604 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.452896 15636 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:00.460520 10049 pending_rounds.cc:107] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:19:00.469967 10049 raft_consensus.cc:2889] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:00.470824 15661 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:19:00.471488 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:00.471505 15716 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:00.479003 15716 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:00.483433 10127 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:00.481882 15664 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:00.484619 15716 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.470960 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:00.484643 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.493762 10050 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:00.492710 15651 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.503705 15651 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.505172 15666 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:00.506922 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97592.4us
I20260709 13:19:00.507342 10129 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:00.509052 15645 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:00.490175 15664 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:00.513046 15664 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.498827 15604 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:00.516108 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.518119 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.519593 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.520622 10054 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:00.523758 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:00.538729 10048 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:00.541575 10053 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:00.545923 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.546428 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:00.491233 15444 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.564821 15444 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:00.548849 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0686187870001049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:19:00.567566 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0659187870001006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:19:00.567986 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0596187870000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 13:19:00.568377 10019 connection.cc:256] RPC call timeout handler was delayed by -1.044318787000293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
W20260709 13:19:00.568821 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0008540350002022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 13:19:00.568975 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:00.573601 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.574429 10202 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:00.571655 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:00.576510 10196 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:00.581235 10203 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:00.579734 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.599501 10196 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
W20260709 13:19:00.643999 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 111077us
I20260709 13:19:00.595546 15482 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:19:00.657774 10124 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:19:00.567337 15645 consensus_queue.cc:1175] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "01bfaf1ad56a4deea886bbc64dec466c" responder_term: 11 status { last_received { term: 11 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 11 index: 6 } }
W20260709 13:19:00.676818 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 313532us
W20260709 13:19:00.675212 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 228654us
W20260709 13:19:00.675217 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84562.9us
I20260709 13:19:00.678612 10054 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:00.691468 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:00.687963 10126 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
W20260709 13:19:00.705953 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 197302us
W20260709 13:19:00.706926 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 332030us
I20260709 13:19:00.705842 15604 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:19:00.671067 15707 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:19:00.706379 15705 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:00.723443 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52864.5us
I20260709 13:19:00.730837 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.732501 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:00.732755 15734 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:00.733409 15734 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:00.733628 15654 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:00.733875 15734 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:00.744733 15705 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:00.746132 15705 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.762544 15482 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:00.763230 15482 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:00.764374 15482 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.765864 15482 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.772228 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:00.773329 15708 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:00.776690 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:00.777470 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.778676 15708 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:00.779474 15708 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.782094 15708 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:00.782924 15636 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:00.783861 15654 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:00.784708 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.785851 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:00.788975 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.774916 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:00.792722 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79902.3us
I20260709 13:19:00.791072 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.794788 15707 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:00.795518 15707 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:00.796559 15707 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.797333 15369 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:00.798699 15604 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:00.799244 15604 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:00.800423 15604 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.792904 15664 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.818077 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.830185 15412 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:00.830075 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55282.9us
I20260709 13:19:00.832170 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:00.835441 10095 leader_election.cc:400] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:00.835842 10095 leader_election.cc:403] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:00.836934 15450 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:00.837801 15450 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:00.840655 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:00.841820 15412 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:00.858675 15604 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.821264 15708 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:00.878520 15708 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:00.870682 15369 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:19:00.892694 15708 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.864574 15450 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:00.900624 15450 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:00.860738 15444 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:19:00.915668 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.921131 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.923681 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.918804 15450 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.907334 15412 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:19:00.932271 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:00.925767 15731 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:19:00.950354 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80753.5us
I20260709 13:19:00.957594 15734 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:00.958345  9954 catalog_manager.cc:5697] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:00.968392 15651 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:00.969332 15651 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:00.969769 15651 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.970219 15651 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.971643 15645 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:00.972343 15645 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:00.973433 15645 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.975312 15731 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:00.953887 15661 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:00.976125 15661 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.976507 15661 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:00.976428 15731 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:00.999658 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:01.007825 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96840.7us
I20260709 13:19:00.992702 15705 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:01.026218 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74157.4us
I20260709 13:19:01.038437 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:01.041054  9954 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.037182 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:01.044827 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:01.052594  9950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 220) took 72688 ms (client timeout 15000 ms). Trace:
W20260709 13:19:01.053345  9950 rpcz_store.cc:269] 0709 13:17:48.364061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:48.364403 (+   342us) service_pool.cc:224] Handling call
0709 13:19:01.052322 (+72687919us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:55.453058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:55.453698 (+   640us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:55.453742 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:18:55.453761 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:55.454568 (+   807us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:55.454634 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:55.454668 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:18:55.592788 (+138120us) tablet.cc:701] Row locks acquired
0709 13:18:55.598097 (+  5309us) write_op.cc:260] PREPARE: finished
0709 13:18:55.628712 (+ 30615us) write_op.cc:270] Start()
0709 13:18:55.638003 (+  9291us) write_op.cc:276] Timestamp: P: 1783603135628620 usec, L: 0
0709 13:18:55.658289 (+ 20286us) op_driver.cc:348] REPLICATION: starting
0709 13:18:55.714130 (+ 55841us) log.cc:844] Serialized 2079 byte log entry
0709 13:18:57.033156 (+1319026us) op_driver.cc:464] REPLICATION: finished
0709 13:18:57.605555 (+572399us) write_op.cc:301] APPLY: starting
0709 13:18:57.612069 (+  6514us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:57.652114 (+ 40045us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:57.652236 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:57.656712 (+  4476us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:57.657004 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:18:57.657068 (+    64us) write_op.cc:312] APPLY: finished
0709 13:18:57.661076 (+  4008us) log.cc:844] Serialized 57 byte log entry
0709 13:18:57.724076 (+ 63000us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:57.733182 (+  9106us) write_op.cc:454] Released schema lock
0709 13:18:57.738375 (+  5193us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:57.761361 (+ 22986us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:57.761392 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":174134,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":486950,"apply.run_cpu_time_us":31647,"apply.run_wall_time_us":159550,"cfile_cache_hit":22,"cfile_cache_hit_bytes":24987,"num_ops":5,"prepare.queue_time_us":314659,"prepare.run_cpu_time_us":15423,"prepare.run_wall_time_us":423322,"raft.queue_time_us":1025622,"raft.run_cpu_time_us":72923,"raft.run_wall_time_us":357236,"replication_time_us":1374638,"thread_start_us":823902,"threads_started":5,"wal-append.queue_time_us":331133,"wal-append.run_cpu_time_us":82868,"wal-append.run_wall_time_us":2313694}]]}
W20260709 13:19:01.069000 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.070397 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.035019 15412 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:01.025445 15654 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:01.079478 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:01.038312 15708 raft_consensus.cc:2764] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:19:01.082690 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66115.5us
W20260709 13:19:01.085593 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 86055.8us
W20260709 13:19:01.106582 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89087.9us
I20260709 13:19:01.038039 15705 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:01.116171 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.092005  9954 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 10, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.080827 15412 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:01.119199 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.087389 15733 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:01.098661 15645 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:01.101425 15633 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.084709 15444 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:19:01.099373 15747 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:01.159897 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.134145 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.087177 15749 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:19:01.087393 15654 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:01.170399 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 125652us
I20260709 13:19:01.173440 10127 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:01.125393 15733 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.177857  9954 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.122582 15369 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.132071 15645 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:01.118069 15708 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:19:01.175072 15482 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:01.193661 15444 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:01.195468 10020 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:01.196255 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77201.3us
I20260709 13:19:01.197381 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.198516 10131 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:01.198995 15547 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:19:01.176674 15754 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:01.204521 10130 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:01.181217 15716 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:01.206825  9954 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 11 to 12, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.183543 15734 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:01.183805 15733 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:01.189065 15645 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.213970 10129 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:01.184139 15751 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:01.175823 15666 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:01.203035 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.226188 10127 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:01.204474 15737 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:01.207676 15482 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.283299 15737 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:01.284348 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 105779us
I20260709 13:19:01.211228 15667 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:01.227865  9954 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 9, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.176118 15747 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.179652 15633 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.190092 15369 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.297093  9954 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 9, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.227954 15755 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.221009 15754 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:01.327317 10127 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:01.331424 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.296064 15667 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:01.339038 15750 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:19:01.339771 15737 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.344081 15763 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:01.347812 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:01.352279  9954 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 8, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.362159 10201 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:01.206682 15412 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.221524 15716 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.409132 10127 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:01.309458 15444 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:19:01.398061 15667 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:01.412317 15751 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:01.417603  9954 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.427603 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:01.425225 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:01.428383 10201 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:19:01.435335 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87624.8us
W20260709 13:19:01.444275 10095 connection.cc:256] RPC call timeout handler was delayed by -2.36734979799985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:19:01.457028 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.333207 15755 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.403645 15651 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.467154 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:01.467262 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.416796 15746 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:01.476447 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:01.461364 15383 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:01.484473 10201 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:01.427095 15749 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:19:01.487387 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79291.4us
I20260709 13:19:01.489156 10123 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:19:01.523661 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.476563 15735 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:01.528427 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 103307us
I20260709 13:19:01.524823 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:01.525632 10895 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:19:01.529716 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:01.535717 10894 raft_consensus.cc:2410] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:01.527855  9954 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.453064 15661 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.538031 10202 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:01.539770 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.542562 15760 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.542963 15770 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:01.543614 15770 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:01.543589 15760 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.544402 15770 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:01.544696 15761 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:19:01.542595 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:01.547956 10899 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:01.549027 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:01.519393 15383 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:01.550145 15654 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.550616 15751 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:01.552448 15759 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:19:01.547784 15769 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:01.547261 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 85231us
I20260709 13:19:01.548808 15772 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:01.524583 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:01.563496 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:01.565554 15383 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:01.568113 15769 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.532037 15735 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.570494 15735 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:01.571900 10126 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:01.577457  9954 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.579782 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.583765 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.555804 10900 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:19:01.544287 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:01.591512 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:01.594596 10898 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:01.601747 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:01.602988 15755 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.604749 15716 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:01.605655 15716 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:01.605894 10131 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:01.607076 15747 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.607406 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.608728 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.609412 10129 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:01.609798 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.611102 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.612332 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.612936 10126 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:01.612979 15760 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.524279 15766 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:01.613890 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.614495 10126 raft_consensus.cc:2376] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13. Re-sending same reply.
I20260709 13:19:01.615327 10131 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:01.615468 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.616276 10124 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:01.616508 15772 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:01.617163 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.617856 10124 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:01.619408 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.620343 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.621598 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.622359 10128 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:01.623507 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.624588 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.625741 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.626410 10128 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:01.618790 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.547858 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:01.628088 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.628188 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.629544 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.630658 10124 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:01.630019 10899 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:01.466773 15666 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:01.558698 15482 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.643437 10892 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:01.644116 10130 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:01.649732 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.650858 10124 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:01.653015 10196 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:01.663637  9954 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 10 to 11, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.608757 15651 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.668264 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.668416 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.669312 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.698719 10126 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:01.699281 10124 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13. Re-sending same reply.
I20260709 13:19:01.700568 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:01.698812 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 135454us
I20260709 13:19:01.679917 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.576834 15769 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:01.703462 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.704344 10128 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:01.705541 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.699529  9954 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 9 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 10 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:01.708743 10124 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:01.714598 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:01.715749 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 512772us
I20260709 13:19:01.717912  9954 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.720504 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.721652 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.720597 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:01.667747 15747 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.725721 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.697988 15772 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:01.728257 10131 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:19:01.647763 15666 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:01.741183 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.702834 15633 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:01.714201 15369 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:01.739619 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:01.739283 10131 pending_rounds.cc:85] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
W20260709 13:19:01.750259 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 453352us
I20260709 13:19:01.750638 10131 pending_rounds.cc:107] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:19:01.751313 10131 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.752777 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:01.757332 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 77593.2us
I20260709 13:19:01.760210 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.736937 15755 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:01.762507 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.762904 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:01.763414  9954 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.765410 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:01.767323 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:01.769155 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 236753us
I20260709 13:19:01.770686 10128 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:01.795933 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:01.797686 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 72084.9us
I20260709 13:19:01.476370 15754 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:01.804171 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71957.1us
I20260709 13:19:01.752919 15766 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:01.798883 10126 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:19:01.811234 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:01.754912 15716 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:01.811168  9954 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 9 to 10, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:19:01.762573 15708 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:01.821146 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:01.824035 15767 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:19:01.828796 15758 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:01.830789 15761 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:01.739223 15747 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.743345 15772 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:01.912509 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.818825 15765 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:19:01.902432 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 81926.4us
I20260709 13:19:01.839872 10126 pending_rounds.cc:85] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 12
I20260709 13:19:01.908221 10895 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:01.910146 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:01.895114 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49508.8us
I20260709 13:19:01.914256  9954 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:19:02.002239 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.926101 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 184132us
I20260709 13:19:01.991357 10892 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:01.991874 10203 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:02.017617 10126 pending_rounds.cc:107] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.13
W20260709 13:19:02.020097 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.020298 10126 raft_consensus.cc:2889] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.986737 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:01.986856 10896 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:01.831494 15666 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.038906 10202 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:02.044665 10122 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:01.917909 15751 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.019538 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:02.000674 15667 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.007s
I20260709 13:19:02.063247 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:02.022941 15766 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.986423 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:02.070483 10897 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:19:01.908221 15758 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:02.035255 15708 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:02.039909 15661 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:02.021514 15604 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:19:02.106921 10203 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:02.086683 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:02.107239 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:02.043692 15767 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:02.130089 10126 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:02.088748 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:02.105702 10895 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:19:02.086282 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 559756us
I20260709 13:19:02.086993  9954 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:19:02.130860 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.073553 15733 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.782s	user 0.008s	sys 0.005s
I20260709 13:19:02.136085 10122 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
W20260709 13:19:02.088919 15664 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.239s	user 0.009s	sys 0.008s
I20260709 13:19:02.065056 15769 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.149497 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.167836 10891 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.004s
W20260709 13:19:02.107420 15784 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:19:02.135488 15767 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:02.259079 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:02.179350 15645 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.956s	user 0.016s	sys 0.000s
W20260709 13:19:02.106696 15783 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 13:19:02.218976 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:02.265861 10202 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:02.084782 15716 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:02.225044 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:02.214527  9954 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.222204 15749 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:02.278678 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.249434 10891 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
W20260709 13:19:02.250707 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77218.3us
I20260709 13:19:02.172101 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:02.252662 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:02.250634 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:02.083940 15667 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.127139 15469 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.168s
I20260709 13:19:02.106933 15779 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.245s
I20260709 13:19:02.084036 15763 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 13:19:02.086529 15766 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.230000 15788 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:02.173243 15733 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.258108 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:02.285941 10126 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:02.287142 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.245936 10129 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
W20260709 13:19:02.293076 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 121087us
I20260709 13:19:02.335628 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:02.326892 10199 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:02.416054 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2667) took 1197 ms. Trace:
I20260709 13:19:02.432557 10895 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:02.331692 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:02.301173  9954 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 10, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:19:02.435822 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 177871us
I20260709 13:19:02.439561 10204 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:02.440578 10129 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:02.325693 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:02.437554 10123 rpcz_store.cc:276] 0709 13:19:01.218530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.222794 (+  4264us) service_pool.cc:224] Handling call
0709 13:19:01.285393 (+ 62599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.521472 (+236079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.415557 (+894085us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b540023b4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:02.415938 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467809024}
W20260709 13:19:02.317772 10865 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.965s (SENT)
I20260709 13:19:02.446496 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.452201 10202 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:02.453500 15746 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.458004 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:02.309425 15780 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:02.262676 15765 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:02.335876 15666 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:02.318464 15450 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.375s	user 0.013s	sys 0.001s
I20260709 13:19:02.475768 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.276760 15749 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:02.405483 15661 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:02.301055  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:02.308210 15781 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:02.288666 15664 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:02.307816 15782 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:02.416076 15747 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:02.265282 15604 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.196s
I20260709 13:19:02.319780 15772 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:02.418426 15705 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:02.334124 15795 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:02.331488 15667 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:02.422039 15757 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:02.502871 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.413903 15412 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:02.531484 10054 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:02.455854 15779 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:02.507992 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74370.1us
I20260709 13:19:02.506932 15661 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.261540 15769 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.315344 15777 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.211s
W20260709 13:19:02.516153 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 198419us
W20260709 13:19:02.476596 15735 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.905s	user 0.014s	sys 0.000s
I20260709 13:19:02.486528 10196 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:02.540864 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.289868 15716 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.272558 15645 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.522689 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:02.542521 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:02.534237 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:02.503990 10049 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:02.538324 10052 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:02.475485 15636 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:19:02.323800 15708 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.487805 15746 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.327368 15767 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.533843 15779 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.530658 15757 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:02.563917 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2666) took 1368 ms. Trace:
I20260709 13:19:02.513242 15450 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.544512 15661 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:02.544020 15705 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:02.539980 15772 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:02.565793 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 107916us
I20260709 13:19:02.566448 10203 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:02.567775 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 548718us
I20260709 13:19:02.570542 10125 rpcz_store.cc:276] 0709 13:19:01.195731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.200586 (+  4855us) service_pool.cc:224] Handling call
0709 13:19:01.219929 (+ 19343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.421549 (+201620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.421614 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.427310 (+  5696us) log.cc:844] Serialized 26 byte log entry
0709 13:19:01.727215 (+299905us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:01.727297 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.727420 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:02.563643 (+836223us) raft_consensus.cc:1685] finished
0709 13:19:02.563671 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.563844 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":177749,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10838,"thread_start_us":387022,"threads_started":2,"wal-append.queue_time_us":609791}
I20260709 13:19:02.526834 15782 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.525663 15664 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.600112 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:02.578243 15770 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.033s	user 0.017s	sys 0.000s
I20260709 13:19:02.601922 15770 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.603312 10130 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.551453 15735 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.603914 15733 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:02.608000 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:02.609208 15796 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.609941 15796 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.612339 15772 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.583868 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:02.567077 15734 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.590006 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.594609 15801 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:02.599277 15716 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:02.571185 15779 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.587184 15765 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.539011 15795 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.623653 15757 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.624645 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.569829 15749 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.451557 15788 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:02.627358 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.579401 15633 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:02.587613 15705 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:02.625458 10051 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:02.662076 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:02.663527 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:02.648291 10130 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:02.571836 15754 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.687252 10056 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:02.617986 15798 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.691282 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.653926 15705 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:02.724529 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.748981 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.627583 15734 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.695417 15633 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:02.833949 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.840139 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:02.843917 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 236029us
I20260709 13:19:02.849936 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:02.836244 10055 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:02.858194 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:02.627738 15801 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:02.870787 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.854663 15369 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:19:02.877620 15782 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:02.885987 10055 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:02.705371 15754 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:02.890009 10055 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:02.887573 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.852902 15757 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:02.893195 10056 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:02.892684 10055 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:02.893922 10055 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:02.895102 10049 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:02.850090 15795 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:02.706730 15798 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:02.912060 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 54021.6us
I20260709 13:19:02.891443 15733 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.908983 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:02.924561 10049 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:02.897130 15369 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.910051 15810 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:02.906960 15782 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.915441 15787 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.918668 15755 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:02.950194 15772 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:02.991288 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.961769 15810 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:02.996971 15810 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:02.953723 15716 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:19:02.967156 15769 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:02.960009 15633 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.015230 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:03.021481 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68308.7us
I20260709 13:19:03.001211 15735 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:03.042544 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:03.037747 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:02.992504 15751 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.917s	user 0.015s	sys 0.000s
I20260709 13:19:03.048084 10896 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:02.957896 15369 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.048940 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:03.049683 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.003s	sys 0.006s
I20260709 13:19:02.971827 15766 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:03.052398 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.003s	sys 0.006s
I20260709 13:19:03.052464 10051 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:03.053231 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.054626 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.055868 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:03.056878 10899 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:03.060787 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.028524 15810 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:03.069612 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:02.986452 15782 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.070926 10202 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:03.050314 15716 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:19:03.063235 15813 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.060544 15707 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:02.991375 15755 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.004509 15788 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:03.064985 15811 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:19:03.065918 10203 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:03.066041 15801 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.066923 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.003266 15746 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.061399 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:03.071082 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:03.042052 15781 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.082665 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:03.083653 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46772.7us
I20260709 13:19:03.007061 15450 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:02.986812 15787 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.059870 15751 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.097405 10051 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:03.100875 10895 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:19:03.059464 15816 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.111538 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.112766 15808 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.113799 15746 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:03.114657 15792 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:03.050597 15769 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:19:03.116448 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:03.116832 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.082617 15754 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.117266 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:19:03.134770 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.135013 15816 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.135910 10197 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:03.137909 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.138152 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:03.138851 10204 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
W20260709 13:19:03.139068 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 84357.4us
I20260709 13:19:03.138942 10205 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.156348 10202 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:03.156494 10203 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:03.156904 10199 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:03.136601 15705 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:03.159885 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43624.1us
I20260709 13:19:03.161099 10198 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:03.162699 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.165017 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:03.171075 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.174485 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:03.174970 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.176908 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.098784 15766 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:19:03.181625 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.184139 10202 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:19:03.182606 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.177863 10200 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:19:03.162742 15661 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.002s
I20260709 13:19:03.156512 15801 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.134791 15798 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.155583 15667 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.194546 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:03.190827 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 82305.5us
I20260709 13:19:03.115317 15792 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.196378 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.199474 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.177309 15450 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:03.201122 10202 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:03.177438 15751 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.202000 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:03.205273 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.205933 15547 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:03.206398 15547 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:03.207919 15770 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.195050 15661 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.208943 15770 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.197224 15788 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:19:03.213079 15824 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:03.215404 15412 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.005s
I20260709 13:19:03.216635 10127 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:03.216291 15733 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:03.218714 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3398) took 1634 ms (client timeout 1538 ms). Trace:
I20260709 13:19:03.219210 15795 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.220711 15828 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:03.223188 15792 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.110316 15781 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.227365 15800 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.228896 15799 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.229801 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:03.203276 15798 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:19:03.190222 15746 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:03.198341 15801 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:03.231976 15823 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:03.096616 15813 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.261631 15801 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.262764 15813 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:03.235951 10127 rpcz_store.cc:269] 0709 13:19:01.583883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.585074 (+  1191us) service_pool.cc:224] Handling call
0709 13:19:03.218393 (+1633319us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:03.218584 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3670947584}
I20260709 13:19:03.263477 15737 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:03.253180 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.265640 15781 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.266240 10895 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:03.266782 15450 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:03.263819 10126 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:03.267410 15746 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:03.267518 15450 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:03.268040 15746 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:03.269866 15746 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.270337 15705 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.274230 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.255975 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.276561 10899 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 9: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:03.280362 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.281147 10202 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:03.280081 10122 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:03.283700 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:03.272610 15746 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:03.287981 15764 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.288724 15764 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.293344 15782 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.293620 15444 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.247917 15651 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.294368 15444 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.294951 15651 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.296900 15444 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.297757 15651 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.284648 15792 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:03.300130 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.299000 15792 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:03.331152 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 135256us
W20260709 13:19:03.332036 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73099.6us
I20260709 13:19:03.333463 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:03.335420 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:03.279732 15667 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.278311 15816 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.344748 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:03.344759 10205 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:03.350268 15444 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:03.351068 15444 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:03.352247 10894 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:03.352398 15444 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.343389 15795 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:19:03.361097 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.362474 10205 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:03.362560 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:03.363507 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.365298 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.366698 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.368479 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.285463 15733 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
W20260709 13:19:03.369329 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.361101 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2188d855354542c984f249b94175c660"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:03.372362 10202 raft_consensus.cc:606] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.340626 15651 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:03.374939 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.375329 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.395676 10895 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:03.379149 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.379724 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.381050 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:03.381884 15775 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:03.384757 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.385707 15786 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:19:03.268932 15450 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.389981 10055 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.263656 15799 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.274834 15800 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.392994 15734 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.399271 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:03.413466 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 254345us
I20260709 13:19:03.419317 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:03.419620 15733 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.428284 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:03.441342 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:03.442309 10202 raft_consensus.cc:2368] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:03.442201 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:03.431782 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52773.7us
I20260709 13:19:03.359325 15667 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.395020 15651 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.443156 10200 raft_consensus.cc:2368] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18.
I20260709 13:19:03.435039 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:03.365661 15795 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:19:03.401589 15775 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:03.446664 10201 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:03.448688 10055 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.478721 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:03.489413 10051 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:03.449684 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 87203.8us
I20260709 13:19:03.448693 15787 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:03.491215 15666 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.969s	user 0.016s	sys 0.000s
I20260709 13:19:03.403084 15769 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:19:03.491834 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.492435 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.447053 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:03.447822 15734 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:03.418015 15777 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:19:03.515601 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.518976 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:03.518154 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92500.8us
W20260709 13:19:03.459700 15757 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 13:19:03.515033 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:03.524060 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 79321.8us
W20260709 13:19:03.529655 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 132486us
I20260709 13:19:03.561690 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:03.460714 15651 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.473019 15768 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.441746 15733 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.591358 10202 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:19:03.592317 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69453.4us
I20260709 13:19:03.583576 15768 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:03.565827 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75136.9us
W20260709 13:19:03.577304 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 174006us
I20260709 13:19:03.577998 10052 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:03.520401 15734 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.584316 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:03.576995 10197 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:03.579447 10198 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:03.588698 10053 pending_rounds.cc:85] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:19:03.597649 10053 pending_rounds.cc:107] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:19:03.597961 10053 raft_consensus.cc:2889] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:03.562559 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 115700us
I20260709 13:19:03.472520 15803 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:03.590238 10051 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:03.565546 15666 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:03.600509 10865 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:03.606034 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76767.4us
I20260709 13:19:03.647326 15809 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.648260 15799 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.711359 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.606459 10894 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:03.604715 15734 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:03.718073 15765 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.821193 10047 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:03.718677 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 122022us
I20260709 13:19:03.824124 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.776655 15795 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:03.719769 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 129694us
I20260709 13:19:03.722213 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:03.575618 15775 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.596585 15768 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:03.769305 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.778987 10055 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:19:03.764945 15779 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.142s	user 0.012s	sys 0.004s
W20260709 13:19:03.783862 15838 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:19:03.603440 15444 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.172s
W20260709 13:19:03.788152 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.592430 15733 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:03.788719 15746 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
I20260709 13:19:03.473344 15834 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
W20260709 13:19:03.792502 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 192079us
I20260709 13:19:03.917363 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:03.719277 15802 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:03.781430 15792 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.831537 10895 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:03.797163 15666 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:03.831724 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:03.942282 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 110638us
I20260709 13:19:03.790325 15798 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.795949 15809 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.825843 15705 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.839504 15799 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.825392 15765 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.831645 15791 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.946197 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.928418 15775 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:03.951435 15831 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.525s	user 0.000s	sys 0.002s
I20260709 13:19:03.949420 15666 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:03.941383 15746 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.835016 15795 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.819005 15821 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 13:19:03.919952 15779 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:03.955528 15705 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:03.955214 15792 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:03.955780 15809 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.968014 15667 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:03.994453 15837 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 13:19:04.012741 15853 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.013425 15853 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.008992 15795 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:04.016412 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.982748 15791 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:04.022549 15842 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:19:03.995626 15705 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:03.996366 15792 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.069967 15792 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.025372 15664 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:03.990329 15746 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.028916 15796 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:03.953811 15798 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.032728 15779 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.037220 15765 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.037637 15769 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.039233 15753 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.053095 15758 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:03.983764 15799 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:04.002791 15835 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 13:19:04.065982 15734 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.054564 15791 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.130630 15746 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.151470 15645 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.079159 15651 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:04.079166 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161864us
I20260709 13:19:04.166116 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:04.168673 15855 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:04.169222 15855 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:04.169615 15855 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.172084 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.170097 15768 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:04.181308 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.141773 15753 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.176327 15758 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:04.188301 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 649) took 2278 ms (client timeout 2151 ms). Trace:
W20260709 13:19:04.188858 10128 rpcz_store.cc:269] 0709 13:19:01.909758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.912152 (+  2394us) service_pool.cc:224] Handling call
0709 13:19:04.186171 (+2274019us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002efc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.188165 (+  1994us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3065143552}
I20260709 13:19:04.190199 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.190927 10128 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:04.191006 15799 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.164280 15666 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.192795 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.193396 15753 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.192785 15734 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.194494 15798 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.195400 15853 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.195530 15645 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:04.196080 15645 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.196455 15645 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.200441 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3352) took 1067 ms. Trace:
I20260709 13:19:04.201023 15768 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.194904 15666 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.205245 15733 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.214200 15734 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:04.217765 15734 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.154899 15792 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.218170 15734 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.202559 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 712) took 1692 ms. Trace:
I20260709 13:19:04.221078 10048 rpcz_store.cc:276] 0709 13:19:02.509802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.509948 (+   146us) service_pool.cc:224] Handling call
0709 13:19:02.510362 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.570858 (+ 60496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.570895 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.578369 (+  7474us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:02.578431 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.202301 (+1623870us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:04.202348 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.202499 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122026880}
I20260709 13:19:04.214386 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:04.200960 10199 rpcz_store.cc:276] 0709 13:19:03.132920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.133087 (+   167us) service_pool.cc:224] Handling call
0709 13:19:03.137423 (+  4336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.157356 (+ 19933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.200193 (+1042837us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b54003548d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:04.200375 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43875712}
I20260709 13:19:04.233253 15755 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.235422 15369 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.217101 15777 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.236426 15777 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.239065 15633 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:04.239886 15633 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.240954 15633 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.225732 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:04.248368 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 243636us
I20260709 13:19:04.249082 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.215334 15803 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:19:04.252473 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.258036 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.259219 10128 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.230464 15792 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.262764 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.264160 15803 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:04.264868 15547 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.266058 15547 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.264312 15758 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:04.267951 15758 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.269341 15733 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.268976 15758 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.269701 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.270448 15758 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.272470 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 728) took 1103 ms. Trace:
I20260709 13:19:04.272714 15705 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.273097 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:04.274256 10895 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:04.274787 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:04.275982 15792 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:04.275151 15777 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.277727 15737 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:04.278210 15737 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.270758 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2678) took 1037 ms. Trace:
I20260709 13:19:04.207929 15855 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.269735 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2680) took 1035 ms. Trace:
I20260709 13:19:04.274156 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.282490 10126 rpcz_store.cc:276] 0709 13:19:03.234056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.246611 (+ 12555us) service_pool.cc:224] Handling call
0709 13:19:03.246989 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.267456 (+ 20467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.269425 (+1001969us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:04.269649 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38152064}
W20260709 13:19:04.287751 10196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2164) took 2380 ms (client timeout 1867 ms). Trace:
I20260709 13:19:04.273000 10197 rpcz_store.cc:276] 0709 13:19:03.169190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.186927 (+ 17737us) service_pool.cc:224] Handling call
0709 13:19:04.272227 (+1085300us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400228b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.272378 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856621952}
W20260709 13:19:04.291216 10196 rpcz_store.cc:269] 0709 13:19:01.907174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.907319 (+   145us) service_pool.cc:224] Handling call
0709 13:19:04.287402 (+2380083us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.287581 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359787264}
I20260709 13:19:04.291949 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.281581 10129 rpcz_store.cc:276] 0709 13:19:03.233429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.247735 (+ 14306us) service_pool.cc:224] Handling call
0709 13:19:03.262718 (+ 14983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:03.298666 (+ 35948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:03.298710 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:03.299380 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:19:03.423971 (+124591us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:03.424024 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.270316 (+846292us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:04.270368 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.270426 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:04.270440 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.270692 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":313,"raft.queue_time_us":7472,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":23174,"spinlock_wait_cycles":81387776,"thread_start_us":124130,"threads_started":1,"wal-append.queue_time_us":265743}
I20260709 13:19:04.296080 15651 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.298044 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.298997 10123 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.281894 15737 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.301108 15737 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:04.302995 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.303407 15383 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:04.304001 10124 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.308461 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.310777 10053 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:19:04.310987 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.311591 10053 pending_rounds.cc:85] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:19:04.311966 10053 pending_rounds.cc:107] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:19:04.312230 10053 raft_consensus.cc:2889] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.309262 10048 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.309196 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.313787 10050 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:04.314128 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.314833 10123 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.316721 15841 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:04.317299 15841 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.317848 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.318571 10054 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:04.320370 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.320490 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.320812 10053 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:04.321038 15768 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.321455 10053 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:04.321578 15810 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:04.322216 15810 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.322602 15810 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.324456 15705 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.318300 15841 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.326397 15841 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.329347 10050 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:04.333878 10128 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.334656 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2690) took 1056 ms. Trace:
I20260709 13:19:04.335278 10128 rpcz_store.cc:276] 0709 13:19:03.278003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.257587 (+979584us) service_pool.cc:224] Handling call
0709 13:19:04.334570 (+ 76983us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.337649 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:04.339967 10891 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:04.322410 10053 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.342798 10051 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:04.344740 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.348307 10053 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:04.348866 10124 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.349825 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.350509 10051 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.353609 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.357156 10123 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:04.363708 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.364895 10050 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.369967 10894 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:04.347929 15792 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.390803 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.402601 10128 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:04.403282 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.382474 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.321669 15768 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.414929 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.416193 10123 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:04.416182 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63057.3us
I20260709 13:19:04.419978 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.425051 10051 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.421788 10056 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:04.437048 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.438585 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 730) took 1234 ms. Trace:
W20260709 13:19:04.439626 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 643) took 2681 ms (client timeout 1820 ms). Trace:
I20260709 13:19:04.414770 15705 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:04.328675 15809 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.442947 10048 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.437093 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.440565 10131 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:04.445254 10130 rpcz_store.cc:269] 0709 13:19:01.757784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.757914 (+   130us) service_pool.cc:224] Handling call
0709 13:19:04.439382 (+2681468us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.439542 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931011968}
I20260709 13:19:04.453676 10123 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:04.449002 10128 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:04.449538 10053 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:19:04.427896 15792 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.444965 10198 rpcz_store.cc:276] 0709 13:19:03.204093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.204267 (+   174us) service_pool.cc:224] Handling call
0709 13:19:04.438330 (+1234063us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.438501 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858263424}
I20260709 13:19:04.455072 10053 pending_rounds.cc:85] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:19:04.410970 15755 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:19:04.456833 10050 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:04.457568 10053 pending_rounds.cc:107] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:19:04.459131 10053 raft_consensus.cc:2889] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.459908 10899 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:04.426934 15444 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:19:04.460495 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.463227 15829 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.463402 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.463943 15829 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.464594 15852 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:04.465163 15852 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.465529 15852 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.468248 15829 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.475539 15707 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:04.476542 15707 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:04.477851 15707 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.461977 10130 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:04.480046 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2683) took 1245 ms. Trace:
I20260709 13:19:04.480577 10130 rpcz_store.cc:276] 0709 13:19:03.234958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.456528 (+1221570us) service_pool.cc:224] Handling call
0709 13:19:04.461824 (+  5296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.479609 (+ 17785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.479958 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.440259 15664 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:19:04.490700 10896 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:04.455778 15705 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:04.439917 15734 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.497097 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.520228 15768 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.522097 15809 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.528038 10898 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:04.527554 15792 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.530205 15734 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.511252 15816 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:04.552242 15645 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.553233 15705 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.559201 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.561005 15826 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:04.563410 15863 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:19:04.515589 15756 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:04.570936 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:04.527582 15855 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:04.578311 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144665us
I20260709 13:19:04.582784 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:04.583472 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.584795 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.588683 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:04.596159 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.596930 15772 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:19:04.597146 15737 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:04.589304 15547 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:04.597604 15737 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:04.583276 15826 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:04.598066 15737 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.578193 15791 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:19:04.576330 15810 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.586326 15787 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.612394 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2687) took 1362 ms. Trace:
I20260709 13:19:04.613267 10129 rpcz_store.cc:276] 0709 13:19:03.248706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.595685 (+1346979us) service_pool.cc:224] Handling call
0709 13:19:04.610852 (+ 15167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.611061 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.611092 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.611188 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:04.611238 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.611318 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.611478 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.593362 15633 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:04.631275 15810 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.633455 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:04.649791 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:04.651484 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2686) took 1408 ms. Trace:
I20260709 13:19:04.652655 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:04.652033 10127 rpcz_store.cc:276] 0709 13:19:03.242746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.584993 (+1342247us) service_pool.cc:224] Handling call
0709 13:19:04.591839 (+  6846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.592110 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:04.609403 (+ 17293us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.651011 (+ 41608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.651076 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:04.651147 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.651249 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.651401 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29694,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"thread_start_us":40727,"threads_started":1}
I20260709 13:19:04.634146 15633 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.653364 10129 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:04.659762 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.663688 15661 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:04.664290 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2157) took 3129 ms (client timeout 2172 ms). Trace:
W20260709 13:19:04.664753 10201 rpcz_store.cc:269] 0709 13:19:01.535551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.536696 (+  1145us) service_pool.cc:224] Handling call
0709 13:19:04.664040 (+3127344us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b54001883d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.664202 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4145161088}
I20260709 13:19:04.667589 15444 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.668309 15444 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.621726 15758 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:19:04.673911 10129 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:04.678349 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:04.679157 10201 raft_consensus.cc:606] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.659973 15861 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:04.654114 15787 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.680902 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.681684 10129 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:04.682617 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:04.684590 10050 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:04.660177 15633 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:04.684680 10048 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:04.686728 15874 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:19:04.669442 15444 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.687492 15787 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.691277 15861 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.691797 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.692463 10053 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:04.692409 15861 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.692962 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:04.696518 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 647) took 2882 ms. Trace:
I20260709 13:19:04.697326 10095 leader_election.cc:400] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:04.697762 10095 leader_election.cc:403] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:04.697992 15782 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:04.698078 15756 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:04.698719 15782 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:04.699960 15756 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.699566 15737 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.699703 15782 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.701033 15782 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.702219 15737 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:04.705266 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.706032 10126 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:04.707883 10895 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:04.704031 10095 leader_election.cc:400] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18.
I20260709 13:19:04.708635 10095 leader_election.cc:403] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:04.709789 15754 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18.
I20260709 13:19:04.709837 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:04.710915 15786 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:04.712713 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.713999 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2692) took 1423 ms. Trace:
I20260709 13:19:04.714535 10126 rpcz_store.cc:276] 0709 13:19:03.283790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.705311 (+1421521us) service_pool.cc:224] Handling call
0709 13:19:04.705783 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.706859 (+  1076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.707090 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:04.716311 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.830s (SENT) [suppressed 40 similar messages]
I20260709 13:19:04.716806 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2693) took 1417 ms. Trace:
I20260709 13:19:04.717396 10126 rpcz_store.cc:276] 0709 13:19:03.299726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.715586 (+1415860us) service_pool.cc:224] Handling call
0709 13:19:04.716081 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.716393 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.716430 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.716487 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:04.716530 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.716593 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.716724 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.721028 15645 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.704306 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2691) took 1423 ms. Trace:
I20260709 13:19:04.722350 10128 rpcz_store.cc:276] 0709 13:19:03.280749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.694520 (+1413771us) service_pool.cc:224] Handling call
0709 13:19:04.694881 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.695103 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.703867 (+  8764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.703931 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:04.703986 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.704069 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.704228 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.725041 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2695) took 1342 ms. Trace:
I20260709 13:19:04.725504 10128 rpcz_store.cc:276] 0709 13:19:03.382628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.723805 (+1341177us) service_pool.cc:224] Handling call
0709 13:19:04.724313 (+   508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.724583 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.724619 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.724683 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:04.724748 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.724824 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.724967 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.726754 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.727523 15803 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.697039 10131 rpcz_store.cc:276] 0709 13:19:01.814484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.814662 (+   178us) service_pool.cc:224] Handling call
0709 13:19:04.440174 (+2625512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.453619 (+ 13445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.696195 (+242576us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b54003d4d50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:04.696413 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5873469184}
W20260709 13:19:04.706357 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.734285 10126 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:04.735082 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2694) took 1388 ms. Trace:
I20260709 13:19:04.735469 10126 rpcz_store.cc:276] 0709 13:19:03.346247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.718828 (+1372581us) service_pool.cc:224] Handling call
0709 13:19:04.734064 (+ 15236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.734820 (+   756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.735023 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.730722 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2697) took 1347 ms. Trace:
W20260709 13:19:04.737547 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.737185 10128 rpcz_store.cc:276] 0709 13:19:03.383468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.726495 (+1343027us) service_pool.cc:224] Handling call
0709 13:19:04.726810 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.727028 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.727054 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.730196 (+  3142us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:04.730288 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.730410 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.730637 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.655030 15826 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.744156 15776 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.748481 15776 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.731639 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:04.745100 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.764739 10053 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:04.744217 15826 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.746038 10198 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:04.746187 10896 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:04.747256 15801 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:04.747180 15803 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.778527 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.749979 15767 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.751988 10122 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:19:04.779542 10122 pending_rounds.cc:85] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
I20260709 13:19:04.779836 10122 pending_rounds.cc:107] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:19:04.780108 10122 raft_consensus.cc:2889] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.730949 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2688) took 1480 ms. Trace:
I20260709 13:19:04.780653 10127 rpcz_store.cc:276] 0709 13:19:03.250580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.673416 (+1422836us) service_pool.cc:224] Handling call
0709 13:19:04.684749 (+ 11333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.685865 (+  1116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.685898 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.730637 (+ 44739us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:04.730715 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.730781 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.730888 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.730618 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2689) took 1476 ms. Trace:
I20260709 13:19:04.757395 10196 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:04.757973 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.781953 10129 rpcz_store.cc:276] 0709 13:19:03.253964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.674749 (+1420785us) service_pool.cc:224] Handling call
0709 13:19:04.675458 (+   709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.685826 (+ 10368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.729421 (+ 43595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.777228 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:04.784125 10048 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:04.784508 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.748863 15852 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.785657 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.779286 15754 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:04.794703 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.776261 15816 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:04.795619 15651 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.796487 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.796322 15651 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.798074 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.800673 15775 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:04.802162 15775 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:04.802588 15775 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.804332 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2698) took 1208 ms. Trace:
I20260709 13:19:04.805766 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.805022 10126 rpcz_store.cc:276] 0709 13:19:03.596362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.768524 (+1172162us) service_pool.cc:224] Handling call
0709 13:19:04.803512 (+ 34988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.803812 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.803852 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.803919 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:04.803979 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.804066 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.804231 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:04.683445 15792 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:19:04.813402 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.805546 10864 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.785185 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.810114 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.812109 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.816768 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.812013 10053 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:04.763154 15797 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:04.823371 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.824040 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.776261 15801 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:04.825470 10197 raft_consensus.cc:2376] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13. Re-sending same reply.
W20260709 13:19:04.827864 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111599us
I20260709 13:19:04.828583 10122 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:04.829411 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:04.836608 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:04.810464 15786 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.819756 15651 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.825139 15705 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:04.827314 15861 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:04.854869 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.836643 15763 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:04.856650 10205 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:04.858590 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.860241 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:04.850847 15787 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:04.866114 15787 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.866463 10130 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:04.824954 15816 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:04.871318 10896 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:04.873708 10048 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:04.873667 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.833142 15797 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.840121 15754 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:19:04.892374 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.893222 10199 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:04.893432 15754 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:04.893968 15754 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:04.894544 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.880379 15853 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:04.895529 15853 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:04.895989 15853 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:04.897559 15764 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.848026 15707 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:19:04.856251 15861 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.904840 15861 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.873641 15651 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:04.907840 15884 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:04.879832 15774 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:04.848825 15786 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.909487 15774 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.872882 15829 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:04.910053 15547 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:04.910593 15547 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:04.911058 15776 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.910663 15774 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.911679 15786 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:04.895026 15754 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.887030 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.887964 15855 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:04.889442 15803 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:04.920773 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:04.919472 10205 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:04.867097 15787 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:04.859867 15763 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.916195 15852 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:04.873243 15801 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:04.941685 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.926364 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.926901 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:04.918694 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:04.955770 10201 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:04.998227 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.000265 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.912235 15774 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:04.888116 15816 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.917246 15756 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:05.005462 10896 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:05.007849 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:04.919740 15749 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:04.918025 15754 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.952319 15763 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:05.028076 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 191551us
I20260709 13:19:04.972855 15855 raft_consensus.cc:2764] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:05.035935 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:05.041188 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.042753 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:05.048899 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.011376 15797 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:05.054486 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:05.057649 10895 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:04.915825 15547 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.073154 15768 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:05.073884 15768 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:05.076726 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.082544 15800 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.048182 15801 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.085668 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 653) took 2287 ms. Trace:
I20260709 13:19:05.085805 15813 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.087910 15813 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.088479 15853 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.070894 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.100616 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:05.101694 10128 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:05.101876 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:05.103269 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.103415 10129 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:05.104516 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:05.105158 10129 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:05.105532 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.105700 10204 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:05.106463 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.107843 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 1.39657e+06us
I20260709 13:19:05.102770 10122 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.109205 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:05.110168 10129 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:05.086700 10125 rpcz_store.cc:276] 0709 13:19:02.798826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.799011 (+   185us) service_pool.cc:224] Handling call
0709 13:19:02.850636 (+ 51625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.963353 (+112717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.963411 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.974232 (+ 10821us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.466930 (+1492698us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b1800465d00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:04.500569 (+ 33639us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:04.506592 (+  6023us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.510886 (+  4294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.085126 (+574240us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b5400321754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:05.085211 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.085289 (+    78us) raft_consensus.cc:1685] finished
0709 13:19:05.085309 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.085546 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39863,"prepare.queue_time_us":4104,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":65,"raft.queue_time_us":40255,"raft.run_cpu_time_us":503,"raft.run_wall_time_us":41810,"spinlock_wait_cycles":224363520,"thread_start_us":38717,"threads_started":2,"wal-append.queue_time_us":1500958}
W20260709 13:19:05.113000 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 267339us
I20260709 13:19:05.022375 15816 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.118245 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2684) took 1883 ms. Trace:
I20260709 13:19:05.123580 10123 rpcz_store.cc:276] 0709 13:19:03.237346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.459911 (+1222565us) service_pool.cc:224] Handling call
0709 13:19:04.493192 (+ 33281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.493745 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.493792 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.512009 (+ 18217us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.914049 (+402040us) spinlock_profiling.cc:243] Waited 8.89 ms on lock 0x7b1800455980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:05.051948 (+137899us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:05.052032 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.052247 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.109084 (+ 56837us) raft_consensus.cc:1685] finished
0709 13:19:05.109136 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.118158 (+  9022us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":229,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":17141,"spinlock_wait_cycles":20458240,"thread_start_us":136431,"threads_started":1,"wal-append.queue_time_us":560847}
I20260709 13:19:05.133035 15890 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.133709 15890 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.134349 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:05.135064 10049 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:05.136142 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:05.136760 10049 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:05.107508 15708 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.141309 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.144001 10201 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:05.144927 10205 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:05.145747 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:05.114620 15844 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.151044 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:05.155961 10128 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:05.158967 10201 raft_consensus.cc:606] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.102730 15775 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.122265 15845 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:05.196614 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.136155 15890 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.124927 15846 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:05.197552 15844 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.155257 15853 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.204169 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:05.208113 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 99463.2us
I20260709 13:19:05.235837 15828 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:05.276338 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.095523 15768 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.282562 15667 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:05.286743 15810 raft_consensus.cc:993] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
W20260709 13:19:05.285996 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73211.3us
I20260709 13:19:05.155468 15708 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.206576 15816 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.201767 15444 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:05.246371 15775 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.227005 15781 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:19:05.295006 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.295744 15889 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:19:05.235921 15861 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.208087 15845 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.303887 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:05.304150 10122 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:05.226872 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.40273e+06us
I20260709 13:19:05.286139 15803 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:19:05.289543 15766 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:19:05.302754 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98661.8us
W20260709 13:19:05.429977 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.473939 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.458599 10203 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:05.421770 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:05.397625 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.493811 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.289490 15772 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:05.423774 10129 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:19:05.407033 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49221.1us
I20260709 13:19:05.494396 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.422804 10127 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:05.432840 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2685) took 2195 ms. Trace:
I20260709 13:19:05.318727 10126 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:05.527769 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:05.544049 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3358) took 2394 ms (client timeout 2083 ms). Trace:
W20260709 13:19:05.413447 15896 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:19:05.532393 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.529577 10124 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.533782 15757 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.534614 15861 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.534658 10199 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:05.492542 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:05.536784 15845 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.289619 15761 raft_consensus.cc:1081] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:05.536897 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2704) took 1397 ms. Trace:
I20260709 13:19:05.383154 15892 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.310017 15768 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.529986 10198 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:19:05.539984 10130 rpcz_store.cc:276] 0709 13:19:03.237672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.511740 (+1274068us) service_pool.cc:224] Handling call
0709 13:19:04.521681 (+  9941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.866262 (+344581us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b540037dd20. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa2780061c0 00007fa278007231 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c
0709 13:19:04.866990 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.432436 (+565446us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b540037dd20. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa278007a02 00007fa278003dd6 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:19:05.432739 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252144640}
W20260709 13:19:05.542618 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 238852us
I20260709 13:19:05.557440 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.383742 15816 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:05.461680 15633 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
W20260709 13:19:05.371646 15893 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:19:05.561317 10202 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:05.482301 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:05.544682 10203 rpcz_store.cc:269] 0709 13:19:03.149368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.162532 (+ 13164us) service_pool.cc:224] Handling call
0709 13:19:05.543816 (+2381284us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54001d0954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:05.543963 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5225396096}
I20260709 13:19:05.295038 15369 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:05.570871 15369 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.571493 15369 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:05.553869 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.464512 15763 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.333189 15776 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.228s
I20260709 13:19:05.538163 15772 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:05.602173 15772 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.605244 15763 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.612540 15708 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.550750 10131 rpcz_store.cc:276] 0709 13:19:04.139923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.752691 (+612768us) service_pool.cc:224] Handling call
0709 13:19:04.753145 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.753616 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.753647 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.754222 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.883070 (+128848us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:04.883148 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.883224 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.536559 (+653335us) raft_consensus.cc:1685] finished
0709 13:19:05.536597 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.536806 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":128397,"threads_started":1,"wal-append.queue_time_us":353042}
I20260709 13:19:05.614728 10204 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:05.578727 15816 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.576172 15705 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:19:05.558768 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:05.616379 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57684us
I20260709 13:19:05.617198 15705 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:05.617465 15801 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:05.617942 15705 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:05.618079 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:05.618294 15828 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:05.619201 15666 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:05.619660 15666 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:05.618119 15801 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:05.620081 15666 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.621059 15801 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.631338 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.633226 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.561427 15469 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.381510 15444 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:05.659462 15469 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.660567 15444 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.548895 15757 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.549742 15733 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.563902 15667 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:05.563962 15853 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:19:05.560000 15766 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:05.561267 15892 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.570312 15802 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:05.428027 15813 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:05.576148 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.676862 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.576145 10124 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:05.481153 15547 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.478008 15855 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
W20260709 13:19:05.594596 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62316us
I20260709 13:19:05.546813 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:05.681455 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 134762us
I20260709 13:19:05.440187 15844 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:05.599723 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45986.8us
I20260709 13:19:05.690141 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.691720 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.694517 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.600472 15761 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:05.637586 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.425410 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:05.695640 15781 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.695883 15844 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:05.696266 15813 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:05.696825 15813 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.696997 15844 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:05.697299 15813 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:05.622519 15705 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.698558 15892 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.640853 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.699666 10052 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:05.701732 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 276458us
I20260709 13:19:05.644012 15816 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:05.566954 15633 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.705765 15853 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:05.674677 15766 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:05.707508 15766 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:05.708118 15633 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.592077 15845 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:05.648451 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:05.709656 10053 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:05.649341 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.711337 10200 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:05.648458 10201 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:05.650292 10047 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:05.651064 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.714367 10203 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:05.651091 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.652276 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:05.653311 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:05.715065 10050 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:05.716195 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.668120 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.683799 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.718379 10202 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:05.695366 15761 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:05.698688 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.674084 15802 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:05.736833 10049 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:05.736218 15802 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.737910 15802 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.641391 15369 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.740007 10197 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:05.741135 15369 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.664939 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.742642 10197 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:05.743398 15469 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.743989 10202 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:05.744510 15757 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.670390 15667 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:05.745353 15667 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:05.747861 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2179) took 2400 ms. Trace:
I20260709 13:19:05.748369 10200 rpcz_store.cc:276] 0709 13:19:03.347009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.567094 (+220085us) service_pool.cc:224] Handling call
0709 13:19:04.664503 (+1097409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:05.711919 (+1047416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.740246 (+ 28327us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001883d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:05.747568 (+  7322us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001883d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:05.747754 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4929580160}
I20260709 13:19:05.758348 10050 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:05.710289 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:05.769704 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59469.2us
I20260709 13:19:05.770074 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.667320 15828 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.666844 15733 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.726217 10056 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:05.775404 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.776408 10056 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:05.726485 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44222.2us
I20260709 13:19:05.778321 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.728447 15761 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.729162 10199 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:19:05.729278 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.782517 10203 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:05.759867 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.732880 15813 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:05.764683 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 85639.5us
I20260709 13:19:05.718096 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:05.771106 10050 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.770844 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:05.767645 15829 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.773587 10053 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:05.789471 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:05.727293 15878 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:05.763942 10054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.044s	sys 0.062s
I20260709 13:19:05.791867 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:05.792620 10049 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:05.793490 15781 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:05.771967 15836 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:05.799351 15816 raft_consensus.cc:2764] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:19:05.773856 15892 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:05.801084 15892 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:05.803371 15829 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.806563 15829 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.782092 15733 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.807572 15878 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.810402 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:05.822417 10053 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:05.811180 10197 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:19:05.791370 10054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.045s	sys 0.063s
W20260709 13:19:05.799552 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100949us
I20260709 13:19:05.825934 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:05.812186 10204 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:05.780452 15469 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:05.767879 15757 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.787698 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:05.833392 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45827.6us
I20260709 13:19:05.774847 15666 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.820869 10198 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:05.836668 10050 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:05.803399 15845 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:05.870571 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44775.5us
I20260709 13:19:05.871974 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:05.873586 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:05.802492 15892 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.820868 15786 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:05.882944 10050 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:05.775101 15667 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.782557 15828 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:05.901180 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65728.4us
I20260709 13:19:05.901510 10053 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:05.903141 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:05.909191 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.668s (SENT)
I20260709 13:19:05.909816 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:05.852145 15839 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:05.864585 15836 raft_consensus.cc:2764] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:05.831808 15878 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:05.868350 15708 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:05.841624 15735 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:19:05.867985 15781 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:19:05.869855 15845 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.932312 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:05.941691 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.900550 15786 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:05.936331 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:05.852883 15757 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.875633 15816 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:19:05.875774 15754 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:19:05.964001 15836 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:19:05.975986 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:05.893324 15755 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:19:05.892380 15844 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:19:05.976953 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:05.976783 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44670.4us
I20260709 13:19:05.923579 15708 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:05.983418 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47212.2us
I20260709 13:19:05.906199 15772 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.865340 15469 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:05.921522 15839 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:05.960477 15845 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.961663 15792 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:06.014424 10130 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:06.008894 15547 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:19:06.017657 10129 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:06.017969 10052 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:06.021685 15469 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.022023 15757 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:05.987057 15708 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.023269 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:06.024061 10050 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.005548 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:05.993659 15781 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:06.027055 10048 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:05.993054 15803 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:19:05.967159 15786 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:05.994046 10127 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:06.008337 10049 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:05.989822 15749 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:06.041862 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:06.042946 10053 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:06.061964 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:06.062294 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56902.8us
I20260709 13:19:06.027487 15781 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:06.031275 15792 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.082123 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:06.083101 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:05.989662 15787 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:06.074391 15383 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:06.092005 15666 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:06.096048 15749 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:06.111616 15749 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.085837 10124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2188d855354542c984f249b94175c660"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:19:06.115612 10124 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:06.116082 10124 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:06.096369 15772 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.092686 15781 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:06.120122 15781 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.121948 15781 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.122130 15803 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:06.092805 15735 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:06.124997 15803 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:06.127285 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.098097 15667 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.100327 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:06.143924 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 43689.9us
W20260709 13:19:06.127072 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.033s	sys 0.026s
W20260709 13:19:06.146780 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.033s	sys 0.026s
I20260709 13:19:06.093489 15792 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:06.147193 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 664914us
I20260709 13:19:06.103705 15383 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.108026 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:06.148012 10130 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.137089 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:06.061643 15802 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:06.138921 10124 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.161684 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.077193 10050 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:06.126408 15766 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.169065 10053 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:06.171799 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.162859 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:06.175846 10125 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:06.183777 15774 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:06.121948 15895 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.201864 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.198024 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:06.205258 10047 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:06.186475 15862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.020s
I20260709 13:19:06.205470 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.227142 10129 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:06.160241 15802 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:06.165407 15661 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:06.242151 10124 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:06.213469 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.243420 10056 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:06.212096 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:06.155594 15383 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.249939 10050 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
W20260709 13:19:06.250586 10131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.018s	sys 0.011s
I20260709 13:19:06.209137 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:06.172813 15787 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:06.248966 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.249066 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:06.251761 10899 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:06.251616 10898 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
W20260709 13:19:06.233575 15862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.005s	sys 0.020s
W20260709 13:19:06.254194 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.009s	sys 0.021s
I20260709 13:19:06.254313 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:06.249887 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.015s	sys 0.023s
W20260709 13:19:06.260422 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.015s	sys 0.023s
W20260709 13:19:06.250994 10131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.018s	sys 0.011s
I20260709 13:19:06.261605 10898 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:19:06.259651 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.009s	sys 0.021s
I20260709 13:19:06.248472 15757 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.265749 15757 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:06.227555 15797 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:06.268893 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 620157us
I20260709 13:19:06.239302 15667 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.277485 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.284391 10123 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:06.246364 15829 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.289378 10056 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:19:06.204813 15895 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:06.319896 10056 pending_rounds.cc:85] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 6
I20260709 13:19:06.274411 15797 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.320539 10056 pending_rounds.cc:107] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:06.256512 15828 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.317045 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.322026 10123 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:06.303318 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.325495 15797 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:06.336087 15733 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:06.339466 10051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.013s	sys 0.024s
W20260709 13:19:06.340114 10051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.013s	sys 0.024s
I20260709 13:19:06.346480 10055 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:06.217386 10130 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:06.367091 10056 raft_consensus.cc:2889] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:06.291045 15667 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:06.395628 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.293784 15829 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:06.377792 15855 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.005s	sys 0.032s
W20260709 13:19:06.421386 15705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.010s	sys 0.027s
I20260709 13:19:06.315681 15802 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.439972 15895 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.432657 15733 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:06.434939 15761 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.007s
I20260709 13:19:06.449208 15792 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:06.461925 15905 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.362s	user 0.002s	sys 0.000s
W20260709 13:19:06.467473 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.472769 15912 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.353s	user 0.001s	sys 0.003s
W20260709 13:19:06.474669 15909 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.355s	user 0.003s	sys 0.000s
I20260709 13:19:06.495500 15839 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:06.434197 15855 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.005s	sys 0.033s
I20260709 13:19:06.493534 15761 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:06.496709 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 662) took 1076 ms. Trace:
I20260709 13:19:06.504844 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 421) took 1457 ms. Trace:
W20260709 13:19:06.437980 15705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.010s	sys 0.028s
I20260709 13:19:06.505491 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:06.504963 10128 rpcz_store.cc:276] 0709 13:19:05.420780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.431379 (+ 10599us) service_pool.cc:224] Handling call
0709 13:19:05.431875 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.432384 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.432428 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.433135 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.914771 (+481636us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:05.914842 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.914955 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.473785 (+558830us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:06.496352 (+ 22567us) raft_consensus.cc:1685] finished
0709 13:19:06.496377 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.496632 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1589,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":122,"spinlock_wait_cycles":16427904,"thread_start_us":481060,"threads_started":1,"wal-append.queue_time_us":686468}
W20260709 13:19:06.504941 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.006s	sys 0.043s
I20260709 13:19:06.458638 15813 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:06.496613 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.014s	sys 0.032s
W20260709 13:19:06.508702 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.014s	sys 0.032s
I20260709 13:19:06.513043 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:06.507947 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.006s	sys 0.043s
W20260709 13:19:06.521109 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.003s	sys 0.027s
I20260709 13:19:06.522794 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.524401 10128 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:06.526095 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.525969 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:06.348203 15787 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:06.539130 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.020s	sys 0.028s
I20260709 13:19:06.505481 10896 rpcz_store.cc:276] 0709 13:19:05.038616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.047265 (+  8649us) service_pool.cc:224] Handling call
0709 13:19:05.047670 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.096514 (+ 48844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.096543 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.126080 (+ 29537us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.663650 (+537570us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:05.663694 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.752015 (+ 88321us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b54004650d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:05.752070 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.461095 (+709025us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:06.496107 (+ 35012us) raft_consensus.cc:1685] finished
0709 13:19:06.496136 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.496279 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11484,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6976,"spinlock_wait_cycles":210105856,"thread_start_us":467675,"threads_started":1,"wal-append.queue_time_us":888076}
I20260709 13:19:06.527078 10128 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:19:06.539673 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.020s	sys 0.028s
W20260709 13:19:06.541935 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 668408us
I20260709 13:19:06.543238 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:06.523352 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 651456us
I20260709 13:19:06.548679 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:06.544441 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:06.544975 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.017s	sys 0.032s
W20260709 13:19:06.521777 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.003s	sys 0.027s
W20260709 13:19:06.551786 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.017s	sys 0.032s
W20260709 13:19:06.553972 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 644808us
W20260709 13:19:06.552939 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 682878us
I20260709 13:19:06.565482 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:06.566854 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.530026 15763 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.030s	sys 0.020s
I20260709 13:19:06.529433 15802 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.536880 15633 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:06.523066 15766 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:06.530603 15810 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:06.639424 15810 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.639905 15810 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.642370 15839 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:06.547726 15757 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.643204 15839 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:06.644284 15839 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:06.602378 15763 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.030s	sys 0.021s
I20260709 13:19:06.644950 15761 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.602500 15776 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:06.645663 15781 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:06.645995 15776 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.646421 15776 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:06.646287 15781 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:06.646857 15667 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:06.630211 10096 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.763s (SENT)
W20260709 13:19:06.651707 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.622546 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73884.3us
I20260709 13:19:06.632707 15666 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:06.549384 15813 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:06.552171 15705 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:06.651518 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:06.617972 15861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.023s
W20260709 13:19:06.663419 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.645044 15792 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:06.686990 10126 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.011s	sys 0.019s
I20260709 13:19:06.670257 15666 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:06.671599 15813 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:06.693235 10126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.011s	sys 0.020s
I20260709 13:19:06.652963 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:06.676522 15816 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.017s	sys 0.012s
W20260709 13:19:06.709542 15836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.013s	sys 0.020s
W20260709 13:19:06.678802 15754 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.022s
I20260709 13:19:06.673700 15705 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:06.676523 15633 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:06.631431 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66051.5us
W20260709 13:19:06.734030 10127 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.036s
W20260709 13:19:06.761104 10127 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.001s	sys 0.037s
W20260709 13:19:06.718976 10048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.001s	sys 0.013s
W20260709 13:19:06.762490 10048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.002s	sys 0.014s
W20260709 13:19:06.718508 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.014s
W20260709 13:19:06.763329 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.001s	sys 0.016s
W20260709 13:19:06.687685 15845 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.007s
W20260709 13:19:06.693130 15861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.000s	sys 0.024s
I20260709 13:19:06.763837 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:06.764791 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:06.772575 10198 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:06.767234 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 735) took 1132 ms. Trace:
I20260709 13:19:06.773538 10055 rpcz_store.cc:276] 0709 13:19:05.634572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.664647 (+ 30075us) service_pool.cc:224] Handling call
0709 13:19:06.766828 (+1102181us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54001b8d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:06.767104 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1373523200}
W20260709 13:19:06.719367 15444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.000s	sys 0.026s
W20260709 13:19:06.733268 15816 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.018s	sys 0.013s
W20260709 13:19:06.732987 15836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.014s	sys 0.020s
W20260709 13:19:06.733917 15755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.017s	sys 0.008s
I20260709 13:19:06.739473 15813 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:06.737375 15749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.010s	sys 0.005s
W20260709 13:19:06.776746 15444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.001s	sys 0.026s
I20260709 13:19:06.797473 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:06.761372 15754 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.010s	sys 0.023s
I20260709 13:19:06.785116 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:06.732924 15855 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:06.713358 15666 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:06.789040 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:06.788337 15755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.018s	sys 0.008s
W20260709 13:19:06.795819 15708 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.005s
I20260709 13:19:06.790045 15667 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:06.778692 10049 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.016s	sys 0.002s
W20260709 13:19:06.781497 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.003s	sys 0.010s
I20260709 13:19:06.798328 10204 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:06.799131 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:06.800806 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 147957us
W20260709 13:19:06.820818 10122 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.013s	sys 0.070s
W20260709 13:19:06.821087 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.001s	sys 0.022s
W20260709 13:19:06.823809 15661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.013s	sys 0.007s
W20260709 13:19:06.824260 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 273944us
W20260709 13:19:06.861716 10129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.004s	sys 0.012s
W20260709 13:19:06.909847 10129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.012s
W20260709 13:19:06.863937 15801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.007s	sys 0.012s
W20260709 13:19:06.916985 15801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.007s	sys 0.013s
I20260709 13:19:06.921005 15633 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:06.923710 15878 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.965s	user 0.013s	sys 0.002s
I20260709 13:19:06.924270 15878 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:06.805238 15749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.011s	sys 0.005s
W20260709 13:19:06.870808 10050 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.011s
W20260709 13:19:06.872484 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 242142us
W20260709 13:19:06.880755 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.001s	sys 0.010s
I20260709 13:19:06.818884 15920 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.906337 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.003s	sys 0.011s
I20260709 13:19:06.926854 15878 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:06.903770 15927 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:06.901129 15855 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:06.906200 10122 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.014s	sys 0.071s
I20260709 13:19:06.906395 10203 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.906111 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.002s	sys 0.022s
W20260709 13:19:06.906123 10049 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.017s	sys 0.003s
I20260709 13:19:06.795375 15816 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:06.906107 15667 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:06.914417 15906 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.816s	user 0.000s	sys 0.004s
I20260709 13:19:06.915591 15945 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:06.916193 15931 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:06.779217 15845 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:06.883631 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.011s	sys 0.006s
W20260709 13:19:06.936496 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.002s	sys 0.010s
W20260709 13:19:06.936926 10050 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.005s	sys 0.011s
W20260709 13:19:06.909571 15661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.013s	sys 0.007s
I20260709 13:19:06.951292 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:06.938131 15801 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.930722 15829 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.005446 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:06.955065 10056 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:07.009547 10899 raft_consensus.cc:2368] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
W20260709 13:19:06.790167 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.002s	sys 0.011s
W20260709 13:19:07.010594 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.003s	sys 0.011s
I20260709 13:19:07.011364 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:06.892643 15797 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:06.935848 15925 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:06.931461 15754 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:06.805600 10053 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.003s	sys 0.020s
I20260709 13:19:07.002986 10898 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:19:06.998059 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.004397 10055 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:07.006750 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.015s	sys 0.010s
W20260709 13:19:07.041738 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 787481us
I20260709 13:19:07.043781 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:07.040593 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:07.056555 10055 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:07.057684 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:07.058264 10055 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:06.992672 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.911021 15708 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.022684 10122 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:07.013835 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:07.026827 10896 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a82842926e645c08dde847bfe182c31"
dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:34750
I20260709 13:19:07.095959 10896 raft_consensus.cc:606] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:07.027222 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:06.954380 15927 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:07.122010 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.123901 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:07.124289 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 185069us
I20260709 13:19:07.125469 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:07.036923 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91503us
W20260709 13:19:07.127803 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.115s (SENT)
I20260709 13:19:07.128314 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:07.037914 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:07.038805 10053 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.004s	sys 0.020s
I20260709 13:19:06.968457 15933 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:06.969129 15667 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:07.006398 15945 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:07.136921 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.882205 15892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.005s	sys 0.012s
I20260709 13:19:07.146595 10205 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:07.147137 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:07.038563 15754 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:07.154023 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:06.949417 15920 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.161823 10170 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:07.111853 15781 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:07.167362 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 123930us
W20260709 13:19:07.169169 10096 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.216s (SENT)
W20260709 13:19:07.172222 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.178735 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:07.241400 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72284.4us
I20260709 13:19:07.028728 15925 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:07.197388 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 184445us
I20260709 13:19:07.247825 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:07.248231 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:06.939244 15705 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.179399 10020 leader_election.cc:400] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:07.262272 10020 leader_election.cc:403] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 13:19:07.148161 15763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.013s	sys 0.041s
I20260709 13:19:07.218310 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2730) took 2850 ms. Trace:
I20260709 13:19:07.168232 15924 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.003429 15661 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:07.224058 10050 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:07.287145 15861 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:07.162438 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:07.326701 10892 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:07.236879 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:07.242048 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.243857 10096 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.560s (SENT)
I20260709 13:19:07.241391 10169 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:07.128902 15952 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:07.185191 15839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.026s	sys 0.024s
I20260709 13:19:07.159283 15919 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.350891 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 225535us
I20260709 13:19:07.252940 10055 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
W20260709 13:19:07.224900 15892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.011s	sys 0.016s
I20260709 13:19:07.357959 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:07.257678 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:07.159291 15829 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:07.225673 15908 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:19:07.267535 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 98252.4us
I20260709 13:19:07.369717 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:07.178493 15958 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.268263 10054 rpcz_store.cc:276] 0709 13:19:04.319620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.328305 (+  8685us) service_pool.cc:224] Handling call
0709 13:19:04.328780 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.139616 (+810836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.139655 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.155700 (+ 16045us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.473981 (+318281us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b48000030d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:19:05.848279 (+374298us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:05.848322 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.848397 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.784341 (+935944us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:07.169338 (+384997us) raft_consensus.cc:1685] finished
0709 13:19:07.169363 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.169536 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":810393,"raft.queue_time_us":810530,"raft.run_cpu_time_us":1648,"raft.run_wall_time_us":331332,"spinlock_wait_cycles":162894976,"thread_start_us":643072,"threads_started":1,"wal-append.queue_time_us":843607}
I20260709 13:19:07.220376 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:07.146577 15945 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:07.167079 15895 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:07.018396 15801 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.249517 15929 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:07.224223 10899 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:07.330751 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:07.333177 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 91926.7us
I20260709 13:19:07.270778 15931 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:07.332365 15952 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.362646 15924 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:07.289149 15781 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:07.357349 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:07.359498 10900 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:07.361728 15383 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.093s	user 0.012s	sys 0.002s
W20260709 13:19:07.343715 15839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.026s	sys 0.024s
I20260709 13:19:07.342741 15878 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:07.489769 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.432538 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.003s	sys 0.003s
W20260709 13:19:07.490660 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.003s	sys 0.004s
W20260709 13:19:07.492273 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.402855 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:07.495445 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:07.496582 10122 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:07.253654 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2753) took 1072 ms. Trace:
W20260709 13:19:07.436795 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 192950us
W20260709 13:19:07.499866 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.830s (SENT)
I20260709 13:19:07.500317 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:07.503793 15878 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:07.504616 15920 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:07.505265 15920 raft_consensus.cc:993] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: : Instructing follower 01bfaf1ad56a4deea886bbc64dec466c to start an election
I20260709 13:19:07.504940 15878 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.505913 15920 raft_consensus.cc:1081] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Signalling peer 01bfaf1ad56a4deea886bbc64dec466c to start an election
I20260709 13:19:07.397318 15925 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.507789 15925 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:07.458072 15958 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.460155 15952 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:07.510869 15967 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
W20260709 13:19:07.512559 15923 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.385s	user 0.001s	sys 0.000s
W20260709 13:19:07.514305 15942 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:19:07.515472 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2750) took 1335 ms. Trace:
I20260709 13:19:07.515950 10129 rpcz_store.cc:276] 0709 13:19:06.180261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.310636 (+130375us) service_pool.cc:224] Handling call
0709 13:19:06.311088 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.910363 (+599275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.910399 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.911021 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.003497 (+ 92476us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.003550 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.003647 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.515204 (+511557us) raft_consensus.cc:1685] finished
0709 13:19:07.515228 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.515401 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5,"raft.queue_time_us":603629,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":126,"thread_start_us":677164,"threads_started":2,"wal-append.queue_time_us":231095}
W20260709 13:19:07.517669 15918 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.836s	user 0.006s	sys 0.000s
I20260709 13:19:07.411027 15731 raft_consensus.cc:993] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:07.397969 15926 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:07.415580 15936 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.444s	user 0.001s	sys 0.001s
I20260709 13:19:07.469528 10899 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:07.375669 15917 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:07.281122 15855 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.465618 10893 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:07.433974 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 185847us
I20260709 13:19:07.508625 15369 raft_consensus.cc:993] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:07.411453 15895 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.518585 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2786) took 1388 ms. Trace:
W20260709 13:19:07.519553 15964 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:19:07.626461 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.521020 15839 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.521294 15965 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
I20260709 13:19:07.525271 15845 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.472908 15763 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:19:07.527204 15844 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:07.527665 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2736) took 1482 ms. Trace:
I20260709 13:19:07.529392 15801 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:07.456110 15892 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:07.531436 15910 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.408s	user 0.003s	sys 0.002s
W20260709 13:19:07.532234 15911 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.412s	user 0.000s	sys 0.009s
W20260709 13:19:07.527522 15956 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:19:07.535513 10122 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:07.689857 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2782) took 1561 ms. Trace:
I20260709 13:19:07.536624 15952 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:07.538796 15810 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.898s	user 0.011s	sys 0.000s
W20260709 13:19:07.692811 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.024s	sys 0.073s
W20260709 13:19:07.693380 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.024s	sys 0.073s
I20260709 13:19:07.407648 15836 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:19:07.542953 15797 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:07.343199 15927 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.498571 15755 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:07.157423 15933 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.677207 15839 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.553890 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2759) took 1087 ms. Trace:
I20260709 13:19:07.700947 10128 rpcz_store.cc:276] 0709 13:19:06.466312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.543643 (+ 77331us) service_pool.cc:224] Handling call
0709 13:19:06.544206 (+   563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.945061 (+400855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.945120 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.963738 (+ 18618us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.219088 (+255350us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:07.219162 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.219251 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.553594 (+334343us) raft_consensus.cc:1685] finished
0709 13:19:07.553619 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.553813 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6839,"raft.queue_time_us":371273,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":648194,"threads_started":2,"wal-append.queue_time_us":587583}
I20260709 13:19:07.555581 15816 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:07.555774 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55947.2us
I20260709 13:19:07.510916 15958 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:07.458843 15661 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.560411 15708 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:07.561028 15776 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.914s	user 0.012s	sys 0.001s
I20260709 13:19:07.493077 15383 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.256479 10203 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:07.501001 10123 rpcz_store.cc:276] 0709 13:19:06.181234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.323185 (+141951us) service_pool.cc:224] Handling call
0709 13:19:06.349371 (+ 26186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.349663 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:06.884063 (+534400us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.252953 (+368890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.253161 (+   208us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:07.253243 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.253354 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.253562 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33501,"prepare.queue_time_us":565193,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":65367,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":32916,"thread_start_us":650044,"threads_started":2,"wal-append.queue_time_us":347564}
I20260709 13:19:07.335216 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:07.577428 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 415671us
W20260709 13:19:07.579504 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.029s	sys 0.068s
W20260709 13:19:07.719453 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.030s	sys 0.068s
I20260709 13:19:07.377182 15829 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.700519 15755 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.729210 15829 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.729538 15755 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.697661 15810 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:07.581889 10204 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.002s
I20260709 13:19:07.731396 10204 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:07.580907 15781 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:07.699460 15749 raft_consensus.cc:1081] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
W20260709 13:19:07.558872 15786 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.504s	user 0.015s	sys 0.000s
I20260709 13:19:07.732837 15810 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:07.733291 15786 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.736153 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2735) took 1691 ms. Trace:
I20260709 13:19:07.736763 10127 rpcz_store.cc:276] 0709 13:19:06.044927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.047424 (+  2497us) service_pool.cc:224] Handling call
0709 13:19:06.051955 (+  4531us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.761761 (+709806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.761813 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.784571 (+ 22758us) log.cc:844] Serialized 52 byte log entry
0709 13:19:07.129942 (+345371us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:07.166890 (+ 36948us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.168355 (+  1465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.168457 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.735826 (+567369us) raft_consensus.cc:1685] finished
0709 13:19:07.735852 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.736035 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31625,"prepare.queue_time_us":49171,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":945423,"raft.run_cpu_time_us":445,"raft.run_wall_time_us":40896,"thread_start_us":928592,"threads_started":2,"wal-append.queue_time_us":798282}
I20260709 13:19:07.601073 15917 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:07.739691 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2796) took 1475 ms. Trace:
I20260709 13:19:07.740149 15917 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.740306 10047 rpcz_store.cc:276] 0709 13:19:06.263992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.469546 (+205554us) service_pool.cc:224] Handling call
0709 13:19:06.469912 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.470374 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.470416 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.476182 (+  5766us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.847253 (+371071us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:06.847561 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.848084 (+   523us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.739386 (+891302us) raft_consensus.cc:1685] finished
0709 13:19:07.739412 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.739608 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7001,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":15463,"thread_start_us":351838,"threads_started":1,"wal-append.queue_time_us":791047}
I20260709 13:19:07.602638 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:07.742851 15943 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:07.743078 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140524us
I20260709 13:19:07.744270 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:07.593528 10169 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:07.443722 15929 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:07.613884 15802 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.951s	user 0.011s	sys 0.003s
I20260709 13:19:07.614321 15926 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:07.624204 15895 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:07.520641 15966 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:19:07.608493 10053 rpcz_store.cc:276] 0709 13:19:06.130177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.190944 (+ 60767us) service_pool.cc:224] Handling call
0709 13:19:06.191507 (+   563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.200799 (+  9292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.200859 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.201656 (+   797us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.132199 (+930543us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:07.132282 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.132388 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.518302 (+385914us) raft_consensus.cc:1685] finished
0709 13:19:07.518334 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.518500 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482,"raft.queue_time_us":8706,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":21117,"thread_start_us":871658,"threads_started":1,"wal-append.queue_time_us":903048}
W20260709 13:19:07.676291 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.681968 10126 rpcz_store.cc:276] 0709 13:19:06.045320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.045530 (+   210us) service_pool.cc:224] Handling call
0709 13:19:06.075225 (+ 29695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.734661 (+659436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.734716 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.735601 (+   885us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.848074 (+112473us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.848111 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.848186 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.527392 (+679206us) raft_consensus.cc:1685] finished
0709 13:19:07.527416 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.527594 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2029,"raft.queue_time_us":839079,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":770510,"threads_started":2,"wal-append.queue_time_us":669863}
W20260709 13:19:07.685985 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.604285 15853 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:07.690467 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:07.358224 15754 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:07.694133 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 680480us
I20260709 13:19:07.685827 15844 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:07.695974 10049 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.005s	sys 0.033s
I20260709 13:19:07.690372 10052 rpcz_store.cc:276] 0709 13:19:06.128699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.132620 (+  3921us) service_pool.cc:224] Handling call
0709 13:19:06.133169 (+   549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.133728 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:06.133775 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.134441 (+   666us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.763986 (+629545us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:06.764036 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.764103 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.689567 (+925464us) raft_consensus.cc:1685] finished
0709 13:19:07.689593 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.689778 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1293,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":22831,"thread_start_us":629123,"threads_started":1,"wal-append.queue_time_us":775923}
I20260709 13:19:07.686113 15954 raft_consensus.cc:1081] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:07.552151 15924 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:07.598577 15666 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.000s
I20260709 13:19:07.706082 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2757) took 1523 ms. Trace:
I20260709 13:19:07.719677 15803 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:07.722882 15776 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:07.775547 15920 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:19:07.725116 15383 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:07.698998 15836 consensus_queue.cc:1243] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:07.730719 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2210) took 2693 ms. Trace:
I20260709 13:19:07.732460 15781 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:07.590237 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75526.6us
W20260709 13:19:07.543057 15813 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.723s	user 0.007s	sys 0.004s
I20260709 13:19:07.775298 15801 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:07.747640 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 154241us
I20260709 13:19:07.781239 10125 rpcz_store.cc:276] 0709 13:19:06.182556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.379606 (+197050us) service_pool.cc:224] Handling call
0709 13:19:06.430669 (+ 51063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.849726 (+419057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.850039 (+   313us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.853615 (+  3576us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.954877 (+101262us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:06.954926 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.955003 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.705823 (+750820us) raft_consensus.cc:1685] finished
0709 13:19:07.705848 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.706011 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":484282,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":63,"thread_start_us":517190,"threads_started":2,"wal-append.queue_time_us":269444}
I20260709 13:19:07.755577 10128 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:07.369000 15919 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:07.767225 10049 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.006s	sys 0.033s
I20260709 13:19:07.773983 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:07.736258 15786 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.863955 10053 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:19:07.758893 15943 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:07.759781 15878 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:07.741469 15839 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:07.858700 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2773) took 2090 ms. Trace:
I20260709 13:19:07.764564 10122 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:07.858103 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 455329us
I20260709 13:19:07.891700 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:07.860900 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44211.1us
I20260709 13:19:07.861724 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:07.864202 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2780) took 1944 ms. Trace:
I20260709 13:19:07.865473 10047 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a82842926e645c08dde847bfe182c31"
dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:48538
I20260709 13:19:07.792135 10196 rpcz_store.cc:276] 0709 13:19:05.037121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.111304 (+ 74183us) service_pool.cc:224] Handling call
0709 13:19:05.112139 (+   835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.112579 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.112729 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.113286 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.470221 (+356935us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18001d71c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:05.886252 (+416031us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:05.886300 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.886385 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.531537 (+1645152us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:07.730456 (+198919us) raft_consensus.cc:1685] finished
0709 13:19:07.730482 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.730646 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4635,"raft.queue_time_us":6590,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":1961,"spinlock_wait_cycles":106859136,"thread_start_us":415778,"threads_started":1,"wal-append.queue_time_us":995767}
W20260709 13:19:07.867596 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73684.7us
I20260709 13:19:07.868528 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:07.786491 15802 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:07.948616 10895 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:07.786293 15754 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:07.937466 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:07.924185 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.952011 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:07.895500 10048 rpcz_store.cc:276] 0709 13:19:05.768631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.029374 (+260743us) service_pool.cc:224] Handling call
0709 13:19:06.029766 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.030198 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.030235 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.052711 (+ 22476us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.762939 (+710228us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.762988 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.763086 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.791407 (+1028321us) raft_consensus.cc:1685] finished
0709 13:19:07.791431 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.858602 (+ 67171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78918,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":627452,"threads_started":1,"wal-append.queue_time_us":1622825}
I20260709 13:19:07.899320 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2756) took 1717 ms. Trace:
I20260709 13:19:07.948949 10047 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:07.956259 10047 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:07.955480 10130 rpcz_store.cc:276] 0709 13:19:06.182217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.380271 (+198054us) service_pool.cc:224] Handling call
0709 13:19:06.384820 (+  4549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.878421 (+493601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.878571 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.982068 (+103497us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.720175 (+738107us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.720252 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.720374 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.898976 (+178602us) raft_consensus.cc:1685] finished
0709 13:19:07.899007 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.899225 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":530098,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1170266,"threads_started":2,"wal-append.queue_time_us":667308}
W20260709 13:19:07.952915 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 84445.4us
I20260709 13:19:07.950223 10049 rpcz_store.cc:276] 0709 13:19:05.919994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.073218 (+153224us) service_pool.cc:224] Handling call
0709 13:19:06.136149 (+ 62931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.136448 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:06.967218 (+830770us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.863802 (+896584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.863878 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.863927 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.863999 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.864136 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1338092,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":2700,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":25415,"thread_start_us":1692124,"threads_started":2,"wal-append.queue_time_us":809618}
I20260709 13:19:07.890654 15845 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.788062 15929 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:07.961198 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.962250 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:07.778828 15979 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.186s
I20260709 13:19:07.896167 15952 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.794310 15813 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.966193 15845 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.948786 15933 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:07.863170 15761 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:19:07.890256 15878 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.969983 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:07.977566 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 209841us
I20260709 13:19:07.948412 15839 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.870244 15943 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:07.959797 15954 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:07.789497 15369 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.178s
I20260709 13:19:07.950121 15926 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:07.959882 15802 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.031220 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:08.031924 15845 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.003432 15954 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.032899 15954 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:08.040510 15667 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.862s	user 0.016s	sys 0.000s
I20260709 13:19:08.041116 15667 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.953702 15776 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:07.789592 15666 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.866942 15801 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.975844 15786 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:07.795735 15985 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.187s
I20260709 13:19:07.979230 15929 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.791210 15924 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:07.973093 15982 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:08.006263 15986 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.003216 15839 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.003747 15763 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:07.778893 15977 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.235s
I20260709 13:19:07.953258 15844 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.014518 15926 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:07.987968 15878 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.025995 15919 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.027987 15979 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:07.990119 15383 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:08.028301 15813 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.045990 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2804) took 1066 ms. Trace:
I20260709 13:19:08.123672 10050 rpcz_store.cc:276] 0709 13:19:06.979354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.283260 (+303906us) service_pool.cc:224] Handling call
0709 13:19:07.333553 (+ 50293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.764610 (+431057us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:07.764653 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.764705 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:07.764743 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.045562 (+280819us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:08.045643 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.045883 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1724,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9185,"spinlock_wait_cycles":990462208}
I20260709 13:19:08.126266 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:08.046924 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62401.2us
I20260709 13:19:08.022661 15369 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:08.127628 15369 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:08.128845 15369 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.130446 15369 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:08.062228 10196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:19:08.062917 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.064128 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:08.133095 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:08.052286 15945 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.636s	user 0.015s	sys 0.000s
I20260709 13:19:08.135943 15945 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.075677 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:08.070887 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.043320 15781 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.140762 15781 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.138917 15945 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.141422 15781 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.141975 10050 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.077718 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.147572 10055 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:08.148689 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.149366 10055 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:08.150367 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.090837 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.100622 10054 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:19:08.152194 10056 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:08.153013 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.074554 15845 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.114537 15979 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.115338 15839 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.003451 15943 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:08.154199 15845 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.104354 15986 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.155995 15979 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.157096 15986 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.076732 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.153728 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.145812 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.159246 10055 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
I20260709 13:19:08.159363 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 665) took 1032 ms. Trace:
I20260709 13:19:08.159948 10895 rpcz_store.cc:276] 0709 13:19:07.127443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.127652 (+   209us) service_pool.cc:224] Handling call
0709 13:19:08.159161 (+1031509us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400640854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:08.159295 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1507819392}
I20260709 13:19:08.160570 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.161614 10055 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:08.162593 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.153115 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.163461 10056 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.163836 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.159430 10052 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.158568 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.166065 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.166388 10048 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:08.143708 15939 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.235s	user 0.000s	sys 0.004s
I20260709 13:19:08.167285 10054 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:08.168020 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.144536 15781 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:08.167335 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.178647 15878 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.181865 15878 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:08.163991 15943 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.182870 15666 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.184322 15943 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:08.184551 15802 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.188503 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:08.175541 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:08.192301 15948 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.951s	user 0.002s	sys 0.000s
I20260709 13:19:08.090946 15929 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:08.197324 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.199247 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:08.258968 10055 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:08.200109 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:08.260066 10894 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:08.202039 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78713.8us
W20260709 13:19:08.202695 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.162539 15845 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.197376 15954 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.094583 15989 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:08.211181 10048 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:08.213575 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:08.196895 15915 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.402s	user 0.012s	sys 0.004s
W20260709 13:19:08.237540 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.254560 10056 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:08.257233 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:08.205931 15916 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.422s	user 0.007s	sys 0.003s
W20260709 13:19:08.262283 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.424286 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.419739 15987 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:19:08.319134 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.432767 10050 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.389321 15797 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.435451 15797 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.386943 15992 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:08.422942 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.423424 10896 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.441116 15797 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:08.281690 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.422755 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.442061 10055 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:08.442446 10056 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
W20260709 13:19:08.264659 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89239.2us
I20260709 13:19:08.443008 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.439617 15954 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:08.309751 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.443681 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.426486 10052 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 14. Ops: []
W20260709 13:19:08.448982 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.449239 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.445204 10020 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:08.450275 10020 leader_election.cc:403] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:19:08.450106 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 188927us
I20260709 13:19:08.440043 15705 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:08.446501 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 318690us
I20260709 13:19:08.447425 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.450106 10056 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:08.467618 10205 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:08.469856 10204 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:08.472373 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.472884 10053 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:08.451776 10196 raft_consensus.cc:606] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:08.473511 10056 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:08.467397 10203 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:08.217931 15667 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.218328 15708 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.476164 10197 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:08.470896 10055 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:08.485996 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:08.421900 15926 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.253404 15383 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:08.425148 15989 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.387946 15801 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.492359 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:08.501327 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50449us
I20260709 13:19:08.501983 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.503419 10055 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:08.503445 10201 raft_consensus.cc:2368] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
W20260709 13:19:08.506700 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.446405 15933 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.521044 15383 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.521672 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:08.522619 10198 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:08.508867 15989 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.528436 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 667) took 2910 ms. Trace:
I20260709 13:19:08.500736 15992 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:08.529680 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.528966 10131 rpcz_store.cc:276] 0709 13:19:05.617810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.622288 (+  4478us) service_pool.cc:224] Handling call
0709 13:19:05.622710 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:05.623198 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.623241 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:05.624002 (+   761us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.261375 (+637373us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.261415 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.261488 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.195980 (+1934492us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:08.528073 (+332093us) raft_consensus.cc:1685] finished
0709 13:19:08.528109 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.528358 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1120,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":5187,"spinlock_wait_cycles":264728320,"thread_start_us":620993,"threads_started":1,"wal-append.queue_time_us":1046865}
I20260709 13:19:08.530546 10055 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:08.450834 15945 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.463464 15952 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.450563 15813 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:08.533490 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.476045 15661 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.537444 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.475788 15928 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:08.536870 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65259.6us
W20260709 13:19:08.538702 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.551463 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.538708 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:08.543607 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.548030 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 727) took 2928 ms. Trace:
I20260709 13:19:08.548523 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.557557 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.558322 10053 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.558931 10056 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:19:08.558297 10047 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.002s
W20260709 13:19:08.569625 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.574970 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.554847 10051 rpcz_store.cc:276] 0709 13:19:05.628634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.628899 (+   265us) service_pool.cc:224] Handling call
0709 13:19:05.636110 (+  7211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.636632 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.636672 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.642223 (+  5551us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.340697 (+698474us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:06.856005 (+515308us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.857582 (+  1577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.858234 (+   652us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.205049 (+1346815us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:08.531213 (+326164us) raft_consensus.cc:1685] finished
0709 13:19:08.547781 (+ 16568us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.547958 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1190071,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":1135,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":250,"spinlock_wait_cycles":275076864,"thread_start_us":1193361,"threads_started":2,"wal-append.queue_time_us":1043845}
I20260709 13:19:08.581408 10203 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:19:08.568758 10047 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.580565 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.581852 10198 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:19:08.584941 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46316.9us
W20260709 13:19:08.583424 10096 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.861s (SENT)
I20260709 13:19:08.587919 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:08.583478 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.589275 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.585011 10055 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:08.591734 10056 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:19:08.593700 10096 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.889s (SENT)
W20260709 13:19:08.594786 10096 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.970s (SENT)
I20260709 13:19:08.595254 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:08.596652 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.597661 10056 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:08.592123 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:08.629170 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:08.631326 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.593466 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.643462 10051 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:08.700733 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.707612 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.708488 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.709116 10055 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:08.747368 10047 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:08.727963 10896 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:08.711076 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 160569us
I20260709 13:19:08.709292 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:08.748505 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:08.754721 10893 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:08.754932 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.746892 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.756588 10055 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 9.
W20260709 13:19:08.710752 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 201319us
I20260709 13:19:08.757054 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.765749 10198 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:08.709302 10053 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.775105 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.758251 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:08.778532 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.782254 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 914) took 1447 ms. Trace:
I20260709 13:19:08.784704 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.785701 10050 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.785998 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.501561 15708 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.790163 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.804855 10050 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:08.805670 10056 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:08.782416 10055 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:08.536375 15933 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:08.589865 15801 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:08.549955 15945 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.549703 15952 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:08.815104 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 223064us
I20260709 13:19:08.824637 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:08.816684 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63828us
I20260709 13:19:08.583422 15993 raft_consensus.cc:2764] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:19:08.552976 15813 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.806980 10050 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:08.834380 10050 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:08.834980 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:08.822196 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64005.9us
I20260709 13:19:08.837378 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:08.822729 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:08.838848 10056 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:08.552088 15661 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:08.860805 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.783198 10892 rpcz_store.cc:276] 0709 13:19:07.334498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.334809 (+   311us) service_pool.cc:224] Handling call
0709 13:19:08.782010 (+1447201us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540053b454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:08.782171 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835419520}
I20260709 13:19:08.563876 15928 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:08.819661 15924 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:08.943156 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.971483 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.035635 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:08.970727 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.042376 10202 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:09.008844 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.884413 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56241.2us
W20260709 13:19:09.046931 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 452144us
I20260709 13:19:09.002005 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:08.982465 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:09.001546 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164300us
W20260709 13:19:09.051548 10894 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.782s	user 0.012s	sys 0.000s
I20260709 13:19:09.055824 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.056699 10205 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:09.057163 10205 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:09.054203 10894 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:08.875571 10053 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:09.036135 15921 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 2.127s	user 0.005s	sys 0.000s
I20260709 13:19:09.041054 15661 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:09.051733 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:09.039799 16003 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:09.084501 10205 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.097023 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:09.098248 10051 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:08.870563 15920 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:08.889456 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.107149 10203 raft_consensus.cc:2368] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
W20260709 13:19:09.046492 15961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.014s	sys 0.032s
W20260709 13:19:09.032058 16000 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:19:09.083452 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.112967 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:09.025403 15444 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:19:09.094722 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48410.9us
I20260709 13:19:08.840874 15803 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:09.193305 10895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.019s	sys 0.065s
I20260709 13:19:08.868979 16006 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.841470 15926 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:08.949973 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.108701 10205 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:09.088953 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 435824us
I20260709 13:19:08.885661 15929 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:09.075031 15776 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:09.119750 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.009s	sys 0.025s
W20260709 13:19:09.204512 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.009s	sys 0.025s
I20260709 13:19:09.136355 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:09.204875 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 759523us
I20260709 13:19:09.205390 10051 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:09.191702 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140051us
I20260709 13:19:08.976082 10199 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
W20260709 13:19:09.193962 10895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.019s	sys 0.065s
W20260709 13:19:09.196082 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 118193us
W20260709 13:19:09.111884 15961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.015s	sys 0.032s
I20260709 13:19:09.208990 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.206267 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.216289 10131 raft_consensus.cc:2368] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:09.212369 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.218065 10127 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:09.220911 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.221446 10127 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:09.221607 10123 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:09.223686 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.222909 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.226145 10127 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:09.231889 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.233167 10128 raft_consensus.cc:2368] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:09.234375 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.235286 10126 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:09.243263 10197 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:09.266855 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.272221 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.273626 10126 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:09.337815 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:09.352677 10127 raft_consensus.cc:2410] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:09.392515 10124 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 15. Ops: []
I20260709 13:19:09.394986 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:09.396253 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.020s	sys 0.025s
W20260709 13:19:09.401378 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.021s	sys 0.025s
I20260709 13:19:09.419673 10129 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:09.203969 15878 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:09.443740 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.445060 10130 raft_consensus.cc:2368] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:09.226173 15926 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.437029 15792 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:19:09.449544 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.450698 10127 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.456841 10197 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:19:09.458294 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.347918 10900 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:09.343886 10125 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:09.462185 10202 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:09.482759 10123 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:09.486318 10124 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:09.338328 15444 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:09.206792 16003 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:19:09.559192 15989 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.031s	user 0.014s	sys 0.000s
I20260709 13:19:09.456171 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:09.573444 10051 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
W20260709 13:19:09.498297 16008 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:09.499248 15933 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:09.500820 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:09.454862 16005 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.655s	user 0.002s	sys 0.000s
I20260709 13:19:09.511519 15945 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:09.519826 15763 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.521322 15813 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:09.224982 16006 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.526741 15708 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:09.532460 10205 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:09.532843 15928 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:09.536365 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.538241 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.538748 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.603758 10204 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.603972 10129 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:09.539939 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:09.545526 10893 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.790s	user 0.012s	sys 0.000s
I20260709 13:19:09.605453 10893 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:09.602674 10198 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:09.606289 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 920) took 1045 ms. Trace:
I20260709 13:19:09.548187 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:09.550894 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.552086 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.609472 10202 raft_consensus.cc:2410] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:19:09.540712 15878 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:09.193678 15803 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:09.342159 16000 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:19:09.249703 15776 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.341646 15929 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:09.460824 10900 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:09.571281 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.008s	sys 0.061s
W20260709 13:19:09.630815 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.008s	sys 0.062s
I20260709 13:19:09.632485 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 917) took 1771 ms. Trace:
I20260709 13:19:09.579063 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.581435 10130 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:09.563714 15952 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:09.586553 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.637457 10124 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:09.558350 15924 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:09.568506 16013 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:09.600927 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:09.578186 15927 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.610600 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.602586 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:09.606886 10893 rpcz_store.cc:276] 0709 13:19:08.560539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.582720 (+ 22181us) service_pool.cc:224] Handling call
0709 13:19:09.606202 (+1023482us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:09.614471 10127 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:09.592378 15933 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.632930 10891 rpcz_store.cc:276] 0709 13:19:07.861470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.861752 (+   282us) service_pool.cc:224] Handling call
0709 13:19:07.892203 (+ 30451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.251737 (+359534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.251774 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.355594 (+103820us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.870508 (+514914us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:08.870563 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.870658 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.632221 (+761563us) raft_consensus.cc:1685] finished
0709 13:19:09.632242 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.632424 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72460,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6888,"thread_start_us":841049,"threads_started":2,"wal-append.queue_time_us":724327}
I20260709 13:19:09.634133 10123 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:19:09.634390 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.021s
I20260709 13:19:09.596061 15945 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:09.598081 16010 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:09.666040 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63586.2us
I20260709 13:19:09.641222 10900 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.638561 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.639745 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.670519 10129 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
W20260709 13:19:09.578596 15944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.021s	sys 0.030s
I20260709 13:19:09.684343 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:09.580304 15989 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.609341 15813 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:09.659081 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.002s	sys 0.021s
I20260709 13:19:09.628494 15803 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.663749 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.704838 10201 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:09.706198 10202 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:09.706478 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.674263 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:09.701563 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2761960509999426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23610000000000003s for a response
W20260709 13:19:09.718739 10096 connection.cc:256] RPC call timeout handler was delayed by -1.1925734169999487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2718s for a response
W20260709 13:19:09.719542 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.868s (SENT) [suppressed 24 similar messages]
I20260709 13:19:09.695945 16013 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:09.697299 15924 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:09.685184 15944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.022s	sys 0.031s
I20260709 13:19:09.600298 15763 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.721103 10198 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:09.657490 15929 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.704329 15952 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:19:09.724006 10096 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.164s (SENT)
I20260709 13:19:09.726999 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.662326 15933 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:09.731761 10096 connection.cc:256] RPC call timeout handler was delayed by -1.0231977070000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260709 13:19:09.629015 15444 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.733476 16013 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:09.737229 16019 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:09.737345 16013 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.664889 15945 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.747658 10201 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:09.751573 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.753193 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.755561 10202 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:09.762341 10198 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:09.764928 10895 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.001s	sys 0.027s
I20260709 13:19:09.733822 15924 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:09.768349 10895 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.001s	sys 0.027s
I20260709 13:19:09.768834 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.713464 15878 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.709286 15813 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:09.775012 15792 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.782930 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 426) took 1584 ms. Trace:
I20260709 13:19:09.756950 16015 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:09.757396 15952 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:09.788722 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 39) took 1728 ms. Trace:
I20260709 13:19:09.791296 10196 rpcz_store.cc:276] 0709 13:19:08.052303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.062045 (+  9742us) service_pool.cc:224] Handling call
0709 13:19:09.780607 (+1718562us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400359dd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:09.780752 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736517376}
I20260709 13:19:09.752113 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.793200 10201 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
W20260709 13:19:09.792418 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.783371 10204 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:19:09.800858 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:09.772348 15708 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:09.780040 15776 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:09.792352 15920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.006s	sys 0.002s
W20260709 13:19:09.813452 15920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.007s	sys 0.002s
I20260709 13:19:09.820333 15920 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:09.821331 15920 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.822831 15989 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:09.824234 15989 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:09.799806 15792 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.786365 10895 rpcz_store.cc:276] 0709 13:19:08.198308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.201582 (+  3274us) service_pool.cc:224] Handling call
0709 13:19:08.202007 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.202307 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:08.550029 (+347722us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.776675 (+1226646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:09.776771 (+    96us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:09.776874 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.776991 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.777190 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":739237,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":62,"raft.queue_time_us":521241,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":66,"spinlock_wait_cycles":5240704,"thread_start_us":1573392,"threads_started":3,"wal-append.queue_time_us":1044721}
I20260709 13:19:09.805770 16015 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:09.834409 15853 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:09.829591 16006 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:09.908994 10050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.008s	sys 0.028s
W20260709 13:19:09.910166 10050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.009s	sys 0.029s
W20260709 13:19:09.616456 10047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.011s	sys 0.050s
I20260709 13:19:09.912411 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2816) took 1794 ms. Trace:
W20260709 13:19:09.912606 10047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.011s	sys 0.050s
I20260709 13:19:09.913151 10050 rpcz_store.cc:276] 0709 13:19:08.117444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.864618 (+747174us) service_pool.cc:224] Handling call
0709 13:19:08.865064 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.865346 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.865373 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.865981 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:19:09.911084 (+1045103us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:09.911332 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.911585 (+   253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.911738 (+   153us) raft_consensus.cc:1685] finished
0709 13:19:09.911773 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.912226 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":896705,"threads_started":1,"wal-append.queue_time_us":818401}
I20260709 13:19:09.916040 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2810) took 1800 ms. Trace:
I20260709 13:19:09.921815 10047 rpcz_store.cc:276] 0709 13:19:08.115625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.810404 (+694779us) service_pool.cc:224] Handling call
0709 13:19:08.810824 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.812189 (+  1365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.812223 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.821340 (+  9117us) log.cc:844] Serialized 26 byte log entry
0709 13:19:09.915058 (+1093718us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:09.915225 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.915387 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.915530 (+   143us) raft_consensus.cc:1685] finished
0709 13:19:09.915547 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.915917 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":889,"raft.queue_time_us":6728,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":17775,"thread_start_us":964468,"threads_started":1,"wal-append.queue_time_us":946574}
I20260709 13:19:09.600927 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:09.935958 10095 connection.cc:256] RPC call timeout handler was delayed by -1.234887198000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 13:19:09.599779 10056 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.029s	sys 0.028s
W20260709 13:19:09.926725 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.004s
W20260709 13:19:09.939664 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.005s	sys 0.005s
W20260709 13:19:09.940056 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.004s	sys 0.004s
W20260709 13:19:09.940511 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.005s	sys 0.005s
W20260709 13:19:09.941139 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.941901 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.939779 10056 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.029s	sys 0.029s
I20260709 13:19:09.774298 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.945653 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:09.947258 10127 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:19:09.948886 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.010s	sys 0.014s
W20260709 13:19:09.949528 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.011s	sys 0.014s
W20260709 13:19:09.950115 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 611747us
W20260709 13:19:09.950529 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.951851 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:09.953275 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:09.938959 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:09.954632 10051 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:09.957262 10196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:09.958663 10196 raft_consensus.cc:606] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:09.943351 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.959501 10196 raft_consensus.cc:614] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:09.959921 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.960178 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.961361 10201 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:09.964128  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:19:09.967751 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 334748us
I20260709 13:19:09.968775 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:09.970397 10047 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:09.970487 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 409879us
I20260709 13:19:09.976354 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:09.976697 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:09.917907 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:09.989261 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:09.992082 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:09.995132 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:09.996513 10895 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:09.998317 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:09.999922 10894 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:10.004578 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:09.817057 15708 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.005414 15776 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:10.005878 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:10.008023 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.009028 15763 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:10.010303 15763 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:10.015964 15792 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.023213 15813 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:09.850682 15853 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:10.024253 15813 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.025377 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.025297 15813 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.026326 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.028838 15708 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.029026 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2284) took 1948 ms. Trace:
I20260709 13:19:10.030027 15708 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.030177 10048 rpcz_store.cc:276] 0709 13:19:08.080296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.167036 (+ 86740us) service_pool.cc:224] Handling call
0709 13:19:10.028372 (+1861336us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b54002e9354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.028747 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75500288}
I20260709 13:19:10.031534 15853 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.032650 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:09.992213 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:09.999718 10050 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:10.036754 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.034894 15853 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:10.057693 10051 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:10.035149 15708 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.068018 15961 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.008s	sys 0.015s
I20260709 13:19:10.069581 15929 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.071753 16024 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:10.070575 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3383) took 1362 ms (client timeout 1787 ms). Trace:
I20260709 13:19:10.039887 15763 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:10.068027 10096 leader_election.cc:336] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.648s (SENT)
I20260709 13:19:10.083545 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:10.084456 15763 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.086048 16021 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:10.080564 10199 rpcz_store.cc:269] 0709 13:19:08.708335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.776148 (+ 67813us) service_pool.cc:224] Handling call
0709 13:19:10.069968 (+1293820us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.070285 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435524480}
I20260709 13:19:09.851577 15924 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.105094 15708 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.108824 15933 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.112283 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.114084 10051 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:09.853024 15952 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.118549 16015 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.121120 10048 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:10.125283 10047 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:10.100188 15961 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.008s	sys 0.015s
W20260709 13:19:10.127363 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 157891us
I20260709 13:19:10.108873 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:10.136189 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 587923us
I20260709 13:19:10.139938 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.140744 15813 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.143349 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.139804 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.139747 16024 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:10.153496 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44959.4us
I20260709 13:19:10.154711 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.150322 10051 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:19:10.151273 10096 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.361s (SENT)
I20260709 13:19:10.158550 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:10.159547 16015 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:10.160477 16015 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:10.152438 16029 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:10.152748 10129 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:10.171617 16027 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:10.172641 10124 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:10.173434 16034 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:10.174767 16029 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.145625 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 161486us
I20260709 13:19:10.191762 10050 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
W20260709 13:19:10.154382 15705 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.667s	user 0.013s	sys 0.017s
I20260709 13:19:10.162484 16015 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.221879 10048 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:10.222153 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58271.7us
I20260709 13:19:10.227095 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.228287 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.233703 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.234889 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.228370 10048 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.237191 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.238121 15944 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:10.229005 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.024s (SENT)
I20260709 13:19:10.229385 16024 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.254797 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:10.256086 10129 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:10.225140 15929 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:10.260018 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.264451 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76298.8us
I20260709 13:19:10.271998 16024 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:10.271782 16023 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:19:10.271842 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.274238 10128 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:10.275552 16046 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:10.280859 15927 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.620s	user 0.015s	sys 0.008s
I20260709 13:19:10.281459 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.283226 16035 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:10.285758 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:10.287235 16001 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:10.287503 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.287880 10122 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:10.288021 16001 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.289660 10050 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:10.289945 16032 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.291165 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.291965 15844 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.292954 10050 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:10.294304 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.295267 15763 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:10.295888 15763 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:10.296351 15763 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:10.297389 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2808) took 2196 ms. Trace:
I20260709 13:19:10.297928 10055 rpcz_store.cc:276] 0709 13:19:08.101582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.808919 (+707337us) service_pool.cc:224] Handling call
0709 13:19:10.028711 (+1219792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.029342 (+   631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.029372 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.029448 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:10.029492 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.029581 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.297035 (+267454us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002e9350. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.297179 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2759644032}
I20260709 13:19:10.299507 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.301180 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2279) took 2222 ms. Trace:
I20260709 13:19:10.276587 16046 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.291635 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.277714 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2817) took 2160 ms. Trace:
W20260709 13:19:10.311174 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.312528 10050 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:19:10.312350 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.316291 15929 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.318112 10892 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:10.319880 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65347.5us
W20260709 13:19:10.322535 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 182864us
I20260709 13:19:10.290822 16032 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.329989 16032 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:10.331103 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2328) took 1134 ms. Trace:
I20260709 13:19:10.334240 10050 rpcz_store.cc:276] 0709 13:19:09.196348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.293905 (+1097557us) service_pool.cc:224] Handling call
0709 13:19:10.330875 (+ 36970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.336720 10894 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:10.314898 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.315326 15928 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.259387 16029 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.326102 10020 leader_election.cc:400] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:10.301694 10052 rpcz_store.cc:276] 0709 13:19:08.078619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.168765 (+ 90146us) service_pool.cc:224] Handling call
0709 13:19:08.169233 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.449894 (+280661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.300633 (+1850739us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b540010aed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:10.300971 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584910080}
I20260709 13:19:10.337769 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.359854 10020 leader_election.cc:403] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:10.313053 10053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.010s	sys 0.006s
W20260709 13:19:10.370361 10053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.011s	sys 0.006s
I20260709 13:19:10.371722 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.361500 15929 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.380738 10125 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:10.384395 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:10.385512 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161663us
I20260709 13:19:10.384407 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.364755 15928 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.389063 10196 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:10.394227 16046 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:10.396620 15803 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.006s
W20260709 13:19:10.388917 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.313485 10056 rpcz_store.cc:276] 0709 13:19:08.117724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.868850 (+751126us) service_pool.cc:224] Handling call
0709 13:19:08.869493 (+   643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.869786 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:09.944419 (+1074633us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.210162 (+265743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.220944 (+ 10782us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:10.248036 (+ 27092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.257429 (+  9393us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.277631 (+ 20202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":889767,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":2712,"raft.queue_time_us":65369,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":179,"thread_start_us":1235128,"threads_started":3,"wal-append.queue_time_us":275507}
W20260709 13:19:10.402536 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76776.9us
W20260709 13:19:10.401144 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.355870 16035 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:10.418761 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.432061 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.439775 10050 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:10.436830 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.445485 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.443393 10095 leader_election.cc:400] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:10.450084 10095 leader_election.cc:403] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:10.428889 15929 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.501484 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.441618 15774 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.517062 10096 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.297s (SENT)
I20260709 13:19:10.517264 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.518872 10128 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.519205 15924 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.519735 16049 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:10.415232 15803 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.536801 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 100388us
I20260709 13:19:10.538105 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.536639 15929 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.543043 10055 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:10.543743 16038 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:19:10.544143 16010 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.849s	user 0.017s	sys 0.006s
I20260709 13:19:10.544768 16010 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:10.545799 16010 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:10.547308 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 97281.6us
I20260709 13:19:10.547595 16010 consensus_queue.cc:1243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:10.548244 16024 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:10.548537 15803 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:10.548681 16024 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:10.549979 16024 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.551424 15924 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.553094 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.554179 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2340) took 1027 ms. Trace:
I20260709 13:19:10.554553 10050 rpcz_store.cc:276] 0709 13:19:09.526948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.445212 (+918264us) service_pool.cc:224] Handling call
0709 13:19:10.554097 (+108885us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.555083 16024 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:10.555503 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:10.551837 10123 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 9.
I20260709 13:19:10.556509 16029 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.567375 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2341) took 1040 ms. Trace:
I20260709 13:19:10.537087 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:10.480880 16046 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:10.570112 10056 rpcz_store.cc:276] 0709 13:19:09.549034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.425669 (+876635us) service_pool.cc:224] Handling call
0709 13:19:10.567279 (+141610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.571375 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.444072 16035 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.574358 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44687.2us
W20260709 13:19:10.574667 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.576028 16010 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:10.575827 15941 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:10.579653 15924 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.580927 10203 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:10.583844 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 139467us
W20260709 13:19:10.585158 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.585410 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.597913 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.592058 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:10.587457 15763 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.595387 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:10.597682 15977 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.593093 15774 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:10.600296 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.609660 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.586908 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:10.625078 10050 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:19:10.622395 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105262us
I20260709 13:19:10.632571 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:10.635560 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.599320 15945 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.640565 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.636523 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.637836 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.639806 10052 raft_consensus.cc:2410] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:10.660692 16035 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.661705 16035 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:10.663283 16049 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:10.663949 16049 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:10.665078 15774 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:10.665359 16049 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.634658 15924 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:10.667390 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.667846 15993 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.668190 10056 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:19:10.668598 15993 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:10.671918 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71152.8us
I20260709 13:19:10.673149 15786 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:10.675968 15977 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.679508 15763 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.648258 15929 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.683280 10204 raft_consensus.cc:2376] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:19:10.684055 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2218) took 1569 ms. Trace:
I20260709 13:19:10.688890 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.691412 15813 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:10.694674 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.716876 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.635783 16046 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.678931 15977 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.736017 10050 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:10.684767 10200 rpcz_store.cc:276] 0709 13:19:09.114427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.115085 (+   658us) service_pool.cc:224] Handling call
0709 13:19:10.071597 (+956512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.155598 (+ 84001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.155657 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.155758 (+   101us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:10.155817 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.155959 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.683534 (+527575us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b540035acd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.683751 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82964,"raft.queue_time_us":83946,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":260,"spinlock_wait_cycles":2198260096}
I20260709 13:19:10.733731 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.736265 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.662660 16035 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.666982 16015 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:10.749401 10048 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:10.748004 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.751230 10050 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:10.751828 10055 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:10.752501 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2330) took 1313 ms. Trace:
I20260709 13:19:10.752835 10055 rpcz_store.cc:276] 0709 13:19:09.439320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.358389 (+919069us) service_pool.cc:224] Handling call
0709 13:19:10.752412 (+394023us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.754117 10047 raft_consensus.cc:2410] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:10.754077 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2344) took 1158 ms. Trace:
I20260709 13:19:10.755008 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.754719 10048 rpcz_store.cc:276] 0709 13:19:09.591874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.226478 (+634604us) service_pool.cc:224] Handling call
0709 13:19:10.750008 (+523530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.760488 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.766752 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.771323 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:10.782078 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 2814) took 2665 ms (client timeout 2012 ms). Trace:
I20260709 13:19:10.781775 16029 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:10.786298 10131 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.783758 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:10.783075 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2337) took 1257 ms. Trace:
I20260709 13:19:10.789068 10047 rpcz_store.cc:276] 0709 13:19:09.525995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.688125 (+1162130us) service_pool.cc:224] Handling call
0709 13:19:10.782990 (+ 94865us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:10.789652 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56558.7us
W20260709 13:19:10.786801 10054 rpcz_store.cc:269] 0709 13:19:08.116871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.152742 (+ 35871us) service_pool.cc:224] Handling call
0709 13:19:10.781505 (+2628763us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400369554. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.781781 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30760192}
I20260709 13:19:10.799223 10125 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:10.794006 15844 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.004s
I20260709 13:19:10.801496 10128 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:10.793711 15993 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:10.800554 15926 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.821825 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.819814 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 672) took 1049 ms. Trace:
W20260709 13:19:10.827642 16054 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:19:10.829517 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.830410 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.831524 15927 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:10.832096 15927 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.832522 15927 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:10.838080 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.839952 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.840572 10900 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.840862 10126 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:10.841951 10127 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:10.845482 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.847949 16046 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.838207 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.848636 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:10.849505 10052 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:10.823673 10123 rpcz_store.cc:276] 0709 13:19:09.768224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.768417 (+   193us) service_pool.cc:224] Handling call
0709 13:19:10.813262 (+1044845us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54001c8254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.817451 (+  4189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784247168}
I20260709 13:19:10.850157 10127 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:10.850868 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.850965 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:10.851846 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.852595 10124 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:10.852953 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.853757 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.854159 15957 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.855829  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:10.854931 15957 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.857447  9949 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.860227 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.860783 15957 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.797981 16015 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:10.824653 15977 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:10.863869 15977 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:10.864688 16015 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.865669 10049 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:19:10.865324 15977 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.872176 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.873042 10054 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:10.874006 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2312) took 2349 ms. Trace:
I20260709 13:19:10.874442 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.875224 10051 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:10.876921 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 747) took 2367 ms. Trace:
W20260709 13:19:10.887791 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3435) took 1667 ms (client timeout 1760 ms). Trace:
W20260709 13:19:10.888355 10122 rpcz_store.cc:269] 0709 13:19:09.220067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.223292 (+  3225us) service_pool.cc:224] Handling call
0709 13:19:10.887510 (+1664218us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540014a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.887713 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2443974912}
I20260709 13:19:10.889557 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:10.854859 16040 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.704s	user 0.002s	sys 0.000s
I20260709 13:19:10.892145 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:10.866137 10049 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:10.893306 10049 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:19:10.893883 10049 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:10.874500 10054 rpcz_store.cc:276] 0709 13:19:08.524246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.801533 (+2277287us) service_pool.cc:224] Handling call
0709 13:19:10.872635 (+ 71102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.873610 (+   975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.873891 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816}
I20260709 13:19:10.894191 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2222) took 1359 ms. Trace:
I20260709 13:19:10.894742 16057 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.894970 10203 rpcz_store.cc:276] 0709 13:19:09.534431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.537624 (+  3193us) service_pool.cc:224] Handling call
0709 13:19:09.538131 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.581562 (+1043431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.893795 (+312233us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540012a054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:10.894109 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343610112}
I20260709 13:19:10.896607 10047 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:10.897483 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2307) took 2442 ms. Trace:
I20260709 13:19:10.897805 10900 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
W20260709 13:19:10.901015 16012 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.262s	user 0.004s	sys 0.001s
I20260709 13:19:10.903975 16001 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.895299 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:10.905654 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 758) took 1338 ms. Trace:
I20260709 13:19:10.854769 10124 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 10: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:10.895471 16057 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:10.909154 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.910056 10124 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:10.910374 16057 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:10.898658 10900 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 671) took 1699 ms (client timeout 1717 ms). Trace:
W20260709 13:19:10.906594 16009 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.718s	user 0.005s	sys 0.005s
I20260709 13:19:10.911413 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.912793 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.906437 10198 rpcz_store.cc:276] 0709 13:19:09.567474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.754637 (+187163us) service_pool.cc:224] Handling call
0709 13:19:09.762000 (+  7363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.763233 (+  1233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.905095 (+1141862us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b5400354650. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.905401 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15966592}
W20260709 13:19:10.912063 10900 rpcz_store.cc:269] 0709 13:19:09.198925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.199202 (+   277us) service_pool.cc:224] Handling call
0709 13:19:10.839674 (+1640472us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b58005402d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277ffafe7 00007fa277ffc7de 00007fa27800827c 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c
0709 13:19:10.898556 (+ 58882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768664832}
I20260709 13:19:10.890875 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:10.914654 10197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2226) took 1378 ms (client timeout 1538 ms). Trace:
I20260709 13:19:10.915714 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 909) took 3769 ms. Trace:
I20260709 13:19:10.918203 10894 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:19:10.918123 10197 rpcz_store.cc:269] 0709 13:19:09.536073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.550503 (+ 14430us) service_pool.cc:224] Handling call
0709 13:19:10.914284 (+1363781us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54001d0954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:10.914457 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614737152}
I20260709 13:19:10.919039 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2818) took 2801 ms. Trace:
I20260709 13:19:10.918888 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.917724 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:10.920063 10122 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:10.920755 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:10.921589 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.921891 10095 leader_election.cc:400] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:10.922314 15754 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:10.922683 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:10.897832 10047 rpcz_store.cc:276] 0709 13:19:08.476697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.795579 (+2318882us) service_pool.cc:224] Handling call
0709 13:19:10.818009 (+ 22430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.897116 (+ 79107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.897415 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:10.916725 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.923158 15754 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:10.923873 10122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:19:11.176097 10122 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.176707 10122 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:10.915799 15928 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:10.893136 10129 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:10.923960 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:10.924461 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 743) took 2876 ms. Trace:
I20260709 13:19:10.861145 10123 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:10.919533 10053 rpcz_store.cc:276] 0709 13:19:08.118049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.203477 (+1085428us) service_pool.cc:224] Handling call
0709 13:19:09.385084 (+181607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.522762 (+137678us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:10.416604 (+893842us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.756741 (+340137us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.772485 (+ 15744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.772570 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:10.772689 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.918711 (+146022us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400135e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:10.918769 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.918955 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10251,"prepare.queue_time_us":274434,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":146,"raft.queue_time_us":67298,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":45187,"spinlock_wait_cycles":292140416,"thread_start_us":1051477,"threads_started":3,"wal-append.queue_time_us":251665}
I20260709 13:19:10.918938 10898 rpcz_store.cc:276] 0709 13:19:07.146296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.147252 (+   956us) service_pool.cc:224] Handling call
0709 13:19:07.147662 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.237428 (+3089766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.237501 (+    73us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.237593 (+    92us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:10.237656 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.915230 (+677574us) spinlock_profiling.cc:243] Waited 3.07 s on lock 0x7b54001e7154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:10.915276 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.915467 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6703,"raft.queue_time_us":12050,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":10292,"spinlock_wait_cycles":7063398272}
W20260709 13:19:10.947988 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.182577 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.182811 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:10.955912 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 228471us
I20260709 13:19:10.968268 10128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:19:10.920076 10864 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:10.905720 10054 raft_consensus.cc:2376] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14. Re-sending same reply.
I20260709 13:19:11.149063 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:10.877820 10051 rpcz_store.cc:276] 0709 13:19:08.509645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.798218 (+2288573us) service_pool.cc:224] Handling call
0709 13:19:10.874895 (+ 76677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.876464 (+  1569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.876822 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.150676 10131 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:10.913803 10124 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:10.853796 10052 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:11.154472 10056 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:11.188753 10197 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:11.163040 10125 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:11.156757 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.166314 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2819) took 3038 ms. Trace:
I20260709 13:19:11.167944 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.192394 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.192232 10055 rpcz_store.cc:276] 0709 13:19:08.118325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.767829 (+2649504us) service_pool.cc:224] Handling call
0709 13:19:10.768357 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.768741 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:10.769268 (+   527us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.156807 (+387539us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.156881 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:11.156947 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.157020 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.157167 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1895,"prepare.queue_time_us":10509,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":194734,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":13552,"thread_start_us":192221,"threads_started":1}
I20260709 13:19:11.194139 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:11.195027 10053 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:11.196566 10053 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:11.197029 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 748) took 2611 ms. Trace:
I20260709 13:19:11.171613 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.159650 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.156183 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.198485 10204 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:11.173507 10170 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:11.172567 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:11.183012 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.924094 10127 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:11.201479 15810 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:11.180075 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2820) took 3056 ms. Trace:
I20260709 13:19:11.202183 15810 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:11.187578 10126 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:11.202494 10048 rpcz_store.cc:276] 0709 13:19:08.123416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.769877 (+2646461us) service_pool.cc:224] Handling call
0709 13:19:10.770428 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.771094 (+   666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.771167 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.771866 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.965118 (+193252us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:10.965191 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.965305 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.179805 (+214500us) raft_consensus.cc:1685] finished
0709 13:19:11.179833 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.179997 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1694,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":125,"thread_start_us":192682,"threads_started":1,"wal-append.queue_time_us":403274}
I20260709 13:19:11.180640 10049 rpcz_store.cc:276] 0709 13:19:08.048151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.048362 (+   211us) service_pool.cc:224] Handling call
0709 13:19:08.062796 (+ 14434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:10.894572 (+2831776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.896359 (+  1787us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b54001b8354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:10.924247 (+ 27888us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6316814080}
W20260709 13:19:11.205979 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 286037us
I20260709 13:19:11.206307 10048 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:19:11.207154 10048 pending_rounds.cc:85] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:19:11.206975 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:11.207373 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.208645 16033 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:11.208354 10055 rpcz_store.cc:276] 0709 13:19:08.586096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.194178 (+2608082us) service_pool.cc:224] Handling call
0709 13:19:11.194579 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.194894 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.194936 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.195003 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.195045 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.196709 (+  1664us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.196851 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.209628 16033 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:11.210114 16033 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:11.210386 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.036s	sys 0.063s
I20260709 13:19:11.210572 16033 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:11.210793 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.036s	sys 0.063s
I20260709 13:19:11.191254 10128 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.183447 16057 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:11.168149 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:11.197319 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 749) took 1996 ms. Trace:
I20260709 13:19:11.200513 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:11.203475 15810 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.185089 10893 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:11.204553 16066 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.207453 10048 pending_rounds.cc:107] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:19:11.228430 10048 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.211730 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 2783) took 1010 ms. Trace:
I20260709 13:19:11.208971 10124 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:11.228063 15928 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:11.230253 10128 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:11.164880 16024 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.014s
W20260709 13:19:11.222415 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 299795us
I20260709 13:19:11.222648 10053 rpcz_store.cc:276] 0709 13:19:09.218272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.195903 (+1977631us) service_pool.cc:224] Handling call
0709 13:19:11.196360 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.197021 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.197226 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:11.161513 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.235881 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.229059 10125 rpcz_store.cc:276] 0709 13:19:10.201110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.753519 (+552409us) service_pool.cc:224] Handling call
0709 13:19:11.211673 (+458154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.232041 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:11.238196 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.225436  9949 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 18, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:11.231595 15928 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.243891 10047 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:11.245029 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:11.251430 15661 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:11.243415 15786 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:11.254176 15786 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.254698 15786 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:11.254897 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.255141 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2349) took 1345 ms. Trace:
I20260709 13:19:11.255695 10048 rpcz_store.cc:276] 0709 13:19:09.910230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.205524 (+1295294us) service_pool.cc:224] Handling call
0709 13:19:11.205861 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.254773 (+ 48912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.255007 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.253466 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:11.259048 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.257797 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:11.210870 15994 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:19:11.246516 15928 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.265026 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:11.267912 10053 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:11.283727 10125 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:19:11.284390 10125 pending_rounds.cc:85] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:19:11.284781 10125 pending_rounds.cc:107] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:19:11.285081 10125 raft_consensus.cc:2889] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.286561 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:11.282006 10123 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:19:11.258934 16057 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.298002 16035 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:11.295082  9949 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 16, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:11.295456 15813 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:11.283434 16024 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:11.317154 16057 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:11.318198 10130 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:11.292208 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:11.302924 15761 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:11.303573 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2792) took 1029 ms. Trace:
I20260709 13:19:11.242165 16066 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:11.304138 10125 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:19:11.248420 10122 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:11.295478 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 767898us
I20260709 13:19:11.309422 10123 pending_rounds.cc:85] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:19:11.326012 10123 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:11.326438 10123 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.320499 10131 rpcz_store.cc:276] 0709 13:19:10.274197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.189748 (+915551us) service_pool.cc:224] Handling call
0709 13:19:11.241286 (+ 51538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.291569 (+ 50283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.291614 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.303233 (+ 11619us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.303285 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.303356 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.303498 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.316336 15813 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.310112 10054 pending_rounds.cc:85] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:19:11.297210 16029 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:11.324973 10125 pending_rounds.cc:85] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:19:11.341306 10054 pending_rounds.cc:107] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:11.341889 10054 raft_consensus.cc:2889] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.343281 10125 pending_rounds.cc:107] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:19:11.343892 10125 raft_consensus.cc:2889] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:11.350134 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2793) took 1070 ms. Trace:
I20260709 13:19:11.357442 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2790) took 1073 ms. Trace:
I20260709 13:19:11.358084 10123 rpcz_store.cc:276] 0709 13:19:10.273395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.188084 (+914689us) service_pool.cc:224] Handling call
0709 13:19:11.230866 (+ 42782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.346111 (+115245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.346379 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.360029 10123 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:11.360745 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2801) took 1034 ms. Trace:
I20260709 13:19:11.361016 15957 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:11.362572 16058 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:11.363070 16058 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:11.350700 10124 rpcz_store.cc:276] 0709 13:19:10.274587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.189755 (+915168us) service_pool.cc:224] Handling call
0709 13:19:11.208759 (+ 19004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.232745 (+ 23986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.345071 (+112326us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b540023bed0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:11.345269 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40035072}
I20260709 13:19:11.364300 16058 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.365459 10124 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.365826 16058 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:11.366230 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2804) took 1038 ms. Trace:
I20260709 13:19:11.360759 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2803) took 1033 ms. Trace:
I20260709 13:19:11.368403 10050 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 18. Ops: []
I20260709 13:19:11.369377 10047 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:11.370918 10056 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.367862 10131 rpcz_store.cc:276] 0709 13:19:10.327164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.359672 (+1032508us) service_pool.cc:224] Handling call
0709 13:19:11.360067 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.360319 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.360351 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.360433 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:11.360478 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.360546 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.360686 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.361140 10123 rpcz_store.cc:276] 0709 13:19:10.326493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.359173 (+1032680us) service_pool.cc:224] Handling call
0709 13:19:11.359589 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.360448 (+   859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.360669 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.357478 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:11.376264 10122 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:11.366665 10124 rpcz_store.cc:276] 0709 13:19:10.327452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.364877 (+1037425us) service_pool.cc:224] Handling call
0709 13:19:11.365260 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.365944 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.366147 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.370896 10047 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:11.380240 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2800) took 1054 ms. Trace:
I20260709 13:19:11.357293 15661 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:11.400226 15661 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:11.401259 15735 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.401410 15661 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:11.402040 15735 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:11.403548 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111462us
I20260709 13:19:11.404937 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:11.408960 15761 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:11.409901 15761 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:11.377413 16029 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:11.412336 16029 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:11.385201 16066 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:11.385712 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:11.386340 10052 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
W20260709 13:19:11.396113 15927 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.001s
I20260709 13:19:11.397437 15749 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:11.399372 15813 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:11.405001 15797 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:11.423030 15797 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.423520 15797 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.425014 16060 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:11.425700 16060 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:11.427975 15735 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:11.380851 10125 rpcz_store.cc:276] 0709 13:19:10.326100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.300568 (+974468us) service_pool.cc:224] Handling call
0709 13:19:11.303050 (+  2482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.379965 (+ 76915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.380174 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.382797 10128 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:11.422070 15927 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.430743 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45986 (request call id 2784) took 1175 ms. Trace:
I20260709 13:19:11.433632 10122 rpcz_store.cc:276] 0709 13:19:10.201550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.923659 (+722109us) service_pool.cc:224] Handling call
0709 13:19:11.376805 (+453146us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":311}
I20260709 13:19:11.438441 10128 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:11.438273 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:11.439424 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45986 (request call id 2787) took 1236 ms. Trace:
I20260709 13:19:11.439918 10899 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:11.433372 16066 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:11.482601 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2808) took 1068 ms. Trace:
I20260709 13:19:11.439942 10128 rpcz_store.cc:276] 0709 13:19:10.202737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.968063 (+765326us) service_pool.cc:224] Handling call
0709 13:19:11.439328 (+471265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71424}
I20260709 13:19:11.483237 10124 rpcz_store.cc:276] 0709 13:19:10.415821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.421847 (+1006026us) service_pool.cc:224] Handling call
0709 13:19:11.441136 (+ 19289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.447697 (+  6561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.447748 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.482115 (+ 34367us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:11.482189 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.482294 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.482514 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:11.487483 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:11.506102 10047 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:11.499435 15803 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:11.481791 16066 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:11.509577 15803 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.511402 15927 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:11.426944 16060 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:11.515707 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130145us
I20260709 13:19:11.513518 15928 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.524518 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2822) took 1034 ms. Trace:
I20260709 13:19:11.517658 16066 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:11.517838 15803 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:11.527266 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 349761us
W20260709 13:19:11.530316 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 352403us
I20260709 13:19:11.537750 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:11.549764 15786 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:11.553917 15786 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:11.564134 16060 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:11.572916 16033 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.594033 10128 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:19:11.525822 10049 rpcz_store.cc:276] 0709 13:19:10.489788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.205613 (+715825us) service_pool.cc:224] Handling call
0709 13:19:11.205945 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.206243 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:11.206633 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.524074 (+317441us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.524157 (+    83us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:11.524222 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.524302 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.524447 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":185,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":6190,"spinlock_wait_cycles":2944,"thread_start_us":310763,"threads_started":1}
I20260709 13:19:11.574991 15920 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:11.600736 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.880s (SENT)
I20260709 13:19:11.601693 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:11.600191 15797 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.611245 15927 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:11.632817 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.632917 15928 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:11.639273 16067 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:11.601054 15920 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:11.640111 16067 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:11.640592 16067 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:11.641237 15920 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.642805 15920 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:11.644096 16065 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.476s	user 0.005s	sys 0.000s
I20260709 13:19:11.665335 10128 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:11.666373 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2818) took 1036 ms. Trace:
I20260709 13:19:11.666932 10128 rpcz_store.cc:276] 0709 13:19:10.629889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.619411 (+989522us) service_pool.cc:224] Handling call
0709 13:19:11.664434 (+ 45023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.665993 (+  1559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.666290 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:11.684568 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.691648 16058 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.691880 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 162598us
I20260709 13:19:11.692523 16058 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.693300 10019 leader_election.cc:400] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:11.693771 10019 leader_election.cc:403] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.694176 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 745) took 3533 ms. Trace:
I20260709 13:19:11.695381 16035 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:11.699537 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2819) took 1030 ms. Trace:
I20260709 13:19:11.694636 10050 rpcz_store.cc:276] 0709 13:19:08.160744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.789547 (+2628803us) service_pool.cc:224] Handling call
0709 13:19:10.821448 (+ 31901us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.368886 (+547438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.693855 (+324969us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:11.694068 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257364224}
I20260709 13:19:11.702232 10019 leader_election.cc:400] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:11.702664 10019 leader_election.cc:403] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.700090 10128 rpcz_store.cc:276] 0709 13:19:10.651785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.668160 (+1016375us) service_pool.cc:224] Handling call
0709 13:19:11.681618 (+ 13458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.681908 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.681953 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.682018 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:11.682069 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.682151 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.682332 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:11.703301 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.703950 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.706271 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.706981 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.715956 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 676) took 1764 ms. Trace:
I20260709 13:19:11.724105 10128 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:11.728479 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 431822us
I20260709 13:19:11.716744 10130 rpcz_store.cc:276] 0709 13:19:09.941250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.944466 (+  3216us) service_pool.cc:224] Handling call
0709 13:19:10.924738 (+980272us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:11.160664 (+235926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.318686 (+158022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.701960 (+383274us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:11.705484 (+  3524us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54542464}
I20260709 13:19:11.705084 16058 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:11.751719 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.753674 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 158187us
I20260709 13:19:11.717895 16033 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:11.760733 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:11.767757 10898 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:11.768625 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.728427 16035 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:11.772501 10128 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:11.793259 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 386405us
I20260709 13:19:11.811209 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.816473 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:11.819069 10096 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.940s (SENT)
I20260709 13:19:11.825721 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:11.764242 15803 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:11.829284 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:11.833747 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 138623us
I20260709 13:19:11.835664 10198 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:11.844300 10199 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:11.852252 10197 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:19:11.843058 16049 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:11.865619 10019 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:11.842307 15928 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:11.869370 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 605163us
W20260709 13:19:11.872129 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61942.9us
I20260709 13:19:11.877463 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2848) took 1032 ms. Trace:
W20260709 13:19:11.879225 10054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.012s
W20260709 13:19:11.901409 10054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.010s	sys 0.012s
W20260709 13:19:11.888830 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93027.3us
I20260709 13:19:11.901042 10056 rpcz_store.cc:276] 0709 13:19:10.844793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.371894 (+527101us) service_pool.cc:224] Handling call
0709 13:19:11.372250 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.372538 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:11.419738 (+ 47200us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.876916 (+457178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.877005 (+    89us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:11.877075 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.877178 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.877369 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48045,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":68,"raft.queue_time_us":5427,"thread_start_us":433264,"threads_started":1}
I20260709 13:19:11.922749 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.933972 10899 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:11.933972 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:11.934852 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:11.936725 10200 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:11.937099 10200 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:11.937493 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:11.938007 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:11.938707 10205 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:11.938844 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:11.939749 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78394us
W20260709 13:19:11.943336 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.989s (SENT)
I20260709 13:19:11.945921 10203 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:11.930842 10131 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.031s
W20260709 13:19:11.933795 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53271.3us
W20260709 13:19:11.961301 10131 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.000s	sys 0.032s
I20260709 13:19:11.962952 10201 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:11.963465 10201 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:11.968679 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:11.973280 16079 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:19:11.975483 16077 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:19:11.976415 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100216us
I20260709 13:19:11.976779 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2828) took 1316 ms. Trace:
I20260709 13:19:11.977385 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:11.979771 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:11.980023 16082 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:11.982059 10198 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:11.977442 10051 rpcz_store.cc:276] 0709 13:19:10.660362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.238869 (+578507us) service_pool.cc:224] Handling call
0709 13:19:11.256238 (+ 17369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.313776 (+ 57538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.313815 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.314613 (+   798us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.798340 (+483727us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:11.798418 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.798545 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.976461 (+177916us) raft_consensus.cc:1685] finished
0709 13:19:11.976493 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.976687 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26925,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":181,"thread_start_us":483132,"threads_started":1,"wal-append.queue_time_us":658472}
W20260709 13:19:11.979816 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50635us
I20260709 13:19:11.992796 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:11.922670 16024 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:19:11.984534 16078 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:19:12.004758 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.005718 10055 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:12.009534 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:12.010018 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:12.010306 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.011255 10056 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:12.009929 10891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.006s	sys 0.031s
W20260709 13:19:12.012287 10891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.031s
I20260709 13:19:12.014549 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.008745 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.015334 10050 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
W20260709 13:19:12.021708 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.027s (SENT)
I20260709 13:19:12.015591 10049 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
W20260709 13:19:12.022955 10122 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.020s	sys 0.065s
I20260709 13:19:12.032521 10898 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:12.045275 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:12.044924 15810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.006s	sys 0.037s
I20260709 13:19:12.067032 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:12.081918 10893 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:12.074026 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.083550 10055 raft_consensus.cc:2410] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:19:12.008621 10201 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.095670 10203 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:12.100420 10201 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:12.085237 15810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.006s	sys 0.038s
I20260709 13:19:12.102784 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2797) took 1798 ms. Trace:
I20260709 13:19:12.128167 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2794) took 1825 ms. Trace:
I20260709 13:19:12.129213 10056 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:19:12.128934 10127 rpcz_store.cc:276] 0709 13:19:10.302993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.201557 (+898564us) service_pool.cc:224] Handling call
0709 13:19:11.201958 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.247539 (+ 45581us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.247569 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.253534 (+  5965us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.535695 (+282161us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:11.539983 (+  4288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.540103 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.127793 (+587690us) raft_consensus.cc:1685] finished
0709 13:19:12.127829 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.128067 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37532,"raft.queue_time_us":40173,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":18529,"thread_start_us":281631,"threads_started":1,"wal-append.queue_time_us":556987}
I20260709 13:19:12.115911 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:12.144418 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2834) took 1481 ms. Trace:
I20260709 13:19:12.145418 10054 rpcz_store.cc:276] 0709 13:19:10.662600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.305839 (+643239us) service_pool.cc:224] Handling call
0709 13:19:11.306276 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.363830 (+ 57554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.363863 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.364395 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.902680 (+538285us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.902746 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.902856 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.143915 (+241059us) raft_consensus.cc:1685] finished
0709 13:19:12.143952 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.144284 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":537295,"threads_started":1,"wal-append.queue_time_us":754370}
I20260709 13:19:12.134505 15977 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:12.163848 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111737us
W20260709 13:19:12.164788 16085 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:19:12.165143 10019 leader_election.cc:400] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:12.166483 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:12.167503 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2835) took 1504 ms. Trace:
I20260709 13:19:12.167966 10048 rpcz_store.cc:276] 0709 13:19:10.662973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.307198 (+644225us) service_pool.cc:224] Handling call
0709 13:19:11.307575 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.347680 (+ 40105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.347740 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.363830 (+ 16090us) log.cc:844] Serialized 26 byte log entry
0709 13:19:11.842222 (+478392us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:11.842277 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.842368 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.167222 (+324854us) raft_consensus.cc:1685] finished
0709 13:19:12.167246 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.167430 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17920,"raft.queue_time_us":31543,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":17523,"thread_start_us":477935,"threads_started":1,"wal-append.queue_time_us":649204}
W20260709 13:19:12.169973 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.119516 16075 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
W20260709 13:19:12.171434 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 160888us
I20260709 13:19:12.173434 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2795) took 1870 ms. Trace:
W20260709 13:19:12.173170 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.104944 10126 rpcz_store.cc:276] 0709 13:19:10.304045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.211822 (+907777us) service_pool.cc:224] Handling call
0709 13:19:11.246605 (+ 34783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.286577 (+ 39972us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:11.683469 (+396892us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.102320 (+418851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.102388 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:12.102442 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.102528 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.102709 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":630935,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":8669,"thread_start_us":709862,"threads_started":2}
I20260709 13:19:12.093705 16030 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.247s
W20260709 13:19:12.189198 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.196659 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 201982us
W20260709 13:19:12.201086 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 215187us
I20260709 13:19:12.174312 10129 rpcz_store.cc:276] 0709 13:19:10.303357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.204300 (+900943us) service_pool.cc:224] Handling call
0709 13:19:11.204639 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.205044 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:11.704021 (+498977us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.173004 (+468983us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.173067 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:12.173122 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.173197 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.173349 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":666230,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":845364,"threads_started":2}
I20260709 13:19:12.172353 15786 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.197052 16067 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.004s
I20260709 13:19:12.118569 15985 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.265s
W20260709 13:19:12.259409 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.059s (SENT)
I20260709 13:19:12.260182 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:12.264571 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57572.7us
W20260709 13:19:12.271170 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.272709 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2811) took 1704 ms. Trace:
I20260709 13:19:12.273705 10124 rpcz_store.cc:276] 0709 13:19:10.570033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.484620 (+914587us) service_pool.cc:224] Handling call
0709 13:19:11.502709 (+ 18089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.509168 (+  6459us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:12.046576 (+537408us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.270646 (+224070us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.270739 (+    93us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:12.270827 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.270945 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.272298 (+  1353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3895,"prepare.queue_time_us":466507,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":6288,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"thread_start_us":658382,"threads_started":2}
I20260709 13:19:12.252175 15786 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.258023 16067 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.333961 15810 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:19:12.364324 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 387011us
I20260709 13:19:12.365357 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:12.366865 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.368085 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:12.363247 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2850) took 1517 ms. Trace:
I20260709 13:19:12.370886 10204 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:12.370316 10053 rpcz_store.cc:276] 0709 13:19:10.845454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.388842 (+543388us) service_pool.cc:224] Handling call
0709 13:19:11.389279 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.389558 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:11.875160 (+485602us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.362629 (+487469us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.362726 (+    97us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:12.362815 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.362922 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.363150 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31081,"prepare.queue_time_us":39346,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":35,"thread_start_us":869712,"threads_started":3}
I20260709 13:19:12.348354 16082 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:12.389642 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 137110us
I20260709 13:19:12.390581 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2805) took 2062 ms. Trace:
I20260709 13:19:12.391314 10131 rpcz_store.cc:276] 0709 13:19:10.328153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.371997 (+1043844us) service_pool.cc:224] Handling call
0709 13:19:11.372419 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.372774 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:11.970928 (+598154us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.389967 (+419039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.390061 (+    94us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:12.390159 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.390272 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.390488 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":560076,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"thread_start_us":912300,"threads_started":3}
I20260709 13:19:12.359103 15797 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:12.337294 16107 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:19:12.399250 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2807) took 2031 ms. Trace:
I20260709 13:19:12.371481 16088 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:12.400230 10123 rpcz_store.cc:276] 0709 13:19:10.370404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.373779 (+1003375us) service_pool.cc:224] Handling call
0709 13:19:11.418483 (+ 44704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.448563 (+ 30080us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:11.938025 (+489462us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.398630 (+460605us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.398810 (+   180us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:12.398878 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.398988 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.399175 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59510,"prepare.queue_time_us":453623,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":5161,"raft.queue_time_us":91613,"raft.run_wall_time_us":10010,"spinlock_wait_cycles":22707072,"thread_start_us":744664,"threads_started":3,"wal-append.queue_time_us":371847}
W20260709 13:19:12.399647 15920 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.756s	user 0.012s	sys 0.012s
W20260709 13:19:12.413808 16090 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:19:12.414906 16067 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:12.418430 16035 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:12.434243 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 267884us
I20260709 13:19:12.453424 10201 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:19:12.418562 16092 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:19:12.400552 16087 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:12.456341 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.460538 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92569.6us
W20260709 13:19:12.461282 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.439623 10096 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.147s (SENT)
I20260709 13:19:12.438383 10200 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.476933 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:12.479643 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 402356us
I20260709 13:19:12.432859 15786 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:12.427460 15977 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:12.409497 16117 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:12.432579 15813 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:19:12.505116 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.811s (SENT)
I20260709 13:19:12.485548 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:12.478767 16113 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:19:12.459944 16082 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.460470 16098 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.004s
I20260709 13:19:12.535874 16035 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:12.536448 16035 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.458957 16029 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:19:12.536800 15928 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:12.537562 15928 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:12.538164 15754 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:12.538764 15928 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.543745 16123 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:19:12.546351 16123 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:12.536839 16035 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:12.547284 16100 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
W20260709 13:19:12.548314 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58553.8us
I20260709 13:19:11.921707  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:12.552058 10200 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.555364  9949 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:12.557536  9949 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:12.558174 15797 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:12.517151 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2816) took 1917 ms. Trace:
I20260709 13:19:12.559751 16122 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:19:12.562469 16066 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.020s	user 0.015s	sys 0.009s
I20260709 13:19:12.563201 16066 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.537582 15803 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:12.543785 16105 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:12.566062 16105 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:12.566265 16066 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:12.567198 16105 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.568779 16105 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.569532 16089 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:12.569839 15803 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:12.570230 16089 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:12.570825 16124 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:12.571380 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 131901us
W20260709 13:19:12.572429 10096 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.243s (SENT)
I20260709 13:19:12.572988 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:12.460738 16109 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.057s	user 0.002s	sys 0.001s
I20260709 13:19:12.439965 16088 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:12.558477 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.563422 15761 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:12.475042 16101 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.002s
I20260709 13:19:12.550714 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:12.521806 16102 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.004s
W20260709 13:19:12.587864 10893 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.004s
W20260709 13:19:12.480258 16120 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.054s	user 0.004s	sys 0.000s
W20260709 13:19:12.528918 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 43578.4us
W20260709 13:19:12.579816 16060 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.015s	user 0.014s	sys 0.006s
I20260709 13:19:12.559168 10122 rpcz_store.cc:276] 0709 13:19:10.599686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.497863 (+898177us) service_pool.cc:224] Handling call
0709 13:19:11.498294 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.505463 (+  7169us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:12.043696 (+538233us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.514793 (+471097us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.514900 (+   107us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:12.514984 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.516752 (+  1768us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.517016 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14,"prepare.queue_time_us":477695,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":41,"raft.queue_time_us":37841,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6426,"thread_start_us":910530,"threads_started":3,"wal-append.queue_time_us":309509}
I20260709 13:19:12.522130 16024 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:12.598415 10893 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:12.507884 15813 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:12.606066 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.605248  9949 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 17 to 18, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:12.606241 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2817) took 1996 ms. Trace:
I20260709 13:19:12.606725 15761 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:12.607049 15797 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:12.589318 16082 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:12.610143 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.613317 10125 rpcz_store.cc:276] 0709 13:19:10.624127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.614850 (+990723us) service_pool.cc:224] Handling call
0709 13:19:11.623782 (+  8932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:11.624392 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.624436 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:11.696710 (+ 72274us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.107759 (+411049us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:12.107807 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.107887 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:12.605863 (+497976us) raft_consensus.cc:1685] finished
0709 13:19:12.605895 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.606118 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":456,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":21579,"thread_start_us":336670,"threads_started":1,"wal-append.queue_time_us":575312}
I20260709 13:19:12.592171 16029 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:12.547068 16123 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:12.626250 10205 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.687s	user 0.017s	sys 0.005s
I20260709 13:19:12.646260 10205 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:12.647526 10198 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:19:12.594156 16087 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:19:12.618391 16112 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:19:12.652503  9949 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 11 to 13, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 13 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:12.616390 16117 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:12.659664 16118 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:19:12.669986 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119415us
W20260709 13:19:12.674889 10202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.025s
W20260709 13:19:12.681577 10202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.006s	sys 0.025s
I20260709 13:19:12.675218 10893 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:12.683341 10893 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 12
W20260709 13:19:12.680078 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 305387us
I20260709 13:19:12.683594 10893 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:12.683988 10893 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:12.686146 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:12.687027 10122 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:12.686220 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:12.657411 16060 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:12.664731 16033 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.692406 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:12.696349 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:12.697425 10123 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:12.705025 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 220148us
I20260709 13:19:12.706454 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:12.721748 10126 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:12.716297 10131 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:12.716217 10204 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:12.683418 16121 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.740070 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 750) took 1131 ms. Trace:
I20260709 13:19:12.742463 10200 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:12.741304 10052 rpcz_store.cc:276] 0709 13:19:11.618914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.984978 (+366064us) service_pool.cc:224] Handling call
0709 13:19:12.005915 (+ 20937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.031995 (+ 26080us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:12.099273 (+ 67278us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.338850 (+239577us) log.cc:844] Serialized 14 byte log entry
0709 13:19:12.739514 (+400664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:12.739605 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:12.739689 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.739798 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.739978 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11546,"prepare.queue_time_us":66943,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":107,"raft.queue_time_us":162836,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":153,"thread_start_us":601936,"threads_started":3,"wal-append.queue_time_us":515147}
I20260709 13:19:12.744711 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:12.746752 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:12.738085 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:12.754422 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.075s (SENT)
I20260709 13:19:12.754855 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:12.756999 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 50692.8us
I20260709 13:19:12.723312 16087 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:12.755146 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:12.713011 16033 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:12.751842 15761 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:19:12.768316  9949 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:12.751749 16121 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.773136 15985 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:12.774163 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:12.776217 16122 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:12.779490  9954 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 221) took 57126 ms (client timeout 15000 ms). Trace:
I20260709 13:19:12.788200 10122 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:12.792155 10124 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:12.705852 16060 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:12.738457 15920 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:19:12.746299 16001 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:19:12.687697 15813 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:12.811991 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 237085us
I20260709 13:19:12.829433 10899 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
W20260709 13:19:12.835073 16114 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:19:12.846254  9949 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 13, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 13 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:12.705193 16123 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:12.827266 16058 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:12.851717 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:12.858067 10122 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:12.858475 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 172673us
I20260709 13:19:12.808951 15810 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:12.860178 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:12.861156 10124 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:12.832279 16088 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:12.862792 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 69517.5us
I20260709 13:19:12.861107 16087 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:12.888799 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:12.889770 10129 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:12.891706 10129 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:12.893504 16033 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:12.893179 15810 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.845369 16001 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:12.903024 10863 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.216s (SENT)
I20260709 13:19:12.878510  9949 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 15, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 15 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:12.904723 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:12.894694 16033 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:12.896276 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 122209us
I20260709 13:19:12.924011 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:12.925032 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.925400 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:12.925935 10052 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:12.927069 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:12.927227 16001 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:12.927701 16132 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 13:19:12.935690 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:12.921991 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:12.938019 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:12.938433 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:12.950857 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:12.929999 16123 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:12.928586 16118 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:12.947394 16126 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 13:19:12.910012 16033 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:12.948858 16117 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:19:12.949043 16115 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:12.957437 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:12.961464 16148 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:12.800762  9954 rpcz_store.cc:269] 0709 13:18:15.637644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.637875 (+   231us) service_pool.cc:224] Handling call
0709 13:19:12.752476 (+57114601us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:03.067325 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:03.119740 (+ 52415us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:03.158826 (+ 39086us) write_op.cc:435] Acquired schema lock
0709 13:19:03.161637 (+  2811us) tablet.cc:662] Decoding operations
0709 13:19:03.348960 (+187323us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:03.349064 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:03.349113 (+    49us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:19:03.350868 (+  1755us) tablet.cc:701] Row locks acquired
0709 13:19:03.350945 (+    77us) write_op.cc:260] PREPARE: finished
0709 13:19:03.353325 (+  2380us) write_op.cc:270] Start()
0709 13:19:03.353510 (+   185us) write_op.cc:276] Timestamp: P: 1783603143351085 usec, L: 0
0709 13:19:03.353534 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:03.354425 (+   891us) log.cc:844] Serialized 7854 byte log entry
0709 13:19:03.393919 (+ 39494us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:19:04.167090 (+773171us) op_driver.cc:464] REPLICATION: finished
0709 13:19:04.889655 (+722565us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b1800045a60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:04.889869 (+   214us) write_op.cc:301] APPLY: starting
0709 13:19:04.890149 (+   280us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:05.372460 (+482311us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:05.372525 (+    65us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:05.648560 (+276035us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:05.649021 (+   461us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:19:05.649078 (+    57us) write_op.cc:312] APPLY: finished
0709 13:19:05.649785 (+   707us) log.cc:844] Serialized 180 byte log entry
0709 13:19:06.288643 (+638858us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:06.319726 (+ 31083us) write_op.cc:454] Released schema lock
0709 13:19:06.333310 (+ 13584us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:06.480622 (+147312us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:06.488146 (+  7524us) write_op.cc:454] Released schema lock
0709 13:19:07.516985 (+1028839us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
Metrics: {"mutex_wait_us":181577,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":88985,"apply.run_cpu_time_us":64257,"apply.run_wall_time_us":1490031,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87567,"num_ops":20,"prepare.queue_time_us":362467,"prepare.run_cpu_time_us":20900,"prepare.run_wall_time_us":764162,"raft.queue_time_us":130458,"raft.run_cpu_time_us":31848,"raft.run_wall_time_us":85663,"replication_time_us":813265,"spinlock_wait_cycles":520527104,"thread_start_us":1528581,"threads_started":5,"wal-append.queue_time_us":1576003,"wal-append.run_cpu_time_us":70249,"wal-append.run_wall_time_us":3282267}]]}
I20260709 13:19:12.964432 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:12.964515 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:12.966532 10054 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:19:12.969770 16089 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:12.957543 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:12.978552 10126 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:12.914235 16133 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:12.943078 10122 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10.
I20260709 13:19:13.045666 10055 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:19:13.019030 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.020084 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:13.054767 10049 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:13.025564 10047 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:12.953526 15985 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:13.028694 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:13.011998 16145 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.035379 10052 raft_consensus.cc:2376] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10. Re-sending same reply.
I20260709 13:19:13.031445 10053 raft_consensus.cc:2376] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20. Re-sending same reply.
I20260709 13:19:13.060937 10050 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.032225 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.061666 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.070987 10122 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:13.079475 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:13.019626 16082 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:13.084839 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.085860 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:13.033205 16115 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:13.059989 16128 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 13:19:13.088953 10894 raft_consensus.cc:2368] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
I20260709 13:19:13.098127 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:13.103261 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:13.057956 15985 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:13.104329 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 166358us
I20260709 13:19:13.029944 16066 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:19:13.141270 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2255) took 1155 ms. Trace:
I20260709 13:19:13.075963 16118 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:13.091094 16115 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.153506 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.149202 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.092326 16060 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.171042 15813 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:13.151034 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:13.158015 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.154554 10054 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:13.083308 16145 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:13.160681 16131 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:19:13.162504 16127 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:13.163017 16136 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:19:13.145052 10202 rpcz_store.cc:276] 0709 13:19:11.986248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.986413 (+   165us) service_pool.cc:224] Handling call
0709 13:19:12.012466 (+ 26053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.012799 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.012836 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.066466 (+ 53630us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.682030 (+615564us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:12.682085 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.682184 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.140959 (+458775us) raft_consensus.cc:1685] finished
0709 13:19:13.140984 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.141197 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":510481,"threads_started":1,"wal-append.queue_time_us":702542}
W20260709 13:19:13.164086 16142 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:13.166594 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.167346 16082 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.167565 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:13.162580 16138 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:19:13.073365 16133 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.139732 16121 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.144521 15985 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:13.159296 16118 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:13.157034 16089 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:13.042440 16087 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.187026 15927 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:13.241266 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.243651 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.278398 10052 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:13.278710 10122 raft_consensus.cc:2376] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10. Re-sending same reply.
W20260709 13:19:13.285156 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 327783us
I20260709 13:19:13.217761 16082 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:13.297854 16035 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.000s
I20260709 13:19:13.213799 16058 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:13.242609 16118 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:13.235918 16060 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:13.313179 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.324739 10050 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:13.158447 16033 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.328768 10056 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:19:13.306752 16151 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:19:13.341049 10122 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:13.298717 16035 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:13.349820 16153 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:19:13.353451 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:13.354334 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.303246 16148 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:13.222743 16127 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.366484 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:13.382640 10204 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.379998 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.359527 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:13.364666 16148 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:13.388264 16150 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:19:13.365140 16035 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:13.385116 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:13.389882 10125 raft_consensus.cc:2376] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 10. Re-sending same reply.
I20260709 13:19:13.332792 16058 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:13.395824 16058 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.399262 10204 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:13.378458 16127 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:13.404114 16105 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.835s	user 0.013s	sys 0.005s
I20260709 13:19:13.407274 16133 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.409471 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:13.357666 16152 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:19:13.417449 10049 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:13.421420 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:13.440626 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.444777 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.445978 10131 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:13.441908 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:13.382097 16082 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:13.456583 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 121624us
I20260709 13:19:13.313373 15927 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.421137 16121 raft_consensus.cc:2764] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:19:13.452225 16066 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:13.483100 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.505715 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:13.470727 15927 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.506654 16035 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:13.488806 16066 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:13.531095 16117 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:13.552970 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.550704 16118 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.558108 16118 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.559362 16133 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:13.563000 16166 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:19:13.565009 16105 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:13.565800 16147 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:13.565279 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:13.554813 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.569850 16167 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:13.572454 16160 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:13.574389 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:13.577291 16172 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:13.578284 16118 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:13.579411 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.576480 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:13.539846 16145 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.583997 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:13.604960 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 116052us
I20260709 13:19:13.608541 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:13.611280 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:13.551990 16117 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:13.618582 10900 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:13.614053 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:13.615803 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:13.612228 10203 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:13.624929 10170 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:13.611366 16160 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:13.622303 16117 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.626816 16161 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:13.625023 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:13.652209 16168 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:19:13.590607 16118 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:13.650985 15810 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:13.656319 16087 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:13.663372 16159 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:19:13.665437 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:13.661319 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 82005.3us
I20260709 13:19:13.684321 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.663956 16163 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:13.678824 16118 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.664584 16161 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:13.676566 16145 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:13.712425 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.612969 16115 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:13.718019 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 149165us
I20260709 13:19:13.671458 15786 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:13.682575 16087 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:13.733724 10122 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:13.690706 15810 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:13.741743 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:13.744086 10891 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:19:13.766975 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 82782.6us
I20260709 13:19:13.768570 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:13.716425 16161 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.771845 10126 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:13.772843 10096 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.838s (SENT)
I20260709 13:19:13.772918 10048 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:13.731499 16160 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:13.716743 16163 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:13.783725 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.773880 16159 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:19:13.736205 16087 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:13.777288 10196 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:13.685154 16066 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.776154 15810 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:13.784795 16035 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:13.821189 16060 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:13.821913 16060 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:13.836941 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111388us
I20260709 13:19:13.855754 15810 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.860771 16115 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:13.862428 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89665.1us
W20260709 13:19:13.863202 10096 connection.cc:256] RPC call timeout handler was delayed by -2.3697331020000547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:19:13.873937 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75175.2us
W20260709 13:19:13.829857 16108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.035s	sys 0.070s
I20260709 13:19:13.875936 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:13.867920 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:13.886720 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.308s (SENT)
I20260709 13:19:13.887343 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:13.878490 16108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.035s	sys 0.070s
I20260709 13:19:13.889081 16148 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:13.889564 10096 connection.cc:256] RPC call timeout handler was delayed by -2.3391331020000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 13:19:13.957229 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.957855 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.958209 10096 connection.cc:256] RPC call timeout handler was delayed by -2.0998276400000577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40080000000000005s for a response
W20260709 13:19:13.958678 10096 connection.cc:256] RPC call timeout handler was delayed by -2.0062276399999064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4112s for a response
I20260709 13:19:13.890102 15813 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:13.887845 16181 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:13.897954 16089 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.618s	user 0.003s	sys 0.009s
W20260709 13:19:13.901307 16178 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.304s	user 0.005s	sys 0.000s
I20260709 13:19:13.965530 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:13.902384 16179 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.304s	user 0.002s	sys 0.000s
W20260709 13:19:13.912938 16123 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.916s	user 0.014s	sys 0.002s
I20260709 13:19:13.913565 16087 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:13.886379 16115 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:13.916577 15985 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.002s
I20260709 13:19:13.920157 16033 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.922055 16119 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:13.983402 16119 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:13.984506 16119 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.985872 16119 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:13.871858 16175 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:19:13.860954 16163 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.989355 10049 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:13.959101 10096 connection.cc:256] RPC call timeout handler was delayed by -1.9769423900002039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:19:13.991386 10096 connection.cc:256] RPC call timeout handler was delayed by -1.8806423900000482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:19:13.991777 10049 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:13.991816 10096 connection.cc:256] RPC call timeout handler was delayed by -1.7960423900001388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:19:13.992605 10096 connection.cc:256] RPC call timeout handler was delayed by -1.7636423900000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
W20260709 13:19:13.992908 10096 connection.cc:256] RPC call timeout handler was delayed by -1.7199331020001409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2931s for a response
W20260709 13:19:13.993238 10096 connection.cc:256] RPC call timeout handler was delayed by -1.6295423900000969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:19:13.993300 16060 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:13.967144 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80789.1us
W20260709 13:19:13.927060 16180 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.328s	user 0.002s	sys 0.000s
W20260709 13:19:13.960608 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85613.7us
I20260709 13:19:13.995468 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:13.995944 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 761) took 1002 ms. Trace:
I20260709 13:19:13.996337 10891 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:13.932796 16058 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.996960 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:13.997323 16058 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:13.998064 16087 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:13.998638 16066 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.998898 16108 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:13.999982 16058 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:13.977578 16115 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:13.931758 16144 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.999325 16066 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.000777 16117 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.001623 16066 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.002192 10049 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:14.001611 16144 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.004479 10895 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:13.978459 15985 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.007694 15985 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:14.000886 16115 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:14.010317 16030 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:14.011446 10124 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:14.013481 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 686) took 1001 ms. Trace:
I20260709 13:19:14.013945 10124 rpcz_store.cc:276] 0709 13:19:13.012152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.019681 (+  7529us) service_pool.cc:224] Handling call
0709 13:19:14.013244 (+993563us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54002efc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:14.013371 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163206016}
W20260709 13:19:13.993939 10096 connection.cc:256] RPC call timeout handler was delayed by -1.4459423900000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2257s for a response
I20260709 13:19:14.001017 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:14.015547 10096 connection.cc:256] RPC call timeout handler was delayed by -1.3982423900001777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.231s for a response
W20260709 13:19:14.016070 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2299042849999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 13:19:14.016570 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2056042849999358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 13:19:14.016997 10096 connection.cc:256] RPC call timeout handler was delayed by -1.1300042850000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:19:13.996475 10050 rpcz_store.cc:276] 0709 13:19:12.993728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.325819 (+332091us) service_pool.cc:224] Handling call
0709 13:19:13.326209 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.342325 (+ 16116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.342447 (+   122us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.378920 (+ 36473us) log.cc:844] Serialized 52 byte log entry
0709 13:19:13.599485 (+220565us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:13.720252 (+120767us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.721337 (+  1085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.721424 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.926366 (+204942us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:13.995586 (+ 69220us) raft_consensus.cc:1685] finished
0709 13:19:13.995700 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.995875 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27268,"prepare.queue_time_us":36294,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":8057,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":78020,"spinlock_wait_cycles":66621312,"thread_start_us":312504,"threads_started":2,"wal-append.queue_time_us":219898}
I20260709 13:19:14.043663 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.049546 16144 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:14.050213 10096 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.713s (SENT)
I20260709 13:19:14.051095 16001 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:14.051573 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 676) took 1322 ms. Trace:
W20260709 13:19:14.051734 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.868s (SENT)
I20260709 13:19:14.051995 10899 rpcz_store.cc:276] 0709 13:19:12.729585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.729731 (+   146us) service_pool.cc:224] Handling call
0709 13:19:14.051368 (+1321637us) spinlock_profiling.cc:243] Waited 72.4 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:14.051506 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166588288}
I20260709 13:19:14.042078 16160 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:14.053524 16118 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.054232 16160 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.054847 16160 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.056219 16118 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:14.060042 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.061059 16133 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:14.061206 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:14.062749 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.063956 16123 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:14.052325 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:14.070438 10895 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:14.062815 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:14.050558 16161 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.042791 16121 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:14.090888 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.069890 16123 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:14.083544 16118 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:14.088409 16116 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.088991 16117 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:14.108165 16035 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:14.140157 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49358.5us
I20260709 13:19:14.110072 16029 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:14.143842 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.146579 16123 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:14.157850 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 106155us
W20260709 13:19:14.160661 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.904s (SENT)
I20260709 13:19:14.124536 15813 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:14.139678 16181 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:14.127558 16118 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:14.175761 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113002us
I20260709 13:19:14.167953 16060 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:14.165866 16195 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:14.139544 16116 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.170639 16035 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:14.174731 16181 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.175037 16118 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:14.140910 16133 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:14.194253 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83796.9us
I20260709 13:19:14.198415 16181 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:14.206704 16149 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.247s	user 0.001s	sys 0.007s
I20260709 13:19:14.208657 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2835) took 1510 ms. Trace:
I20260709 13:19:14.209127 10123 rpcz_store.cc:276] 0709 13:19:12.697964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.741381 (+ 43417us) service_pool.cc:224] Handling call
0709 13:19:12.766944 (+ 25563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.794018 (+ 27074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.794081 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.799717 (+  5636us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.890852 (+ 91135us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:12.890895 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.890971 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.205877 (+1314906us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:14.208274 (+  2397us) raft_consensus.cc:1685] finished
0709 13:19:14.208301 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.208576 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16178,"raft.queue_time_us":16427,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":9174,"spinlock_wait_cycles":120426752,"thread_start_us":52879,"threads_started":1,"wal-append.queue_time_us":134161}
W20260709 13:19:14.210983 10096 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.243s (SENT)
I20260709 13:19:14.211625 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:14.190590 16195 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:14.228964 16171 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.610s	user 0.000s	sys 0.003s
I20260709 13:19:14.230736 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2873) took 1230 ms. Trace:
W20260709 13:19:14.231997 16177 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.606s	user 0.002s	sys 0.001s
I20260709 13:19:14.231274 10047 rpcz_store.cc:276] 0709 13:19:13.000239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.334660 (+334421us) service_pool.cc:224] Handling call
0709 13:19:13.350382 (+ 15722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:13.350909 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.350949 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:13.351651 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:19:13.472986 (+121335us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:13.473068 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.473155 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.228250 (+755095us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:14.230309 (+  2059us) raft_consensus.cc:1685] finished
0709 13:19:14.230343 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.230652 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1211,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"spinlock_wait_cycles":80071680,"thread_start_us":120763,"threads_started":1,"wal-append.queue_time_us":194143}
I20260709 13:19:14.234287 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2872) took 1236 ms. Trace:
I20260709 13:19:14.236605 16133 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.234902 10056 rpcz_store.cc:276] 0709 13:19:12.999470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.330370 (+330900us) service_pool.cc:224] Handling call
0709 13:19:13.415072 (+ 84702us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.479475 (+ 64403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.479519 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.480453 (+   934us) log.cc:844] Serialized 52 byte log entry
0709 13:19:13.584827 (+104374us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:13.609580 (+ 24753us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.610482 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.610572 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.231259 (+620687us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:14.233782 (+  2523us) raft_consensus.cc:1685] finished
0709 13:19:14.233928 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.234199 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6102,"prepare.queue_time_us":27765,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":25287,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":12392,"spinlock_wait_cycles":6805248,"thread_start_us":74007,"threads_started":1,"wal-append.queue_time_us":136296}
I20260709 13:19:14.216902 16195 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:14.259315 16198 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:19:14.269703 16203 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:19:14.271821 16193 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:19:14.292127 16200 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.293139 16200 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.294833 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:14.295737 16200 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:14.296590 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:14.297298 10051 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:14.301698 16001 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:14.303104 16001 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:14.303596 16001 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:14.306792 16001 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.308024 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:14.308943 16001 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.309201 15761 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:14.309959 15761 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:19:14.290853 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.306844 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.331377 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 128599us
I20260709 13:19:14.310424 15761 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.261024 15920 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.354888 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:14.367688 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:14.372390 10054 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:14.347496 16189 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.160s	user 0.001s	sys 0.001s
I20260709 13:19:14.375151 10051 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:14.334455 16163 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.386247 10054 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:14.386224 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:14.398715 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:14.360734 16161 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:14.347078 16121 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.411023 16121 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.278939 16115 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.402843 10899 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:14.403584 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:14.409397 10020 leader_election.cc:400] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
I20260709 13:19:14.417126 16115 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:14.417529 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:14.441383 10054 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:14.443295 15810 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:14.444067 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:14.444538 16060 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.444938 10050 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:14.282387 16035 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.369258 15920 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.456413 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:14.399093 15813 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:14.406010 16208 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:14.467912 15813 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:14.469211 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 258282us
I20260709 13:19:14.458613 16058 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:19:14.462174 16197 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:14.406720 16133 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.473881 10053 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:14.473801 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:14.397898 16030 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
W20260709 13:19:14.503934 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.257s (SENT)
I20260709 13:19:14.480204 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:14.495544 10051 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:14.458841 16115 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:14.396965 16163 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.529312 10050 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:14.490538 16089 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.455528 16060 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.505496 16116 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:14.401609 16066 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:14.595489 16180 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:19:14.598166 16192 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 13:19:14.519608 16133 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:14.611634 16213 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 13:19:14.531086 16197 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:14.614917 16119 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.629s	user 0.020s	sys 0.000s
W20260709 13:19:14.616101 16160 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.009s
I20260709 13:19:14.619650 16024 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.621456 16163 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:14.519199 16115 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:14.530783 15754 raft_consensus.cc:476] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Not starting pre-election -- already a leader
W20260709 13:19:14.622680 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 130400us
I20260709 13:19:14.521944 16167 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:14.478297 16132 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
W20260709 13:19:14.664068 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.658979 16214 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.699225 10131 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:14.724740 10049 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:14.619632 16195 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.668793 16089 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:14.755311 10096 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.310s (SENT) [suppressed 39 similar messages]
W20260709 13:19:14.785539 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 307032us
I20260709 13:19:14.750931 16160 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.796665 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.801813 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:14.806228 10128 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
W20260709 13:19:14.814260 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59042.5us
W20260709 13:19:14.815338 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.718s (SENT)
I20260709 13:19:14.754047 16115 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:14.820043 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.821027 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:14.822844 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.826403 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.783500 16167 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:14.828917 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:14.836685 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:14.837823 10050 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:14.841836 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.753628 16024 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.843622 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:14.844982 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.820978 16160 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:14.858549 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:14.834487 16188 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.689s	user 0.005s	sys 0.000s
I20260709 13:19:14.851293 10128 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:14.828166 16201 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.822619 16207 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.860848 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.839214 16212 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:14.818717 16121 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:14.804308 16214 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:14.914928 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99618.1us
I20260709 13:19:14.917810 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:14.832999 16167 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:14.957593 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.959385 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.960683 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:14.961663 10128 raft_consensus.cc:2410] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:19:14.977895 16217 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 13:19:14.912937 16211 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.390s	user 0.002s	sys 0.000s
W20260709 13:19:14.866451 16118 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.655s	user 0.024s	sys 0.000s
W20260709 13:19:15.028700 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.014s	sys 0.025s
I20260709 13:19:14.856803 16119 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:15.031677 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.015s	sys 0.025s
I20260709 13:19:15.032608 10131 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:15.046514 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.069460 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:14.906275 16207 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.078115 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.044071 15786 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.090628 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.823s (SENT)
I20260709 13:19:14.857002 16197 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:14.909417 16201 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.104707 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:15.095268 16218 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.362s	user 0.000s	sys 0.002s
I20260709 13:19:15.038390 16118 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:15.132458 16230 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:15.133632 16201 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.109930 15786 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.137621 16207 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:15.138157 15761 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.002s
I20260709 13:19:15.144088 16089 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:15.146144 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.154502 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:15.043223 16119 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.107832 16066 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.181886 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.185281 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 245037us
I20260709 13:19:15.186893 10053 raft_consensus.cc:2368] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18.
I20260709 13:19:15.187266 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.188764 10129 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.192360 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.124621 16133 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.137787 15786 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:15.229193 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.246601 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.248063 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.931s (SENT)
I20260709 13:19:15.248703 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:15.253728 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.254992 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.247655 10053 raft_consensus.cc:2376] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:19:15.256983 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.257710 10053 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:15.258466 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.268685 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.268786 10123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.004s	sys 0.018s
I20260709 13:19:15.225867 16066 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.269810 10051 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
W20260709 13:19:15.269779 10123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.004s	sys 0.019s
I20260709 13:19:15.283793 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:15.287499 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.294792 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.294687 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:15.172009 15761 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.300060 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.297596 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.269279 16207 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:15.285804 15813 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:15.295547 16066 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.255877 16195 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.329579 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.378610 10051 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:15.383641 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:15.347379 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 116287us
W20260709 13:19:15.383798 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74350.2us
I20260709 13:19:15.323567 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.390350 10125 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:15.389892 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.394951 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.136s (SENT)
W20260709 13:19:15.317582 16181 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.111s	user 0.010s	sys 0.005s
W20260709 13:19:15.325524 16001 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.016s	user 0.015s	sys 0.008s
I20260709 13:19:15.408646 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.397994 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:15.409338 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.412693 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.040s
W20260709 13:19:15.322870 16234 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:19:15.413141 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.008s	sys 0.040s
I20260709 13:19:15.417261 10050 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:15.421721 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.421943 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:15.423198 10898 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:15.427790 10894 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:19:15.325276 16121 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.442765 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.320989 15813 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.447345 10129 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:15.449435 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:15.458294 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.428063 16207 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:15.463766 10198 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:15.463670 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.462193 10196 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:15.465564 10203 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:15.467036 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:15.411854 16181 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.479782 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.483312 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2891) took 1133 ms. Trace:
I20260709 13:19:15.484122 10900 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:15.476118 10125 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:15.437495 16035 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:15.483793 10055 rpcz_store.cc:276] 0709 13:19:14.349478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.349604 (+   126us) service_pool.cc:224] Handling call
0709 13:19:14.377785 (+ 28181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.385725 (+  7940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.385759 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.408286 (+ 22527us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.839703 (+431417us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.839754 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.839827 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.483002 (+643175us) raft_consensus.cc:1685] finished
0709 13:19:15.483025 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.483234 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":845,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":27860,"spinlock_wait_cycles":5888,"thread_start_us":348474,"threads_started":1,"wal-append.queue_time_us":635214}
I20260709 13:19:15.481269 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.488994 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.486443 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.482941 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.493820 10050 raft_consensus.cc:2376] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10. Re-sending same reply.
I20260709 13:19:15.446249 16235 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
W20260709 13:19:15.500306 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113989us
I20260709 13:19:15.477360 16240 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:15.516048 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.517032 10050 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.519906 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.485769 16212 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.502959 16241 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:15.557013 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.501562 16119 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.569487 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.570621 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.678s (SENT)
I20260709 13:19:15.570650 10051 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
W20260709 13:19:15.529229 16232 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.427s	user 0.000s	sys 0.003s
I20260709 13:19:15.560357 16145 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:15.572048 16231 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:19:15.572299 16145 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.573570 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2855) took 1255 ms. Trace:
I20260709 13:19:15.573623 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2890) took 1224 ms. Trace:
W20260709 13:19:15.576159 16243 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:19:15.577387 16248 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:19:15.573998 10124 rpcz_store.cc:276] 0709 13:19:14.317720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.317862 (+   142us) service_pool.cc:224] Handling call
0709 13:19:14.322939 (+  5077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.323621 (+   682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.323667 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.354070 (+ 30403us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.836701 (+482631us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:14.836766 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.836866 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.573282 (+736416us) raft_consensus.cc:1685] finished
0709 13:19:15.573311 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.573493 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":435679,"threads_started":1,"wal-append.queue_time_us":902084}
I20260709 13:19:15.571249 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:15.579946 16249 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:15.579013 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2864) took 1195 ms. Trace:
I20260709 13:19:15.574307 10052 rpcz_store.cc:276] 0709 13:19:14.348882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.349153 (+   271us) service_pool.cc:224] Handling call
0709 13:19:14.379564 (+ 30411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:14.380025 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:14.380055 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:14.398658 (+ 18603us) log.cc:844] Serialized 52 byte log entry
0709 13:19:14.752404 (+353746us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:14.752466 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.752551 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.495813 (+743262us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:15.573186 (+ 77373us) raft_consensus.cc:1685] finished
0709 13:19:15.573218 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.573534 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1798,"raft.queue_time_us":124,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":10191,"spinlock_wait_cycles":14564608,"thread_start_us":276801,"threads_started":1,"wal-append.queue_time_us":503531}
I20260709 13:19:15.587056 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.588325 10051 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:15.584901 10126 rpcz_store.cc:276] 0709 13:19:14.383587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.458729 (+ 75142us) service_pool.cc:224] Handling call
0709 13:19:14.461963 (+  3234us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:14.689319 (+227356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.689354 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:14.723706 (+ 34352us) log.cc:844] Serialized 78 byte log entry
0709 13:19:15.161912 (+438206us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:15.161986 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.162093 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.578770 (+416677us) raft_consensus.cc:1685] finished
0709 13:19:15.578793 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.578949 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7177,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97,"spinlock_wait_cycles":58624,"thread_start_us":588199,"threads_started":2,"wal-append.queue_time_us":655993}
I20260709 13:19:15.592947 10199 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:15.595816 10050 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:15.600384 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:15.555074 16233 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.322s	user 0.000s	sys 0.002s
I20260709 13:19:15.513338 16118 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.505685 16121 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:15.529189 16240 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.607247 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.574879 16145 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.613951 10050 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:15.582405 16207 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:15.501566 16035 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.623946 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.621683 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:15.625295 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.629776 10055 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:15.625799 10096 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 4.112s (SENT)
I20260709 13:19:15.630910 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:15.631543 10050 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:15.626865 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:15.632776 10052 raft_consensus.cc:2368] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:15.633469 10050 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:15.634217 15803 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:15.634514 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.634727 10899 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:15.635344 16255 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:15.635519 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.635936 16255 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:15.629170 10051 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:15.638501 10051 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:15.554644 16251 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:19:15.635365 10198 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:15.643493 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.560765 16212 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.649643 10052 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:15.651619 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.656329 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:15.658785 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:15.659915 10122 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:15.626170 15761 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:15.665704 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 221291us
I20260709 13:19:15.628979 16066 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.668908 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:15.673527 10128 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:15.676578 15803 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:19:15.680888 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:15.681842 10198 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:15.683530 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2889) took 1335 ms. Trace:
W20260709 13:19:15.684513 16247 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:19:15.685321 16253 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:19:15.687057 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2860) took 1348 ms. Trace:
W20260709 13:19:15.687945 16244 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:19:15.684087 10047 rpcz_store.cc:276] 0709 13:19:14.348516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.359361 (+ 10845us) service_pool.cc:224] Handling call
0709 13:19:14.389996 (+ 30635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.403028 (+ 13032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.403069 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.982201 (+579132us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b2c000014c8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:19:15.042538 (+ 60337us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.415958 (+373420us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:15.416021 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.416135 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.683104 (+266969us) raft_consensus.cc:1685] finished
0709 13:19:15.683157 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.683456 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15025,"raft.queue_time_us":12663,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":28883,"spinlock_wait_cycles":12321664,"thread_start_us":312057,"threads_started":1,"wal-append.queue_time_us":506107}
I20260709 13:19:15.687719 10127 rpcz_store.cc:276] 0709 13:19:14.338594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.358486 (+ 19892us) service_pool.cc:224] Handling call
0709 13:19:14.372629 (+ 14143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.707205 (+334576us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:14.707248 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.747508 (+ 40260us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.158256 (+410748us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:15.158323 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.158446 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.686797 (+528351us) raft_consensus.cc:1685] finished
0709 13:19:15.686821 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.686986 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14017,"raft.queue_time_us":95518,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"thread_start_us":670426,"threads_started":2,"wal-append.queue_time_us":721739}
I20260709 13:19:15.689503 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 774) took 1065 ms. Trace:
I20260709 13:19:15.642220 16207 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.689980 10054 rpcz_store.cc:276] 0709 13:19:14.624676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.624853 (+   177us) service_pool.cc:224] Handling call
0709 13:19:14.625265 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.666925 (+ 41660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.666984 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.686112 (+ 19128us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.240201 (+554089us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:15.240264 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.240368 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.689249 (+448881us) raft_consensus.cc:1685] finished
0709 13:19:15.689276 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.689439 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39962,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":19246,"thread_start_us":485941,"threads_started":1,"wal-append.queue_time_us":752423}
I20260709 13:19:15.701954 10131 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:15.703936 16163 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:15.704169 16181 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.704548 16163 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:15.649609 16145 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:15.705902 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.173s (SENT)
I20260709 13:19:15.705449 16145 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.705672 16163 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.708333 16145 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.708930 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:15.709292 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:15.624248 16121 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.711654 10125 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:15.712484 16121 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.714789 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:15.716542 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:15.717612 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.718099 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:15.719105 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:15.663143 16249 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.712942 16240 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:15.706965 16163 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:15.750109 16246 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:19:15.717945 16001 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:15.747915 15813 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.776757 15813 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:15.744289 16249 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.731493 16181 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:19:15.782413 16167 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.000s
I20260709 13:19:15.782855 16181 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:15.783445 16181 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:15.783623 16196 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:15.784436 16181 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.784915 16252 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:15.774843 10899 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:15.713429 16255 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.781595 16249 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:15.789815 16181 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:15.799471 16115 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.964s	user 0.010s	sys 0.007s
I20260709 13:19:15.802721 16115 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.801671 15920 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.801591 16255 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:15.804322 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.734515 16145 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:15.789673 16035 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.816174 16145 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.782904 16167 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.835454 16145 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.852830 16115 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.854723 16167 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.857191 16212 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:15.872298 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:15.873167 10893 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:15.871845 15813 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.875393 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:15.875267 15813 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.868465 16269 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:19:15.885993 16167 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:15.907011 16167 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.909562 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2899) took 1466 ms. Trace:
I20260709 13:19:15.910142 10049 rpcz_store.cc:276] 0709 13:19:14.443079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.445583 (+  2504us) service_pool.cc:224] Handling call
0709 13:19:14.462660 (+ 17077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.799350 (+336690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.902913 (+1103563us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540010b8d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:15.909450 (+  6537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602140928}
I20260709 13:19:15.913138 16167 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:15.954726 16060 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:15.970240 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 98030.1us
I20260709 13:19:15.971745 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:15.985042 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:15.981941 16266 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:15.974743 16212 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:19:15.986821 16266 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.987447 16266 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:15.987843 16275 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:15.988183 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.988411 16275 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:15.988687 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.988860 16275 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:15.982213 16269 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:15.990381 16269 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:15.987282 16212 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:15.991756 16212 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:15.991513 16269 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:15.992763 16212 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.994115 16212 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:15.994249 16277 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.994827 16277 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:15.996029 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:15.997478 16277 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.998149 16277 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.007397 15813 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.008543 15813 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:16.008982 15813 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.993199 16269 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:16.009352 15813 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:15.996076 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.011600 10131 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.015044 16275 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.016481 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.017904 16275 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:15.998725 16277 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.021229 16145 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.022572 10891 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:16.036123 16167 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:16.002748 10050 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:16.018738 16275 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:19:16.018839 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.042166 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.049671 16145 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:16.050297 10096 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.538s (SENT)
I20260709 13:19:16.055972 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.053362 16024 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.045413 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.055975 10123 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:16.063411 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:16.065254 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.568s (SENT)
I20260709 13:19:16.066084 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.067977 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.068917 10123 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.072544 10131 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:16.064074 16269 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.063763 16167 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:16.076694 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.613s (SENT)
I20260709 13:19:16.074313 10131 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:16.087800 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.631s (SENT)
I20260709 13:19:16.088503 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.089504 16024 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.090003 16277 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:16.083074 16254 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.606s	user 0.004s	sys 0.000s
I20260709 13:19:16.075865 16279 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:16.093492 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.094318 10131 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:16.098902 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 102971us
I20260709 13:19:16.101145 10123 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:16.102768 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.104002 10131 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:16.105084 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.106077 15813 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.107774 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.108628 10894 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:16.109917 15813 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.111387 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:16.112221 10894 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:16.117573 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.118713 16167 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:16.119413 16167 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:16.121986 16167 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:16.228796 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.235783 15813 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.237535 15813 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.252096 15813 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.251763 16266 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.262629 16266 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.266746 16214 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.294710 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.296218 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.297080 10122 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:16.299726 16207 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.296408 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.324450 10125 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:16.330658 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.333949 10122 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:16.330747 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.335772 10125 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:16.337816 10122 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:19:16.338445 10122 pending_rounds.cc:85] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 11
I20260709 13:19:16.338800 10122 pending_rounds.cc:107] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:19:16.339253 10122 raft_consensus.cc:2889] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.344594 16252 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:16.370253 16258 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:16.466837 16258 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:16.468057 16258 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.469408 16258 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:16.473886 16035 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:16.374027 10055 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.744s	user 0.007s	sys 0.010s
I20260709 13:19:16.475361 10055 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:16.475901 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2375) took 1068 ms. Trace:
I20260709 13:19:16.476220 10055 rpcz_store.cc:276] 0709 13:19:15.407892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.605813 (+197921us) service_pool.cc:224] Handling call
0709 13:19:16.475832 (+870019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:16.478183 16130 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:16.406378 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:16.429052 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.388746 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:16.478852 16130 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.474633 16035 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.490453 16035 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.486982 16130 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.492637 16035 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.494035 16035 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.496528 16035 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.500049 16035 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.500749 16035 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.503679 16121 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.481343 10894 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:16.504756 16121 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.508602 16287 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.509758 16035 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.510707 16035 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:16.509824 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111877us
I20260709 13:19:16.511426 16035 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.512315 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:16.513368 16238 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.219s	user 0.000s	sys 0.004s
I20260709 13:19:16.513940 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:16.514890 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2894) took 2073 ms. Trace:
I20260709 13:19:16.515502 16035 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.516248 16035 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.515673 10048 rpcz_store.cc:276] 0709 13:19:14.441298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.448075 (+  6777us) service_pool.cc:224] Handling call
0709 13:19:14.448985 (+   910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.459650 (+ 10665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.459702 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.474756 (+ 15054us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.032399 (+557643us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:15.032446 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.032526 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.512710 (+1480184us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:16.514532 (+  1822us) raft_consensus.cc:1685] finished
0709 13:19:16.514554 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.514821 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9180,"raft.queue_time_us":11927,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":19983,"spinlock_wait_cycles":10986240,"thread_start_us":470241,"threads_started":1,"wal-append.queue_time_us":703833}
I20260709 13:19:16.517014 16035 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.520649 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.522078 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.523514 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.527050 16205 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:16.529042 16205 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:16.509423 16287 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.531561 16121 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.532415 16121 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:16.532819 16121 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:16.533272 16121 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:16.529536 16205 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.534969 16205 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.540397 16035 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.564062 16035 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:16.564781 16035 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.565382 16035 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.559468 16205 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.537833 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:16.567018 10198 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:16.568778 16035 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:16.569252 16035 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.569540 16205 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.569650 16035 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:16.563174 16287 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.571367 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.574107 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.576864 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.575037 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:16.580657 16121 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:16.578989 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 3.149s (SENT)
I20260709 13:19:16.583556 16121 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.586696 10894 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:19:16.587348 10894 pending_rounds.cc:85] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 11
I20260709 13:19:16.587611 10894 pending_rounds.cc:107] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:19:16.587869 10894 raft_consensus.cc:2889] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.588104 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.589465 10891 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:16.593703 15754 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.595220 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.595397 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.593788 16029 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.596755 16207 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:16.597033 16029 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.599329 16035 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.601686 16035 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.601401 15754 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.602334 16035 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.590648 16287 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:16.603806 15754 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.604430 15754 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:16.603060 16035 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.597395 16207 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:16.614189 16035 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:16.615017 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.614665 16207 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.616248 16035 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:16.616652 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.618081 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.618448 16029 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.619055 16121 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:16.618140 16035 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.636137 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:16.638029 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:16.641584 16200 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:16.642786 16200 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.643244 16115 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:16.644109 16115 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.644284 16200 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.645051 16115 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.649152 10127 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:16.666229 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.676621 16035 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:16.678792 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.683790 10128 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:19:16.689455 10200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.035s	sys 0.013s
W20260709 13:19:16.690025 10200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.035s	sys 0.013s
I20260709 13:19:16.685675 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.688762 16133 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:16.688726 16060 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:19:16.705221 16121 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:19:16.710752 16121 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:16.711129 16121 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.704246 16133 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.712065 16121 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.712949 16133 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.713655 16121 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.704878 16035 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:16.715749 16035 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.716773 16035 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.706094 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.741850 16205 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:16.744737 16205 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.745793 16205 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.740687 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.762331 16115 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:16.766355 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.767248 10127 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:16.767742 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.768368 10894 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:16.776739 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.777083 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.777557 10894 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:16.779803 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.755203 16200 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:16.780566 10894 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:16.780459 10127 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:16.786458 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.787371 10126 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:16.787654 16029 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:16.786552 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.788863 10123 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:16.789721 16207 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:16.792483 10891 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:16.794003 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.795018 10894 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:16.796550 15754 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.796566 16029 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.796798 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.797317 15754 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.798506 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.798727 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:16.799525 10048 raft_consensus.cc:2410] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:19:16.799608 10895 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.801017 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.801792 10048 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.802932 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.803694 10048 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:16.797461 16029 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.804858 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.806869 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:16.807269 10124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.143s	user 0.023s	sys 0.006s
I20260709 13:19:16.807961 10048 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:16.810770 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.811548 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.812415 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.811661 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.812897 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.811674 10056 raft_consensus.cc:2368] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:16.812425 10055 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.813804 10051 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:16.815588 10051 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:16.811228 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.816659 10122 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:16.817718 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.818176 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.809530 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:16.819705 10050 raft_consensus.cc:2376] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15. Re-sending same reply.
I20260709 13:19:16.819847 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.820654 10047 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:16.820720 10055 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.791368 16200 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:16.822109 16200 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.822750 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.822870 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.822144 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.823917 10123 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:16.823822 10052 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:16.824704 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.824941 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.825593 10123 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:16.826659 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.827456 10055 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:16.827965 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.816946 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:16.832607 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.006s	sys 0.011s
I20260709 13:19:16.819772 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:16.841216 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.008s	sys 0.012s
I20260709 13:19:16.840168 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.842269 10125 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.837150 10054 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.830953 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.841799 10127 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:16.831923 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.845844 10050 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:16.849692 16266 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.850816 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:16.834569 16292 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:19:16.851847 10049 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:19:16.852620 10124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.024s	sys 0.007s
I20260709 13:19:16.841396 10051 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:19:16.830283 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.860934 10055 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:16.862692 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.864264 10895 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:16.873875 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:16.875334 10055 raft_consensus.cc:2376] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16. Re-sending same reply.
I20260709 13:19:16.874300 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.829481 16133 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.878654 10053 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.873749 16208 raft_consensus.cc:476] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Not starting pre-election -- already a leader
I20260709 13:19:16.885018 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.885825 10049 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:16.886418 10051 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:19:16.887297 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.887614 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:16.886790 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.888044 10055 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:16.889245 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:16.888393 10049 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:16.889763 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.890352 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.890501 10051 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:16.890805 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.891021 10056 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:16.891762 10049 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:16.892146 10125 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:16.897667 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.898402 15754 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.898700 10050 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.903429 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.903975 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:16.888324 10047 raft_consensus.cc:2376] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18. Re-sending same reply.
I20260709 13:19:16.904655 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.905347 10054 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:16.906378 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.907845 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:16.909473 10125 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.879926 16133 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.910657 10123 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:16.912238 10055 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.912734 10051 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.865394 16207 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.912329 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.934559 10051 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:16.937299 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:16.938442 10894 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:16.908829 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.913043 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2859) took 2550 ms. Trace:
I20260709 13:19:16.913905 15754 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:16.941987 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:16.942796 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.930377 16208 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:16.946741 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:16.950517 10055 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:16.942759 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:16.939471 10130 rpcz_store.cc:276] 0709 13:19:14.326855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.327014 (+   159us) service_pool.cc:224] Handling call
0709 13:19:14.346065 (+ 19051us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:14.656105 (+310040us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:14.690628 (+ 34523us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.714337 (+1023709us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18000518e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:19:16.842366 (+1128029us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:16.847314 (+  4948us) log.cc:844] Serialized 52 byte log entry
0709 13:19:16.847625 (+   311us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:16.847666 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.847788 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.860539 (+ 12751us) raft_consensus.cc:1685] finished
0709 13:19:16.876877 (+ 16338us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.877371 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35864,"prepare.queue_time_us":34253,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":121655,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":5071,"spinlock_wait_cycles":3199744,"thread_start_us":1312746,"threads_started":2,"wal-append.queue_time_us":1151555}
I20260709 13:19:16.953995 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:16.946154 16133 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:16.956149 16121 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:16.956598 16121 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.957005 16121 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:16.959797 16130 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:16.955415 16133 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.961130 16133 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:16.968659 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.391s	user 0.025s	sys 0.044s
W20260709 13:19:16.969156 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.391s	user 0.025s	sys 0.044s
I20260709 13:19:16.951803 16029 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:16.973292 16029 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:16.974080 16029 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:16.975028 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 1.26025e+06us
I20260709 13:19:16.977332 10894 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:16.980424 10122 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:16.977814 10131 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:16.977980 15797 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:16.982163 15797 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:16.978878 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:16.983726 10128 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:16.985399 15797 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:16.986861 15797 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:16.930775 16207 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:16.990382 16207 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:16.997076 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:17.012844 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.012786 10122 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:17.014098 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.015038 10203 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:17.011646 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.016062 10053 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:16.997141 16029 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.016155 16087 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:17.016927 16087 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.017328 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.019579 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.021119 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.023447 10894 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:17.024597 16108 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.025242 16108 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.025668 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:17.027804 16116 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:17.028566 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.030751 10051 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:17.034839 10129 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:17.035852 10122 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:17.036538 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:17.038455 10122 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:17.038604 10129 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:17.003950 10130 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:17.007680 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:17.009033 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.046713 10055 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
W20260709 13:19:17.009593 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 1.34162e+06us
W20260709 13:19:17.010430 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56548.2us
I20260709 13:19:17.053615 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:16.952859 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.009449 10131 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:17.055827 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 103052us
I20260709 13:19:17.049547 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.056927 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.068027 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.072156 10125 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:17.074404 16221 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:17.079334 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.080135 10196 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.082471 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.082899 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:17.084302 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.085913 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:17.086627 10205 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:17.087299 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:17.084796 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.058010 16226 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.091661 16029 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:17.022984 16107 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:19:17.077961 16117 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:17.092190 16145 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:17.092867 16145 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:17.092190 16029 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.093374 16145 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:17.093643 16029 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:17.028595 16116 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.097810 16116 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:17.059255 16167 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:17.099237 16108 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:17.079670 16089 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.085662 16205 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:17.100555 16205 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:17.100386 16089 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.101018 16205 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:17.099732 16167 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:17.103596 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 700) took 1457 ms. Trace:
I20260709 13:19:17.108466 10124 rpcz_store.cc:276] 0709 13:19:15.646065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.646222 (+   157us) service_pool.cc:224] Handling call
0709 13:19:15.648520 (+  2298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.853126 (+1204606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.853184 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:16.853920 (+   736us) log.cc:844] Serialized 26 byte log entry
0709 13:19:16.934819 (+ 80899us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:16.934856 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.934943 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:17.103303 (+168360us) raft_consensus.cc:1685] finished
0709 13:19:17.103329 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.103514 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128692,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":1241593,"threads_started":2,"wal-append.queue_time_us":189855}
I20260709 13:19:17.109910 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.124581 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2264) took 1596 ms. Trace:
I20260709 13:19:17.126310 10200 rpcz_store.cc:276] 0709 13:19:15.561176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.585774 (+ 24598us) service_pool.cc:224] Handling call
0709 13:19:15.607495 (+ 21721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:15.631021 (+ 23526us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:15.647859 (+ 16838us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.043166 (+1395307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.043228 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:17.043279 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.124238 (+ 80959us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:17.124302 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.124484 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352540,"prepare.queue_time_us":27150,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":353035,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":53487360,"thread_start_us":1022629,"threads_started":1,"wal-append.queue_time_us":199398,"wal-append.run_cpu_time_us":1286,"wal-append.run_wall_time_us":1001602}
I20260709 13:19:17.124598 10124 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:17.082530 16087 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:17.102901 16089 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:17.090209 16060 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:17.107095 16035 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:17.107378 16117 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:17.093701 16196 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:17.140455 10124 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:17.128947 16266 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.105120 16121 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:17.171037 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61708.7us
W20260709 13:19:17.171316 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48548.1us
W20260709 13:19:17.173393 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:17.177440 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44876.6us
I20260709 13:19:17.185694 10894 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:17.123404 10096 leader_election.cc:400] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:17.105681 16115 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:17.110807 16167 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:17.133677 10205 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:17.104578 16221 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:19:17.167603 16035 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:17.107928 15810 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:19:17.206204 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82974.7us
I20260709 13:19:17.206527 10205 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:17.208601 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.209524 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.210484 10196 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:17.212697 16035 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.215598 16258 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.215799 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.216609 16258 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.217386 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.218677 16262 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:17.219393 10203 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:17.219937 16262 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:17.220340 16262 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:17.172564 15661 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:19:17.174216 16263 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.221199 16262 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.221725 16263 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.222681 16262 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:17.202275 16115 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.215799 15810 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:17.223500 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:17.225003 10894 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:17.212924 16167 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.200301 16226 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:19:17.202834 16124 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.231969 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:17.228919 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:17.183729 16121 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.236565 16089 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:17.237825 10198 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:17.238462 10198 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:17.238826 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.142021 16029 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.187353 16196 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:17.242101 10201 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:17.239414 10198 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.243703 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.244479 10198 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.241076 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.251950 10205 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:17.253820 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.229319 16115 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:17.256258 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.257186 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:17.175994 16133 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.224227 16263 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:17.259212 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.266158 10202 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:17.267598 15661 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.244411 16116 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:17.268579 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.228339 15810 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:17.261705 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:17.249151 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.256145 16107 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:17.252174 16196 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.263566 16186 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:17.227830 16124 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.265308 16029 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.267482 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:17.240839 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.261044 10199 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:17.242063 15813 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:17.268759 16117 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:17.277151 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 228821us
I20260709 13:19:17.278987 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.290189 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.364643 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:17.312933 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 81047.5us
I20260709 13:19:17.279167 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.435191 16220 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:17.442508 10201 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:17.444092 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:17.442782 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 202005us
I20260709 13:19:17.443981 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.449069 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:17.449402 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:17.450322 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.451710 10204 raft_consensus.cc:2410] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:19:17.452035 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.434497 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:17.436028 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:17.445875 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 358703us
I20260709 13:19:17.450359 10893 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
W20260709 13:19:17.330219 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 68573.8us
I20260709 13:19:17.456588 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:17.412626 16303 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:19:17.278795 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003169c0 after lost signal to thread 8928
W20260709 13:19:17.361163 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85120.5us
W20260709 13:19:17.460456 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800339b40 after lost signal to thread 9988
I20260709 13:19:17.461380 10891 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:17.375798 10198 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:17.464145 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.465140 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.475026 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:17.462457 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800061e60 after lost signal to thread 10080
I20260709 13:19:17.327420 16108 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:17.483363 10197 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:17.484506 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 378062us
I20260709 13:19:17.374119 16089 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:17.480906 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.482076 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.465070 16221 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:17.400183 16208 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:17.460260 16301 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
I20260709 13:19:17.260740 16258 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:17.489697 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.490306 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.268724 15661 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.193s
W20260709 13:19:17.479079 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800307d40 after lost signal to thread 10155
W20260709 13:19:17.528194 10226 debug-util.cc:398] Leaking SignalData structure 0x7b0800307d60 after lost signal to thread 10156
W20260709 13:19:17.480949 15786 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:19:17.469965 16029 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.528654 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48560.6us
I20260709 13:19:17.532121 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:17.534160 16297 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:19:17.536724 16255 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:17.537482 16299 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:19:17.398983 15810 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.537777 16205 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.538895 15810 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:17.387423 16145 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.532301 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.541481 10203 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:17.528620 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48653.1us
I20260709 13:19:17.543474 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:17.546509 10129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.020s	sys 0.023s
W20260709 13:19:17.549455 10129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.021s	sys 0.023s
I20260709 13:19:17.544488 10197 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:17.487264 15977 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:17.547494 10226 debug-util.cc:398] Leaking SignalData structure 0x7b080032a940 after lost signal to thread 10229
W20260709 13:19:17.594406 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08002d3be0 after lost signal to thread 10230
W20260709 13:19:17.596962 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2ac0 after lost signal to thread 10924
I20260709 13:19:17.491922 16121 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.606683 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 172260us
I20260709 13:19:17.488027 16108 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.601668 10205 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:17.609019 15813 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:17.608839 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:17.540354 16226 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:17.548422 16298 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:17.626865 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.492635 16266 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:17.633011 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88528.9us
I20260709 13:19:17.555924 16220 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:17.635842 10197 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:17.637456 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.555862 15977 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.546061 16186 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:19:17.616535 16108 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:17.625005 16208 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.549039 16029 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.659822 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.665832 10196 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:17.666292 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.643231 16220 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.637032 16089 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:17.670858 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.671988 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.648151 15977 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:17.653631 16133 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.616569 16121 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.631039 15813 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.707644 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.648011 16087 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:19:17.710078 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.667850 16309 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:17.675797 16220 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:17.720726 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:17.721601 10200 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:17.725337 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 136399us
I20260709 13:19:17.694267 16035 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:17.680370 16263 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:19:17.739174 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:17.666890 16208 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:17.753523 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:17.759403 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 163741us
I20260709 13:19:17.722002 16133 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:19:17.746062 16316 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.743280 16035 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:17.748425 16308 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:17.823405 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 202955us
W20260709 13:19:17.825475 10096 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.602s (SENT)
I20260709 13:19:17.834702 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.837353 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:17.797068 16205 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.759387 16226 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 13:19:17.852344 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 302402us
I20260709 13:19:17.857661 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.859038 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:17.862074 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106197us
I20260709 13:19:17.863312 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:17.833348 16145 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:17.862159 10170 connection.cc:256] RPC call timeout handler was delayed by -1.7855252790000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:19:17.834501 16308 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:17.875658 10051 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:17.877084 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.896718 10052 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:17.882777 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.853833 16133 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:17.896762 10055 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:19:17.902777 10055 pending_rounds.cc:85] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 13
I20260709 13:19:17.900709 10053 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:17.904109 10055 pending_rounds.cc:107] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:19:17.911549 10055 raft_consensus.cc:2889] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.834177 16316 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:17.800699 16089 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:17.925879 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 101182us
I20260709 13:19:17.904067 16308 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:17.947836 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84570.7us
I20260709 13:19:17.950615 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:17.967170 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:17.945917 10052 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:17.960577 10202 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:17.949842 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:17.902441 16133 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:17.968636 10126 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:19:17.978904 10203 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
W20260709 13:19:17.980120 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 451240us
I20260709 13:19:17.871037 16326 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:19:17.983881 10052 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:17.981359 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:17.970922 16336 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:17.992583 10096 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.665s (SENT)
I20260709 13:19:17.993310 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:18.002151 10053 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:17.945525 16332 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:17.981820 16315 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.172s	user 0.002s	sys 0.000s
W20260709 13:19:18.019212 16314 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:19:18.020287 16310 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 13:19:18.021132 16323 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:18.023880 16320 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:19:18.027552 16332 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:18.029269 16327 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:19:18.030122 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.030982 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.040158 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.043282 15977 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.047039 16326 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:18.048833 16316 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:18.049549 16208 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.995807 16196 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:17.946007 16035 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.064102 16133 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.065618 16329 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:19:18.065922  9949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 121) took 17181 ms (client timeout 15000 ms). Trace:
W20260709 13:19:18.078636 16311 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.226s	user 0.004s	sys 0.000s
I20260709 13:19:17.988649 16121 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:18.082378 16115 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.818s	user 0.014s	sys 0.000s
W20260709 13:19:18.087237 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 70554.4us
I20260709 13:19:18.082872 16029 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:18.097272 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.189013 10052 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:18.105213 10048 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:18.078943 15813 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:18.108165 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 141139us
W20260709 13:19:18.110711 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.182507 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.192433 10053 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:18.190798 10052 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:18.194743 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:18.165954 15797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.001s	sys 0.012s
W20260709 13:19:18.197326 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.108523  9949 rpcz_store.cc:269] 0709 13:19:00.884877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.885211 (+   334us) service_pool.cc:224] Handling call
0709 13:19:18.065761 (+17180550us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:13.527886 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:13.681884 (+153998us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:13.691734 (+  9850us) write_op.cc:435] Acquired schema lock
0709 13:19:13.694371 (+  2637us) tablet.cc:662] Decoding operations
0709 13:19:13.805815 (+111444us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:13.805954 (+   139us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:13.805986 (+    32us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:19:13.858644 (+ 52658us) tablet.cc:701] Row locks acquired
0709 13:19:13.858699 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:19:13.858920 (+   221us) write_op.cc:270] Start()
0709 13:19:13.859142 (+   222us) write_op.cc:276] Timestamp: P: 1783603153858841 usec, L: 0
0709 13:19:13.859167 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:19:13.860270 (+  1103us) log.cc:844] Serialized 3950 byte log entry
0709 13:19:14.468075 (+607805us) op_driver.cc:464] REPLICATION: finished
0709 13:19:15.115644 (+647569us) write_op.cc:301] APPLY: starting
0709 13:19:15.143250 (+ 27606us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:15.379257 (+236007us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:15.401534 (+ 22277us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:15.795769 (+394235us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:15.796379 (+   610us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:19:15.796474 (+    95us) write_op.cc:312] APPLY: finished
0709 13:19:15.797130 (+   656us) log.cc:844] Serialized 97 byte log entry
0709 13:19:15.867694 (+ 70564us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:15.871340 (+  3646us) write_op.cc:454] Released schema lock
0709 13:19:15.888081 (+ 16741us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:15.903509 (+ 15428us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:15.903566 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1713543,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":599837,"apply.run_cpu_time_us":42479,"apply.run_wall_time_us":818409,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45847,"num_ops":10,"prepare.queue_time_us":278552,"prepare.run_cpu_time_us":27232,"prepare.run_wall_time_us":440347,"raft.queue_time_us":319974,"raft.run_cpu_time_us":13794,"raft.run_wall_time_us":232370,"replication_time_us":608655,"spinlock_wait_cycles":91392,"thread_start_us":727917,"threads_started":5,"wal-append.queue_time_us":285651,"wal-append.run_cpu_time_us":63912,"wal-append.run_wall_time_us":2356626}]]}
W20260709 13:19:18.200290 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.201807 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:18.204648 10895 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.003s	sys 0.009s
W20260709 13:19:18.205129 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.004s	sys 0.007s
W20260709 13:19:18.207969 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.015s	sys 0.028s
I20260709 13:19:18.180624 16266 raft_consensus.cc:2764] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:19:18.191569 16310 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:19:18.210839 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.015s	sys 0.028s
W20260709 13:19:18.213495 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 604689us
I20260709 13:19:18.214505 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:18.210238 10895 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.003s	sys 0.009s
W20260709 13:19:18.212867 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.004s	sys 0.007s
I20260709 13:19:18.218822 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:18.222114 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:18.187366 16329 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:18.225911 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:18.226801 10196 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:18.204255 16208 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:18.203717 16133 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.241389 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:18.243134 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.000s	sys 0.013s
I20260709 13:19:18.240379 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:18.249171 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.013s
I20260709 13:19:18.250188 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:18.249693 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 604287us
I20260709 13:19:18.253892 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:18.210160 15813 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:18.264345 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:18.225456 16332 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.197403 16115 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.206614 15797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.001s	sys 0.012s
W20260709 13:19:18.299393 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 306827us
W20260709 13:19:18.269312 10021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.002s	sys 0.014s
W20260709 13:19:18.298612 10049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.006s	sys 0.009s
W20260709 13:19:18.266808 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 318785us
W20260709 13:19:18.301961 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48225us
I20260709 13:19:18.296620 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:18.302736 10021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.002s	sys 0.014s
W20260709 13:19:18.302439 10049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.007s	sys 0.010s
I20260709 13:19:18.251092 16121 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:18.258654 16220 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:19:18.243196 16208 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.306768 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 785) took 1278 ms. Trace:
W20260709 13:19:18.350494 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 53962.4us
W20260709 13:19:18.356300 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45208.3us
I20260709 13:19:18.264415 16322 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:18.363284 10049 rpcz_store.cc:276] 0709 13:19:17.028459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.029963 (+  1504us) service_pool.cc:224] Handling call
0709 13:19:17.030313 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.030554 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:17.353849 (+323295us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.304074 (+950225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.304164 (+    90us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:18.304242 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.306490 (+  2248us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.306681 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20977,"prepare.queue_time_us":155844,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":2095,"raft.queue_time_us":145238,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":5455,"thread_start_us":1272326,"threads_started":3,"wal-append.queue_time_us":392959}
I20260709 13:19:18.368427 10049 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:18.371621 10055 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:18.304791 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.670s (SENT)
W20260709 13:19:18.374536 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.267832 16318 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:18.309692 16121 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.375277 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:18.305102 15813 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:18.305092 16117 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:18.368008 16310 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:19:18.311651 16029 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:18.250912 16200 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 13:19:18.366297 16208 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.403435 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:18.403903 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99200.6us
W20260709 13:19:18.375536 16221 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.833s	user 0.015s	sys 0.000s
I20260709 13:19:18.374421 16344 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.343305 16089 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:19:18.320932 16220 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.412374 15810 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.004s
I20260709 13:19:18.374796 16345 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:18.408731 16322 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:18.406148 16121 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.402676 16347 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:18.301761 16332 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.411471 16340 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:18.410604 16029 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.468811 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2428) took 1584 ms. Trace:
I20260709 13:19:18.481798 10056 rpcz_store.cc:276] 0709 13:19:16.884591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.901755 (+ 17164us) service_pool.cc:224] Handling call
0709 13:19:16.902145 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.902554 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:17.045202 (+142648us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.628482 (+583280us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.637449 (+  8967us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.637662 (+   213us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:17.637997 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.805607 (+167610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.805733 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.468534 (+662801us) raft_consensus.cc:1685] finished
0709 13:19:18.468560 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.468732 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24485,"prepare.queue_time_us":232727,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":4841,"raft.queue_time_us":322155,"raft.run_cpu_time_us":519,"raft.run_wall_time_us":17024,"thread_start_us":817769,"threads_started":4,"wal-append.queue_time_us":631118}
I20260709 13:19:18.452103 16221 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:18.467082 10199 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
W20260709 13:19:18.476502 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75652us
I20260709 13:19:18.378458 15797 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:19:18.419420 16330 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:19:18.458920 16351 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.469848 16345 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.471688 15977 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:18.487457 16029 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.502684 16116 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:18.453696 16344 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.484308 16358 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:19:18.509935 16345 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:18.521062 16116 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:18.517355 15813 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.485224 16347 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.539676 16339 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
W20260709 13:19:18.531275 16337 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:19:18.532029 16346 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:19:18.531023 16350 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:19:18.530424 16359 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:18.565883 16326 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:18.586714 10096 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.861s (SENT)
I20260709 13:19:18.590034 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.597621 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:18.598295 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:18.594734 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:18.598598 10127 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:18.599192 10131 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:18.591305 10894 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:18.602555 10126 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:18.554550 16221 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:18.607457 10096 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.060s (SENT)
I20260709 13:19:18.616860 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:18.609221 10127 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:18.619601 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:18.610365 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:18.510943 16351 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.620357 10122 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:18.621799 10205 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:18.622319 10205 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:18.624363 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.641084 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:18.638334 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 956) took 1378 ms. Trace:
I20260709 13:19:18.638713 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:18.642913 10130 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:18.529354 16355 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:19:18.653448 10131 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:18.642095 10895 rpcz_store.cc:276] 0709 13:19:17.245867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.246022 (+   155us) service_pool.cc:224] Handling call
0709 13:19:17.246441 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.544676 (+298235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.544708 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.593964 (+ 49256us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.216200 (+622236us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:18.221670 (+  5470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.224372 (+  2702us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.623563 (+399191us) raft_consensus.cc:1685] finished
0709 13:19:18.623586 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.623751 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16534,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":875938,"threads_started":2,"wal-append.queue_time_us":758142}
I20260709 13:19:18.639981 10205 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.773984 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.776537 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.777190 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.605129 16340 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.777318 10196 raft_consensus.cc:2376] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18. Re-sending same reply.
I20260709 13:19:18.780896 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:18.623389 16318 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:18.787040 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:18.787876 10054 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:19:18.635991 16287 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.635968 16369 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:18.795105 10197 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:18.795666 10196 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:18.801293 10051 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:18.801496 10196 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:18.802683 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.800573 10197 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:18.745499 16117 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 13:19:18.614101 16326 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.816872  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:18.646585 16200 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
W20260709 13:19:18.808224 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 167214us
W20260709 13:19:18.817618 10202 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3441) took 1549 ms (client timeout 1696 ms). Trace:
W20260709 13:19:18.828184 10202 rpcz_store.cc:269] 0709 13:19:17.267803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.268320 (+   517us) service_pool.cc:224] Handling call
0709 13:19:18.817394 (+1549074us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:18.817544 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917363584}
I20260709 13:19:18.555217 16322 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.817530 10197 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.646535 16221 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.840850 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:18.828984 16108 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.161s	user 0.008s	sys 0.008s
I20260709 13:19:18.845530 10197 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:18.829592 10196 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.846025 10129 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:18.816301 16364 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.847051 10196 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:18.402788 16329 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.823534 16235 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:18.836365 16369 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.872975 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:18.874873 16373 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:18.875667 16373 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.876163 16373 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:18.883911 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:18.884788 10202 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:18.889631 10129 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:18.794466 16318 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.847199 16200 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.792625 16340 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.854892 16353 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
I20260709 13:19:18.835543 16035 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:19:18.808897 16287 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.916729 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 135913us
I20260709 13:19:18.859640 16368 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:18.900817 16342 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:18.898329 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:18.878899 16235 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:18.855733 16108 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.921023 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:18.923887 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:18.918558 16117 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:18.895224 16369 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.942086 16318 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:18.945602 16308 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.974s	user 0.013s	sys 0.000s
I20260709 13:19:18.947577 16368 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.946501 16308 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:18.948324 16368 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.916049 16372 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:18.920351 16342 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:18.952670 16133 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.001s
I20260709 13:19:18.858476 16364 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.958736 10130 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:18.926393 16220 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:18.911360 16200 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:18.969121 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44462.5us
I20260709 13:19:18.958236 16372 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:18.956624 16332 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:18.974565 16235 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.957788 16342 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.950700 16308 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:18.971236 16361 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.228s	user 0.001s	sys 0.001s
I20260709 13:19:19.011637 16375 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:18.950857 16368 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.048403 10196 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:19.047035 10197 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:18.960136 16116 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:18.973506 16351 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.063639 16220 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:19.077710 16220 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.080384 16220 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.081066 16220 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.081563 16220 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.084830 16220 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.085840 16220 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.086596 16369 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.087577 16200 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.086726 16220 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:19.040675 16089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.008s	sys 0.024s
W20260709 13:19:19.088865 16089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.009s	sys 0.024s
I20260709 13:19:19.014562 16108 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.089505 16344 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.013949 16372 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:19.095773 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.097218 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.098364 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.099512 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.100823 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.101475 15661 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:19.099013 10202 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:19.102227 15661 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.101430 10201 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:19.103340 15661 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.107869 16287 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.072575 16235 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:19.123678 16208 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.672s	user 0.006s	sys 0.008s
I20260709 13:19:19.124284 16208 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.126055 16364 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:19.128041 16124 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:19.129878 16121 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 13:19:19.131259 16208 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.130453 16121 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.131836 16340 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.133875 16121 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.135782 16373 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.137492 16342 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.138729 16373 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.140519 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:19.106448 16107 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:19:19.072177 16326 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.066330 16375 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:19.096185 16367 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:19:19.152334 10053 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:19.160633 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.176762 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.164398 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2912) took 1248 ms. Trace:
I20260709 13:19:19.177843 10055 rpcz_store.cc:276] 0709 13:19:17.916587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.941830 (+ 25243us) service_pool.cc:224] Handling call
0709 13:19:17.942203 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.372433 (+430230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.164086 (+791653us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54003cfad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:19.164316 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986890496}
I20260709 13:19:19.180413 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2925) took 1153 ms. Trace:
I20260709 13:19:19.180930 10053 rpcz_store.cc:276] 0709 13:19:18.026744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.193266 (+166522us) service_pool.cc:224] Handling call
0709 13:19:19.151638 (+958372us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:19.152015 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.180066 (+ 28051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.180340 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2475264}
I20260709 13:19:19.165838 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.187815 10196 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:19.178426 16029 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.662s	user 0.013s	sys 0.000s
I20260709 13:19:19.191679 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.176214 16375 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:19.169129 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.011s	sys 0.008s
W20260709 13:19:19.199798 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.011s	sys 0.008s
W20260709 13:19:19.200350 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 693396us
I20260709 13:19:19.172739 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:19.169488 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.013s	sys 0.007s
I20260709 13:19:19.183043 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:19.201596 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2269) took 1754 ms (client timeout 2050 ms). Trace:
I20260709 13:19:19.106379 16116 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.193764 10196 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
W20260709 13:19:19.201957 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.007s
W20260709 13:19:19.202143 10199 rpcz_store.cc:269] 0709 13:19:17.447517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.449440 (+  1923us) service_pool.cc:224] Handling call
0709 13:19:19.201394 (+1751954us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b540039f154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:19.201527 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242288000}
I20260709 13:19:19.213760 16124 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:19.215229 16124 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:19.215691 16124 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:19.221261 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 300323us
I20260709 13:19:19.193974 16029 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.232290 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.233280 10130 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:19.236248 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.239504 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.239775 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.240401 10126 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:19.247591 16388 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.248492 16388 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.228338 10129 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:19.257525 10129 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:19.260218 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.225586 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:19.261320 10129 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:19.225169 16379 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:19.294282 16379 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:19.294797 16379 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.295800 16379 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.296595 16029 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.297290 16379 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:19.269266 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.301100 16345 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.745s	user 0.014s	sys 0.000s
I20260709 13:19:19.301862 16345 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.303361 10047 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:19.304651 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.306161 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.311344 16220 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.312650 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:19.317508 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92441.8us
I20260709 13:19:19.318527 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:19.269533 10131 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.237421 16124 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.279155 16388 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.289675 10130 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:19:19.292552 16375 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.324159 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.333604 10199 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:19.333768 10129 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.335655 10128 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.336205 16364 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:19.338904 10124 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:19.340494 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:19.348531 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:19.362363 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.350984 10131 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:19.351919 16388 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.357832 16329 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.368482 10130 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.368347 10129 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:19.366735 10122 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:19.365315 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:19.369504 10204 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:19.333107 16372 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.371832 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:19.338176 16220 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:19.371461 16329 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.366092 16266 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:19.371696 15786 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:19.362519 16235 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:19.379355 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:19.372172 16336 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:19.385463 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.362756 16029 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:19.362650 16364 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:19.370640 16386 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:19.375727 16124 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.395740 16329 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.388705 16332 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.370390 16357 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:19:19.392997 16029 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.425812 16029 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.425904 16345 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.378345 16358 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:19.427513 16358 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:19.427938 16358 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.429075 16358 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.430784 16358 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.388859 15786 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.431687 15786 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.432261 16372 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.379150 16220 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.434528 16220 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.420619 16386 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.415843 16364 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.389168 16235 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.404699 16375 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.407823 16349 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:19.413933 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2929) took 1240 ms. Trace:
I20260709 13:19:19.370329 16089 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:19.389117 16208 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:19.424283  9948 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.454772 16115 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.454346  9948 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.425220 16336 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.456863  9948 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.461707 16347 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.370922 16388 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.466684 16388 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.376487 16387 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
W20260709 13:19:19.468654 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.471803  9948 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.479491 16186 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:19.480129 16186 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.481178 16186 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.486204 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:19.488850 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.489809 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.490301 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.491797 16363 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:19.492388 16363 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.493763 16087 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:19.493350 16363 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.496119 16332 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:19.449326 16235 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.437888 16386 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.488981 16387 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:19.502924 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:19.503491 16387 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.503782 10891 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:19.504017 16387 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:19.482867 16186 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:19.504140 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:19.447132 10049 rpcz_store.cc:276] 0709 13:19:18.173889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.374040 (+200151us) service_pool.cc:224] Handling call
0709 13:19:18.374703 (+   663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.375017 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:18.497959 (+122942us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.622887 (+124928us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.410306 (+787419us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b1800260380. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:19:19.410641 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.410889 (+   248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.410954 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:19.411012 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.411100 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.413858 (+  2758us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36630,"prepare.queue_time_us":1711424,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":4344,"raft.queue_time_us":16845,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":145714,"spinlock_wait_cycles":15315584,"thread_start_us":200539,"threads_started":2,"wal-append.queue_time_us":278570}
I20260709 13:19:19.495019 16363 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:19.504140 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:19.509253 10893 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:19.494292 16087 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:19.511951 16087 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.514483 16220 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.515904 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2429) took 2631 ms. Trace:
W20260709 13:19:19.516669 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.517145 16220 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.519701 16221 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.520215 10891 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:19:19.523787 16117 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:19.526160 16377 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.453s	user 0.000s	sys 0.003s
I20260709 13:19:19.516412 10050 rpcz_store.cc:276] 0709 13:19:16.884838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.899908 (+ 15070us) service_pool.cc:224] Handling call
0709 13:19:16.900243 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.900608 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:16.900642 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:16.901195 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:19:17.135070 (+233875us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:17.481101 (+346031us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.515299 (+2034198us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b540027d920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7f733 00007fa278004dbf 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c
0709 13:19:19.515369 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.515502 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.515549 (+    47us) raft_consensus.cc:1685] finished
0709 13:19:19.515671 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.515834 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84561,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":206744,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":25098,"spinlock_wait_cycles":41648000,"thread_start_us":657225,"threads_started":3,"wal-append.queue_time_us":378147,"wal-append.run_cpu_time_us":5254,"wal-append.run_wall_time_us":1462493}
I20260709 13:19:19.530438 10200 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:19:19.530606 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:19.531498 10891 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:19.532265 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.533999 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.534916 10196 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:19.451951 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:19.539476 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 252760us
I20260709 13:19:19.541302 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:19.545221 10864 connection.cc:256] RPC call timeout handler was delayed by -1.5217035150001266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18020000000000003s for a response
W20260709 13:19:19.545162 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 93308.6us
I20260709 13:19:19.545812 10893 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:19.546733 10864 connection.cc:256] RPC call timeout handler was delayed by -1.3379538270000921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 13:19:19.549378 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.562516 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.555745 16060 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.564344 16060 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.567394 16325 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.567704 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.568454 10203 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:19.568611 16336 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:19.568984 16221 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:19.569521 16221 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.569897 16221 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:19.572719 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:19.574035 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:19.574770 10900 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.575894 16372 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.576333 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:19.576467 16372 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.574764 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:19.568531 16386 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:19.582650 16386 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.582818 10897 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:19.583081 16386 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:19.583663 16372 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.577282 16342 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:19.587785 16342 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.588516 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.578588 10892 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:19.568109 16325 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.579382  9948 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.579913 16060 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.609939 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.610491 10900 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:19.550871 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.612547 10894 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:19:19.613258 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62496.3us
I20260709 13:19:19.615149 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:19.616331 10055 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:19.617482 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.618975 10048 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:19.619575 10048 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:19.621670 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.624827 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:19.614982 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.640100 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.640960 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.642081 10050 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:19.592772 16345 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.638883 16336 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:19.607462 16372 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.644321 16336 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:19.647323 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.647612 16336 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:19.648274 10197 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:19.648450 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.649914 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.644165 16345 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:19.652801 16393 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:19.653577 16345 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.656840  9948 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.636979 10048 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.660492 10048 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:19.661374 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.647547 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.644212 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.663919 10054 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:19.597600 16316 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:19.593967 16326 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.661278 10049 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:19.673466 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3451) took 1086 ms. Trace:
I20260709 13:19:19.675071 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:19.694447 10126 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:19.679946 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2930) took 1475 ms. Trace:
I20260709 13:19:19.695047 10047 rpcz_store.cc:276] 0709 13:19:18.204139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.399809 (+195670us) service_pool.cc:224] Handling call
0709 13:19:18.400211 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.303805 (+903594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.679458 (+375653us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:19.679834 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076417920}
I20260709 13:19:19.693864 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.682396 10205 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.715448 10891 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:19.716558 10899 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:19.694006 10204 rpcz_store.cc:276] 0709 13:19:18.587611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.587801 (+   190us) service_pool.cc:224] Handling call
0709 13:19:18.588149 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.375166 (+787017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.668135 (+292969us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b54003598d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:19.673366 (+  5231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1795930368}
I20260709 13:19:19.659494 16060 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:19.725704 16385 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:19.646319 15786 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.634204 16287 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.639693 16321 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.600538 16364 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.693113 16375 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:19.662164 16220 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:19.731649 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.732052 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.756896 10123 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:19.645005 16372 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.761269 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.729607 16316 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:19.738142 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 363702us
I20260709 13:19:19.719295 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.773422 10122 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:19.754360 10205 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:19.718209 16329 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.803916 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:19.805095 10047 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:19.806217 10048 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:19:19.730108 15661 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:19.660852 16235 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:19.763283 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 384001us
I20260709 13:19:19.729837 16338 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:19.722033 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.665448 16029 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.770768 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.762362 10123 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:19.782285 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3452) took 1193 ms. Trace:
I20260709 13:19:19.783931 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:19.743763 16167 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.806881 10048 pending_rounds.cc:85] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 11
I20260709 13:19:19.841890 10048 pending_rounds.cc:107] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:19:19.842344 10048 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:19.844270 10050 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:19.836467 10205 rpcz_store.cc:276] 0709 13:19:18.588486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.590295 (+  1809us) service_pool.cc:224] Handling call
0709 13:19:18.590631 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.681760 (+1091129us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b580060b9d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277ffafe7 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4
0709 13:19:19.781089 (+ 99329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.782173 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467400576}
W20260709 13:19:19.830084 16258 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:19:19.815142 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.820745 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.860806 10204 raft_consensus.cc:2368] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:19:19.741390 16287 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.826601 10051 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:19.829252 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.829936 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:19.867301 10128 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:19.867630 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.868723 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:19.764034 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 276176us
I20260709 13:19:19.812335 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.721135 16121 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:19:19.876463 16287 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.722057 16326 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.879084 16326 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.840432 10122 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:19:19.882464 16344 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:19.840236 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.835601 10126 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:19.762348 16375 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:19.891178 10051 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.833600 16116 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:19.771924 16220 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:19.856948 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:19.900148 10048 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:19:19.900334 10051 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:19.745714 16321 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:19.904170 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2738611329998548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:19:19.658923  9948 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:19.823558 10055 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:19.871477 16089 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.873188 15786 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:19.835246 16329 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.831028 16358 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:19.874486 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:19.886574 10124 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:19.883682 10056 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:19.893045 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.896440 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.906733 16326 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.896209 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:19.833992 16107 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:19:19.859783 16385 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:19.903358 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:19.903494 16395 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.908022 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:19.904664 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 375391us
I20260709 13:19:19.911283 10122 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:19.936354 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.831684 10095 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.558s (SENT) [suppressed 21 similar messages]
I20260709 13:19:19.894003 16342 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:19.856875 16263 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:19:19.894116 16344 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:19.936654 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.949949 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.934789 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 122591us
W20260709 13:19:19.931782 16322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.034s	sys 0.042s
I20260709 13:19:19.943907 10202 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:19.936066 16239 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:19.903657 16220 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.952894 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.934792 16398 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:19:19.854496 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.937974 16381 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:19.967103 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.964664 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.982467 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:19.983340 10126 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:19.942580 16348 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:19.987478 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:19.958189 16322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.034s	sys 0.042s
I20260709 13:19:19.915603 16089 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:19.999141 10204 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:19.958086 16239 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.047432 16239 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.049615 16239 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:19.961707 16385 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:20.052341 16385 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.981427 16396 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:20.055619 16396 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:20.056839 16330 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:19:19.910427 16316 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.012360 16321 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:19.987531 16060 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:20.071390 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84046.1us
I20260709 13:19:20.017481 16329 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:20.018258 16379 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.002s
I20260709 13:19:20.019196 16336 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.948770 16326 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:19.951696 16395 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.023880 16235 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.025058 10047 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:20.025831 16089 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.026312 16322 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:20.026474 10126 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:20.027251 16221 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.030340 16029 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.032390 16398 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:19.990710 16116 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:20.042517 10096 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.550s (SENT)
W20260709 13:19:20.043699 16371 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.246s	user 0.005s	sys 0.000s
W20260709 13:19:20.045441 16361 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:20.063357 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 110554us
W20260709 13:19:20.075598 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.099836 10201 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:20.112720 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:20.113637 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.113828 16108 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:20.114519 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:20.072594 16396 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:20.151363 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108880us
W20260709 13:19:20.134191 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 494135us
I20260709 13:19:20.159011 10125 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.193066 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:20.193979 10130 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.153849 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:19.994935 16348 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:20.098635 16373 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.015s	sys 0.050s
I20260709 13:19:20.070966 16121 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:20.217541 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.223681 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69932.2us
W20260709 13:19:20.224043 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.108968 16329 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:20.218457 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:19.965657 16375 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.080886 16364 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:20.151250 16116 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.234668 10895 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:20.222267 16373 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.015s	sys 0.059s
I20260709 13:19:20.107174 16325 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.200938 16372 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:20.221613 16386 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.010s
I20260709 13:19:20.201448 16321 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.198668 16344 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:20.284431 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2297) took 1688 ms. Trace:
I20260709 13:19:20.284953 10198 rpcz_store.cc:276] 0709 13:19:18.595765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.598645 (+  2880us) service_pool.cc:224] Handling call
0709 13:19:18.598976 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.599260 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.599290 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.599821 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.800940 (+201119us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:19.843521 (+1042581us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b18002d4180. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:19:20.042821 (+199300us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:20.106660 (+ 63839us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.115608 (+  8948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.115720 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:20.284149 (+168429us) raft_consensus.cc:1685] finished
0709 13:19:20.284174 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.284361 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30517,"prepare.queue_time_us":1071598,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":29,"raft.queue_time_us":8317,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":158,"spinlock_wait_cycles":14447744,"thread_start_us":419560,"threads_started":2,"wal-append.queue_time_us":175610}
W20260709 13:19:20.287835 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 456152us
I20260709 13:19:20.191910 16398 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.250618 10130 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:20.299372 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58484.1us
W20260709 13:19:20.299350 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 309367us
W20260709 13:19:20.304193 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 339655us
I20260709 13:19:20.300453 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:20.307765 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89445.1us
I20260709 13:19:20.235424 16363 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:20.309118 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.310331 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:20.239796 16379 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:20.313117 10198 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:20.245692 16263 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.311890 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:20.326171 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:20.331403 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:20.331436 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 602011us
I20260709 13:19:20.334929 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:20.297748 16029 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:20.236945 16121 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.340538 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.339951 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.303670 16398 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:20.239219 16329 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.380928 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.333873 16322 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:20.314361 16363 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:20.323035 16089 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:20.333273 16358 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:20.302891 16386 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.352592 16240 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.396984 10899 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:20.297328 16344 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.358017 16108 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:20.355161 16379 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.305491 16321 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.242748 16060 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.394001 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.372668 15813 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.388163 16371 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:20.364043 16357 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:20.361038 16394 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:20.373509 16330 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.371999 16029 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.333683 16263 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.356048 16262 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:19:20.353224 16258 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:19:20.398916 16322 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.418797 16381 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:20.400022 16364 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.493495 16381 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:20.494158 16381 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:20.493984 16364 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.496230 16379 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.497752 16364 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.418169 16363 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:20.426066 16186 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.432916 16352 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.467667 16394 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:20.471297 16408 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:19:20.373023 16116 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.602460 15786 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:20.604593 15786 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.606339 16116 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.333806 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:20.621321 16121 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.444406 16240 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.624027 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 959) took 1054 ms. Trace:
I20260709 13:19:20.625914 16240 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:20.627821 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 293939us
I20260709 13:19:20.629806 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:20.483503 16221 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:20.410053 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.578063 10124 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:20.601310 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:20.636831 10891 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:20.635743 15786 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.602382 16287 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:20.639286 16287 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.640549 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 2912) took 1318 ms. Trace:
I20260709 13:19:20.641202 10128 rpcz_store.cc:276] 0709 13:19:19.321517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.346610 (+ 25093us) service_pool.cc:224] Handling call
0709 13:19:19.347098 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.885047 (+537949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.639813 (+754766us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54000a00d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:20.640286 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195650944}
I20260709 13:19:20.641959 15786 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.459460 15813 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:20.490844 16167 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
W20260709 13:19:20.491636 16208 consensus_meta.cc:287] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.016s	user 0.011s	sys 0.004s
I20260709 13:19:20.643908 16208 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:20.330929 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:20.645818 15813 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.646922 15920 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.631759 16107 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:19:20.578600 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.505927 16329 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.481863 16344 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.694631 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:20.500126 16352 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.461920 16357 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.714349 16357 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.476425 16386 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.461761 16330 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:20.485464 16387 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.981s	user 0.018s	sys 0.003s
I20260709 13:19:20.716826 16387 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.718207 16330 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:20.524729 16394 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.476027 16395 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.730346 10898 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:20.480201 16404 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:19:20.487069 16029 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.714401 16352 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:20.478729 16413 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:19:20.739579 16329 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:20.743556 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:20.479542 16416 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:19:20.747390 16429 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.748438 16429 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.568187  9948 catalog_manager.cc:5697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:20.751674 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.739790 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2434) took 1127 ms. Trace:
I20260709 13:19:20.648080 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3471) took 1001 ms. Trace:
I20260709 13:19:20.624787 10893 rpcz_store.cc:276] 0709 13:19:19.569257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.580214 (+ 10957us) service_pool.cc:224] Handling call
0709 13:19:19.580640 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.660181 (+ 79541us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:19.960159 (+299978us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.284568 (+324409us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:20.284629 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:20.284678 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.623218 (+338540us) spinlock_profiling.cc:243] Waited 79.2 ms on lock 0x7b5400953b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:20.623328 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.623703 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22703,"prepare.queue_time_us":361235,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":2908,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6693,"spinlock_wait_cycles":182160768,"thread_start_us":609404,"threads_started":2,"wal-append.queue_time_us":531336}
I20260709 13:19:20.652243 16107 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.648032 15920 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:20.682010 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 351053us
I20260709 13:19:20.500213 16322 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:20.689595 16121 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:20.690462 16409 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:20.693751 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.694258 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.694289 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.696074 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.699193 10125 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.476943 16348 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.667604 16087 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.764856 16107 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.755697 10201 rpcz_store.cc:276] 0709 13:19:19.646392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.651241 (+  4849us) service_pool.cc:224] Handling call
0709 13:19:19.651621 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.193617 (+541996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.647325 (+453708us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002fbcd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:20.647814 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029533952}
I20260709 13:19:20.769919 16107 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:20.710660 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.772333 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:20.708930 16115 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:20.773888 16115 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:20.775974 16115 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.732416 16326 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:20.725356 16408 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:19:20.618669 16263 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:20.801347 16321 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.759218 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.802348 16321 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.763707 16409 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:20.764060 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.804798 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.804699 16409 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.764811 10198 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:20.766204 10200 raft_consensus.cc:2410] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:19:20.778791 10202 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:20.809286 16387 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.781002 16352 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.811553 16352 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.812923 16336 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.801450 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 962) took 1231 ms. Trace:
I20260709 13:19:20.783536 16121 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:20.814437 10898 rpcz_store.cc:276] 0709 13:19:19.570360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.571797 (+  1437us) service_pool.cc:224] Handling call
0709 13:19:19.572227 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.731192 (+1158965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.800960 (+ 69768us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400640854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:20.801327 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2641822464}
I20260709 13:19:20.784140 16420 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:20.818356 16420 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:20.755313 10052 rpcz_store.cc:276] 0709 13:19:19.612541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.612689 (+   148us) service_pool.cc:224] Handling call
0709 13:19:19.613187 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:19.654592 (+ 41405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.654624 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:19.655274 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 13:19:19.964386 (+309112us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:19.964434 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.964549 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:20.739143 (+774594us) raft_consensus.cc:1685] finished
0709 13:19:20.739190 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.739511 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40858,"raft.queue_time_us":42486,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"thread_start_us":308650,"threads_started":1,"wal-append.queue_time_us":497206}
I20260709 13:19:20.820003 16420 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.823707 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.785290 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.707881 16287 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:20.791428 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.616s (SENT)
I20260709 13:19:20.827376 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:20.829475 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.816917 16121 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.831146 16240 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:20.830690 10125 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:20.833158 16330 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:20.793278 16411 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.374s	user 0.003s	sys 0.000s
I20260709 13:19:20.794813 16347 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.833688 16330 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:20.834414 16347 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.835009 16330 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:20.760145 16428 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
I20260709 13:19:20.743600 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.836709 16330 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.837278 10128 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:20.802587 10900 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:20.839211 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.839732 16347 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.814721 16352 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:20.841032 16387 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:20.817576 15920 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:20.861308 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 88879.8us
I20260709 13:19:20.863395 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:20.870163 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.884440 10124 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:20.885728 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:20.886160 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 551169us
I20260709 13:19:20.859097 16347 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:20.887506 16347 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:20.854404 16352 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.890486 16352 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:20.892539 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.900799  9948 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:20.905009 15661 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.071s	user 0.025s	sys 0.001s
I20260709 13:19:20.906066 16381 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.860976 16429 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.858771 16321 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:20.916543 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.945756 16321 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:20.946171 10130 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:20.951511  9948 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:20.918787 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.920521 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.921599 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:20.925817 16332 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:20.929612 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.932132 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:20.933025 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:20.890515 16347 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:20.934374 15786 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:20.935058 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.936846 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.937328 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.937709 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.938462 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.940184 10205 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:20.936295 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.941337 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.959090 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:20.917320 16352 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:20.964476 10126 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:20.985721 10199 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:20.986057  9948 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:20.987342 16432 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:19:20.987438 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3461) took 1470 ms. Trace:
I20260709 13:19:20.989275 10199 rpcz_store.cc:276] 0709 13:19:19.517436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.525103 (+  7667us) service_pool.cc:224] Handling call
0709 13:19:19.525577 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:20.986388 (+1460811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.987103 (+   715us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001d0454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:20.987353 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3324725120}
I20260709 13:19:20.946584 16321 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:19:20.971887 16332 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.997784 10131 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:20.997990 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:20.998422 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.000080 15920 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.001139 15920 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:21.001950 15920 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.005136 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:21.005389 10129 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:21.007243 16332 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:21.015436 10202 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:21.017205 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.018098 10202 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:21.019716 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:21.020010 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 98639.2us
I20260709 13:19:21.021996 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.006805 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.000183 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:20.994431 10203 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:21.023058 16329 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:20.988682  9948 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 10, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 10 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.029814 16329 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.039806 10198 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:21.050899 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:21.053207 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 62465.6us
I20260709 13:19:21.051882 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:21.056499 16363 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.011s
I20260709 13:19:21.057511 15920 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.059605 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.057152 16363 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.074960 16322 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.087697 10204 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:21.091647 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 231688us
I20260709 13:19:21.105129 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:21.106498 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.092259 16121 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:21.060389 10898 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:21.034978 16329 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.095989 16381 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.100891 16336 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:21.124730 16434 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.125113 15920 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:21.094293 16287 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:21.125569 16434 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.126482 16287 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.128508 10130 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:21.131688 10196 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:19:21.132557 16363 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.134691 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3476) took 1314 ms. Trace:
W20260709 13:19:21.146900 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.135686 10196 rpcz_store.cc:276] 0709 13:19:19.820005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.820280 (+   275us) service_pool.cc:224] Handling call
0709 13:19:21.134408 (+1314128us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540039ec54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:21.134602 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2888146688}
I20260709 13:19:21.112936 16121 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.129290 16287 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.172968 16167 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:21.173593 16167 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.174021 16167 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.168908 16121 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.171756 16321 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.212742 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.213829 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.214848 10204 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.216199 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.217198 10201 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.218339 15920 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:21.218906 15920 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.219318 15920 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:21.222939 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 117595us
W20260709 13:19:21.213832 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111759us
W20260709 13:19:21.225332 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 118965us
W20260709 13:19:21.227882 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 182735us
I20260709 13:19:21.229835 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.229928 16124 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:21.231789 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:21.230861 16124 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.230825 10205 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:21.242518 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 173756us
I20260709 13:19:21.246692 10198 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.234170 16124 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.254458 10200 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:21.255221 16344 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.252133 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.259891 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:21.262215 16449 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.169s	user 0.003s	sys 0.000s
I20260709 13:19:21.266881 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:21.234128  9948 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:21.269974 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 71947.1us
I20260709 13:19:21.270200 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.271113 10201 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:21.273355 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.275462 16322 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.276767 16322 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:21.281605 16448 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.202s	user 0.002s	sys 0.000s
I20260709 13:19:21.283463 16345 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.283809 16287 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.284406 16345 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.228226 16321 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.289896 16321 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.237713 16387 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:19:21.283602 16432 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:21.292120 16344 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.292760 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.293655 16107 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:21.294190 16107 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:21.295915 15661 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:21.295549 16358 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:21.248512 16060 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:19:21.276882 16352 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:21.285229 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:21.341207 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46769.4us
I20260709 13:19:21.357448 10205 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:21.342564 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 69292.9us
W20260709 13:19:21.338307 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.046s (SENT)
I20260709 13:19:21.250532 16326 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:19:21.293675 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:21.355958 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 116809us
I20260709 13:19:21.366626 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:21.362233 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65584.9us
I20260709 13:19:21.296492 15661 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:21.371949 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75872.3us
I20260709 13:19:21.285097 16348 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.337234 16344 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.379505 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.386276 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.361948 16352 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:21.295627 16107 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.371780 15661 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.396786 16107 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.399319 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.399995 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.400372 16407 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.402336 16352 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.404601 16352 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.407763 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:21.409245 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 115701us
I20260709 13:19:21.410208 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.411007 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.411952 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.412451 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.413892 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.414664 16321 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.393673 16429 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.415584 16358 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:21.415341 16321 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.415786 16429 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.416504 16358 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.373761 16387 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:21.391981 16434 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.394073 16348 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.417778 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.357188 16287 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.431715 10051 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:21.433650 16287 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.435343 16087 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:21.436538 16381 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:21.436961 16381 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:21.438439 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.407847 10204 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:21.440547 16330 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:21.419265 16432 consensus_queue.cc:1175] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "db052bef7536453fa2d7f5a00e314a18" responder_term: 20 status { last_received { term: 19 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:19:21.419790 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 165617us
I20260709 13:19:21.443524 16321 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:21.421180 16451 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:19:21.423382 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:21.424314 16322 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.418198 16429 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.446998 16322 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.449579 16322 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.424721 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:21.452212 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:21.453171 15786 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:21.453773 16434 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:21.454452 16434 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:21.439530 16381 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.455659 16434 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.456171 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.456717 16381 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.457000 10899 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:21.458159 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.458490 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.462586 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:21.423940 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.464118 16330 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:21.415357 16349 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:21.464708 16330 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.465005 16349 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.465546 16349 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.465356 16330 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.425911 16348 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.426162 10052 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:21.468586 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.468942 16345 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.344570 16332 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.470340 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.471335 10048 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:21.472015 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.470376 16388 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.472862 10047 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.473058 16388 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.474323 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.475086 10051 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.475893 16388 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.476241 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.462709 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.477589 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.477415 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.478500 10051 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:21.479250 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.480105 10048 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.479956 16429 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:21.479409 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.480854 16429 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.481628 16332 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.482793 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.490273 10051 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.491569 16329 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.492094 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.493170 10053 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:21.483652 16429 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:21.463070 16398 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.987s	user 0.009s	sys 0.012s
I20260709 13:19:21.495625 16398 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.497258 10895 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:21.498193 16329 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.498600 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 463) took 1105 ms. Trace:
I20260709 13:19:21.500124 10895 rpcz_store.cc:276] 0709 13:19:20.393316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.393567 (+   251us) service_pool.cc:224] Handling call
0709 13:19:21.498292 (+1104725us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004650d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:21.498478 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404491008}
I20260709 13:19:21.395411 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:21.503000 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 107693us
I20260709 13:19:21.503154 16363 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:21.503818 16363 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.503948 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.504289 16363 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.504803 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.506727 10198 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:21.505816 16345 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:21.512727 10052 raft_consensus.cc:2410] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:21.352589 16409 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:21.525640 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.531575 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.535390 16345 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:21.541332 10047 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:21.538656 10048 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.542071 10205 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
W20260709 13:19:21.544768 16420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.019s	sys 0.029s
I20260709 13:19:21.552358 16398 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.535480 10051 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:21.556849 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.606113 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.557578 16364 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.559104 16258 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:21.620484 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.563941 16240 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.566344 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:21.570007 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.571636 16336 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:21.562649 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.572731 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.575616 16124 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.575820 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.595666 16432 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:21.596345 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92498us
I20260709 13:19:21.606417 10054 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:21.563301 16357 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.609448 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:21.612126 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:21.616376 16364 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.617426 16345 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.618417 10895 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:21.607280 16420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.020s	sys 0.029s
I20260709 13:19:21.619187 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.563688 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:21.622211 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 226327us
I20260709 13:19:21.558492 16349 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.621403 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.667397 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.667862 10198 raft_consensus.cc:606] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.658628 10894 raft_consensus.cc:2376] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22. Re-sending same reply.
I20260709 13:19:21.672238 10900 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:21.620128 15661 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.683007 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.686301 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:21.686000 16357 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.686641 10053 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.687872 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.688097 15661 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.657867 16087 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.688638 16357 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.689038 16087 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.681041 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.676088 10198 raft_consensus.cc:614] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:21.684492 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.689298 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.677701 10056 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:21.678035 10055 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:21.679947 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.681015 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:21.681322 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:21.682657 10125 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:21.620847 16258 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:21.692768 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:21.702198 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.706367 10047 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.707229 10200 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:19:21.708009 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.708846 10048 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:21.709610 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.710191 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.702198 16087 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.703312 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.711176 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.711313 10052 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:21.631650 16167 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:21.712749 16372 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.006s	sys 0.025s
I20260709 13:19:21.705470 10128 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:21.704420 16258 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.708540 16420 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:21.715608 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.720181 16349 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.720664 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:21.722685 15920 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:21.726830 16121 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.002s
I20260709 13:19:21.727444 16121 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.730110 16121 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.730326 15920 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.731565 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.732703 16124 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.730988 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.700121 10124 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:21.716737 16329 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:21.717306 16330 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.712615 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.741367 10052 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:21.712621 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:21.732594 10056 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 10.
I20260709 13:19:21.658566 16336 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:21.717521 16258 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:21.723271 10053 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:21.740310 10049 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:21.744010 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:21.747547 10204 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:21.748411 10202 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:21.749301 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:21.751726 10054 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:21.753166 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:21.754056 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85014.8us
I20260709 13:19:21.756076 10047 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:21.736073 16372 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.006s	sys 0.025s
I20260709 13:19:21.666368 16363 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.671706 16432 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.760814 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:21.760313 15786 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:19:21.767724 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.768790 10197 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:21.769605 10122 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:21.767825 16348 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:19:21.768775 10124 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:21.776162 10198 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:21.714301 16167 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:21.789938 10199 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.796597 10049 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:21.777441 10052 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:19:21.793289 10205 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:21.779676 16336 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.782559 16330 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.788915 10203 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:21.790000 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 59119.6us
I20260709 13:19:21.740904 16329 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:21.777267 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:21.608491 16409 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.817986 10051 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:21.820806 10050 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.795974 16381 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.821139 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:21.836704 10900 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:21.754952 15920 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:19:21.827477 16087 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.840250 16087 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.842414 16336 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.759855 16121 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:21.848956 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.859525 10900 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:21.861289 16263 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:21.857326 10052 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:21.846153 16329 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.813125 16420 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:21.864440 16364 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:21.864869 16329 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.867700 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:21.868546 10124 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.869882 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:21.870298 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 117263us
I20260709 13:19:21.870756 10125 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.871415 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:21.872361 10130 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:21.873270 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.874078 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:21.874836 10123 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:21.876180 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:21.876430 16121 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:21.877038 10122 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:21.843012 16087 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.877557 16348 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:21.878209 16348 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:21.846339 16349 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:21.878643 16348 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.879215 16349 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:21.879555 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:21.880983 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.881677 16349 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:21.882468 16349 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:21.851004 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:21.869930 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 114709us
I20260709 13:19:21.790936 16432 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:21.884936 10055 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 12.
I20260709 13:19:21.886423 10048 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:21.887742 16087 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:21.888429 16087 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.890203 16087 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:21.882752 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.895824 10056 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:21.895823 10050 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:21.786919 16240 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.898617 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:21.896411 10056 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:21.862785 16409 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:21.903340 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52479.6us
I20260709 13:19:21.882874 16349 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.877578 16121 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:21.914328 16349 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:21.916841 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:21.918418 10900 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:21.919588 16409 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:21.918689 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:21.922552 16322 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.925444 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:21.931560 10201 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:21.933722 10123 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:19:21.934367 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51719.3us
I20260709 13:19:21.923444 10056 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:21.937269 10124 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:21.934372 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:21.948408 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.907163 16379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.017s	sys 0.020s
I20260709 13:19:21.950985 16330 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:21.951781 15786 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:21.955648 10020 leader_election.cc:400] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:19:21.913972 16060 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:21.979995 10020 leader_election.cc:403] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:19:21.950106 16364 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:21.957250 16409 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:21.980530 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.923595 16322 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:19:21.915572 16121 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:21.966840 16124 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:21.981907 16364 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:21.999135 10056 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:21.976063 16379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.017s	sys 0.020s
W20260709 13:19:21.984978 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50678.9us
I20260709 13:19:22.003726 10125 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:22.009538 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.011282 16372 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:19:22.011620 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.012310 16352 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:22.012686 16357 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:22.013015 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:21.972149 16398 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.998703 16364 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.013181 16357 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.044677 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.042632 16398 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.072165 16357 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.073982 16398 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:22.045876 16240 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.075834 15786 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:22.047415 16330 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:22.076081 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.076642 16330 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.077749 16330 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:22.054242 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.056740 16363 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.062268 16381 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:22.013347 16258 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:22.063683 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 108281us
I20260709 13:19:22.013397 16121 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.067960 16379 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:22.042268 16124 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.075544 16434 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:22.050626 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45685.6us
I20260709 13:19:22.076498 15786 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.078902 16240 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:22.082001 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.083922 16258 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.098867 10047 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:22.099495 10047 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:22.100426 10047 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.104489 10047 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.091276 16124 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.108331 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.112587 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.113526 10052 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:22.115649 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.101604 16258 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.112794 16121 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.142176 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:22.131356 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.144063 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.145144 10050 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:22.146457 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.142976 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:22.153167 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.153517 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.153440 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.154475 10052 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:22.155521 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.156733 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.157472 10131 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:22.157380 15786 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.107502 16240 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.162251 16240 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.163306 16364 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.165947 16364 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:22.166635 16349 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.169992 16349 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:22.171497 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.173296 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.174336 10048 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.117240 16124 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.177829 10056 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:22.179240 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:22.209517 10900 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.213160 10894 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:22.215679 16379 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:19:22.221423 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.222340 16379 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.223028 16379 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.225528 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:22.226064 16379 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.227420 16458 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:22.236707 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 134736us
I20260709 13:19:22.238255 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:22.151656 16322 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:19:22.241977 16461 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:22.243978 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:22.206126 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:22.195912 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.245738 10895 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:22.252117 10095 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.018s	sys 0.055s
W20260709 13:19:22.254768 10095 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.019s	sys 0.056s
I20260709 13:19:22.201864 10047 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:19:22.179524 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.186254 16363 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.253223 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:22.260039 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 355074us
W20260709 13:19:22.256456 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 810542us
I20260709 13:19:22.260751 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:22.264475 10123 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.256901 10125 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:22.266440 10047 pending_rounds.cc:85] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:22.269590 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:22.254528 16322 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:22.276768 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:22.268064 10047 pending_rounds.cc:107] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:19:22.284229 10047 raft_consensus.cc:2889] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:22.277313 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.265798 16458 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:22.292282 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.277498 16349 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:22.290302 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46469.3us
I20260709 13:19:22.305238 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.295078 10056 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:22.319484 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81447.9us
I20260709 13:19:22.320503 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:22.355885 10048 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:22.328413 16263 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:22.354306 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:22.357724 10895 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:22.203416 16345 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:19:22.281761 16322 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.367517 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:22.260263 16087 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.262953 16432 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.268520 16363 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.391829 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87da05694091470284f66a8181207e2f"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:19:22.400606 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61023.3us
I20260709 13:19:22.400641 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.282750 16381 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:22.339273 16364 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:22.338620 16458 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:22.402946 16348 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.524s	user 0.014s	sys 0.000s
I20260709 13:19:22.364220 16263 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.380918 16352 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.404479 16322 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.414595 16461 raft_consensus.cc:2764] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:19:22.443784 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:22.413349 16434 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:22.419024 16381 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.418190 16364 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.385670 16358 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:19:22.459586 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.461033 10124 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:22.443266 10123 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:22.463303 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.430945 16458 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:22.471745 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.477316 10125 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:22.456446 16381 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:22.492563 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:22.462997 16364 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.440829 16348 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.483389 16357 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:22.523243 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.499399 16336 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:22.489205 16434 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:22.551656 10124 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:19:22.555027 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62618.7us
I20260709 13:19:22.518682 15661 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:22.560658 15661 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:22.561244 15661 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:22.563205 16467 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:19:22.567233 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:22.567953 16465 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.001s
I20260709 13:19:22.568229 10055 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:22.562342 15661 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.572567 16363 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.574589 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2950) took 1038 ms. Trace:
I20260709 13:19:22.540484 16409 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:22.545307 16336 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:22.587410 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.579282 10053 rpcz_store.cc:276] 0709 13:19:21.535917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.760608 (+224691us) service_pool.cc:224] Handling call
0709 13:19:21.761059 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:21.772235 (+ 11176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.772278 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:21.773130 (+   852us) log.cc:844] Serialized 52 byte log entry
0709 13:19:22.125513 (+352383us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:22.125573 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.125683 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.573960 (+448277us) raft_consensus.cc:1685] finished
0709 13:19:22.573985 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.574434 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10777,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"thread_start_us":351901,"threads_started":1,"wal-append.queue_time_us":509346}
I20260709 13:19:22.548305 16263 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.601655 16263 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:22.604487 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 168947us
I20260709 13:19:22.586573 16336 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.547623 16352 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.617394 16258 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.603878 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:22.639292 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.640478 10130 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:22.651734 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.652230 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.656157 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.635351 16348 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:22.641072 15661 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.637831 16409 raft_consensus.cc:2764] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:19:22.664815 16468 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:19:22.672302 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.674412 10130 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:22.662483 16434 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.681113 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:22.682250 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111636us
W20260709 13:19:22.683689 16469 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.176s	user 0.004s	sys 0.000s
I20260709 13:19:22.684362 16488 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:22.684923 16488 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:22.685262 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2474) took 1118 ms. Trace:
I20260709 13:19:22.686021 16490 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.686501 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.686721 16490 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.685329 16488 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:22.691764 10900 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:22.693624 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:22.693866 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.698266 16330 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.699268 16330 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.699754 16330 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.703743 16124 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.706128 16124 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.685947 10054 rpcz_store.cc:276] 0709 13:19:21.566376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.781083 (+214707us) service_pool.cc:224] Handling call
0709 13:19:21.781471 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.781941 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.781974 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.782588 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:19:22.123687 (+341099us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:22.535746 (+412059us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.537459 (+  1713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.539018 (+  1559us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.639012 (+ 99994us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:22.684825 (+ 45813us) raft_consensus.cc:1685] finished
0709 13:19:22.684978 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.685168 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8030,"prepare.queue_time_us":426133,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":2165,"raft.queue_time_us":3503,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":11923,"spinlock_wait_cycles":34791040,"thread_start_us":705407,"threads_started":2,"wal-append.queue_time_us":653464}
I20260709 13:19:22.717383 10122 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:22.729666 16258 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.732821 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.728034 16167 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:22.708143 16352 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.732821 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.734606 16329 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:22.737354 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:22.740386 16121 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.000s
I20260709 13:19:22.743335 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.744403 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.745931 10053 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:22.760442 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.746075 16167 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.745947 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.751579 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:22.752822 10895 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:22.753928 10898 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:22.754508 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:22.755617 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.762895 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.766985 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.768469 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.769145 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:22.770036 10900 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:19:22.766988 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.771625 10122 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:22.777289 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.777539 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.778338 10049 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:22.778473 16167 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.779201 16087 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.785604 16490 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:22.786091 10124 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:22.789494 16432 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.803769 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.804629 10124 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:22.805629 10049 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
W20260709 13:19:22.805576 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 119177us
W20260709 13:19:22.807663 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.807963 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.808779 10054 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:22.810340 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:22.813490 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:22.822829 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.823091 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 2963) took 1242 ms. Trace:
I20260709 13:19:22.826555 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.823503 10047 rpcz_store.cc:276] 0709 13:19:21.584936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.845020 (+260084us) service_pool.cc:224] Handling call
0709 13:19:21.845596 (+   576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.284953 (+439357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.822780 (+537827us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400086e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:22.823011 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582123904}
I20260709 13:19:22.831439 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.833173 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.833858 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:22.834736 10198 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:22.836421 16349 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.837044 16349 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.839569 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.840418 10125 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:22.808410 16167 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.847304 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.843259 10131 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:22.845245 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.829188 10199 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:22.843619 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.848804 10123 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 14: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:22.852258 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.852921 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:22.854987 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:22.816730 16490 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:22.858664 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.811820 16240 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:22.859011 10124 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:22.863986 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.867856 10131 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:22.864166 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:22.869310 10125 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.815796 16357 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:22.776006 16352 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.868006 16121 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:22.868134 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:22.868906 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:22.925211 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 75843.9us
I20260709 13:19:22.870323 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.884418 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:22.925212 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58483.5us
I20260709 13:19:22.928357 10131 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:22.871584 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:22.886794 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:22.929188 10052 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20. Re-sending same reply.
I20260709 13:19:22.932953 10202 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:22.867990 16379 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:19:22.934867 15661 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.942301 10197 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:22.944808 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:22.947154 10130 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:19:22.944900 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 75133.4us
I20260709 13:19:22.957527 10197 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:22.959782 10196 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:22.865690 16107 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:22.960551 10204 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:22.965868 10126 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:19:22.968125 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 174779us
I20260709 13:19:22.968125 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:22.847218 16322 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.860982 16490 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.970929 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:22.969058 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:22.986052 10196 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:22.853958 16167 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.988473 10125 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:22.989188 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:22.931819 16364 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.940301 16240 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.930778 16357 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.935580 15661 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:22.980556 16490 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.023617 10199 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:22.995152 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.033324 10900 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:22.994014 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.034225 10052 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:22.991863 16409 raft_consensus.cc:2764] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:19:23.012326 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.995283 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:23.038187 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.039721 16357 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.042092 16240 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:22.981570 16349 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.034246 10050 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:22.999425 10127 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:23.059906 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:23.060832 10049 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:23.060889 10053 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.061286 10049 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:23.062271 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:22.982383 16107 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.034474 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:22.969782 16336 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.992008 16499 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.039636 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.044873 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:23.046563 16381 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:19:23.079890 10049 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:22.999377 16363 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:23.047413 10200 raft_consensus.cc:2368] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:23.086568 10050 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:23.049839 10199 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:19:23.028985 10124 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:23.028167 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.057536 10203 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:23.006069 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:22.983623 16458 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:23.065104 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 215597us
I20260709 13:19:23.089135 10049 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.067265 15786 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.023764 16121 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:19:23.068025 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:23.092415 15786 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.092738 16121 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:23.093514 16121 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:23.071472 16352 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.094493 16121 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.058602 16461 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:19:23.073488 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.096060 10125 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:23.059590 16357 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:23.074747 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.076655 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:23.074332 16349 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:23.008111 16398 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:19:23.080850 10127 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:22.993546 16372 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:19:23.066717 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:23.000985 16330 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.091442 10202 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:23.087970 16473 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:23.068617 16496 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:19:23.094990 10198 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:19:23.085032 16240 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.074901 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80051.8us
I20260709 13:19:23.095000 15786 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.088193 16381 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.105019 10130 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:23.101631 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 95664.8us
I20260709 13:19:23.104547 16121 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.128953 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 810) took 1116 ms. Trace:
I20260709 13:19:23.129565 16330 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.113771 16504 raft_consensus.cc:2764] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:19:23.109448 10204 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:19:23.132259 15661 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:23.129493 10201 rpcz_store.cc:276] 0709 13:19:22.013060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.030492 (+ 17432us) service_pool.cc:224] Handling call
0709 13:19:22.052583 (+ 22091us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:22.053141 (+   558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:22.053177 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:22.053964 (+   787us) log.cc:844] Serialized 52 byte log entry
0709 13:19:22.379212 (+325248us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:22.415104 (+ 35892us) log.cc:844] Serialized 14 byte log entry
0709 13:19:22.416531 (+  1427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.416630 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:23.086128 (+669498us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b18003f2500. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:23.128594 (+ 42466us) raft_consensus.cc:1685] finished
0709 13:19:23.128719 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.128871 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1050,"prepare.queue_time_us":56391,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":4694,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":15020,"spinlock_wait_cycles":19512704,"thread_start_us":324669,"threads_started":1,"wal-append.queue_time_us":379117}
I20260709 13:19:23.096787 10128 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:23.117262 10053 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:23.126010 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.135447 10047 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:23.136798 16258 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.137472 16258 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.137735 16489 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:23.138481 16489 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:23.139564 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94776.5us
I20260709 13:19:23.140403 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:23.115010 16357 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.132536 10197 raft_consensus.cc:2368] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:23.141006 16240 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.145167 16322 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.149317 16458 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.150310 10205 raft_consensus.cc:606] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:23.101114 16167 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.046576 16488 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.159071 10900 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:23.167496 10205 raft_consensus.cc:614] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:23.153430 16381 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:23.170522 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48222.2us
I20260709 13:19:23.179625 16336 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.181160 10047 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:23.181342 10168 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.182214 15661 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:23.181777 10047 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:23.186236 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.186968 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:23.192334 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81009.1us
I20260709 13:19:23.181856 16381 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:19:23.115921 16363 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.202442 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.195137 10125 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:23.205370 10126 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.198659 10197 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:23.194542 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:23.189021 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.202442 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.199569 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.206382 10127 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:23.207590 10047 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.208345 10128 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:23.206568 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:23.197753 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:23.210611 15786 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:23.204802 10049 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:19:23.212018 10049 pending_rounds.cc:85] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:19:23.212419 10049 pending_rounds.cc:107] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:19:23.212743 10049 raft_consensus.cc:2889] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.216806 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.220850 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.194175 16432 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:23.239526 16432 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.240737 16489 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.242165 16258 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.243496 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.244333 10128 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:23.248709 16499 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:23.250268 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:23.252724 16488 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:23.253022 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.255795 10047 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:23.224004 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:23.257081 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:23.257825 10201 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.225260 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.228294 16087 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.230988 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.268602 10051 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:23.217497 15786 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.215651 16124 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.233594 16409 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:23.218438 16458 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:23.235728 16398 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:23.272228 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.235880 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:23.237114 16502 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:19:23.193903 16434 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:23.264569 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:23.251047 10049 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:23.269755 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.273765 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:23.284322 10205 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:23.277226 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52065us
W20260709 13:19:23.288206 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52421.5us
I20260709 13:19:23.286204 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:23.289951 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.289122 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.300371 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.298988 16473 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:23.303252 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:23.304968 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81073.8us
I20260709 13:19:23.305979 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:23.307045 16473 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:23.307200 16499 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.268870 16330 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.275799 15786 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.272228 16409 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:23.319205 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54789.8us
I20260709 13:19:23.268146 16087 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.307734 16499 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.310194 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:23.276191 16322 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:23.279002 16258 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.331879 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:23.334892 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 124357us
I20260709 13:19:23.203293 16107 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.324060 16330 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:23.328898 16478 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:23.330106 16488 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:23.337399 16352 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.278702 16458 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.350616 10049 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:23.278048 16124 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.345425 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:23.325919 16372 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:23.358048 10047 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:19:23.390816 10047 pending_rounds.cc:85] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:19:23.391165 10047 pending_rounds.cc:107] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:19:23.391458 10047 raft_consensus.cc:2889] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.336493 16087 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.335901 16358 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.240737 16432 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.338603 16499 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:23.402374 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 113373us
I20260709 13:19:23.338105 16357 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.373920 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:23.375715 10131 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:23.361291 16330 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.335930 16461 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:23.380574 16364 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.335714 16348 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:23.340322 16329 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.395426 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.396493 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:23.396186 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:23.401803 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 115728us
I20260709 13:19:23.443702 10049 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:23.437264 10204 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:23.450327 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.451222 10131 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:23.451649 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:23.452788 10123 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:23.385365 16352 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.404847 16240 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.404940 16473 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:23.361763 16379 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.363888 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:23.410405 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.362994 16488 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:23.408298 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.413159 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.489061 10055 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:23.337580 16322 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.400278 16398 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:23.343839 16258 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.454439 10050 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:23.454389 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:23.351819 16363 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.404304 10205 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:23.462980 10128 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:23.475396 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 130034us
I20260709 13:19:23.478565 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:23.478606 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.478763 10131 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:23.480875 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:23.511229 10131 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:23.511668 10056 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:23.482115 10048 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:23.436144 16329 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.412009 16349 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:23.410380 16358 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.505865 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:23.506824 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144024us
I20260709 13:19:23.514154 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:23.496826 16488 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.512320 10131 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:23.541133 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62688.4us
I20260709 13:19:23.465867 16513 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:23.489475 16379 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.551196 10201 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:23.562222 10126 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:23.444761 16348 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.563824 10131 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.415261 16409 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.505043 16263 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:23.571296 10123 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:23.455823 16372 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:19:23.576776 10131 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:19:23.582852 10131 pending_rounds.cc:85] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 15
I20260709 13:19:23.582614 10049 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:23.583280 10131 pending_rounds.cc:107] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:19:23.583622 10131 raft_consensus.cc:2889] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.533113 16329 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.501471 16121 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.597299 10200 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:23.596890 16167 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:23.602101 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.602285 16409 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:23.604633 16514 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.605880 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50626.7us
W20260709 13:19:23.607074 16507 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:19:23.558955 16379 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.589174 16336 raft_consensus.cc:2764] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:19:23.625531 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111538us
I20260709 13:19:23.630964 10048 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:23.654655 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:23.597652 16513 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:23.658799 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:23.618079 16388 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:23.669773 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 217113us
I20260709 13:19:23.672539 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.618419 16398 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.622908 16167 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:23.673548 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:23.685858 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.692693 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 349337us
I20260709 13:19:23.697165 10168 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.698639 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:23.699723 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:23.683511 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 275246us
I20260709 13:19:23.671068 16388 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.670207 16511 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.724076 16504 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:23.684494 16240 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:23.680013 16398 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:23.682665 16167 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.622015 16121 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.730152 16388 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.741994 10900 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:19:23.761392 10122 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:23.747802 10126 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:23.764634 10126 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:23.698140 16525 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.725687 16434 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.737887 16511 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:23.771035 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 272206us
I20260709 13:19:23.731967 16458 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:23.831451 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.752512 16349 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 13:19:23.792343 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.862252 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.658128 16514 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.794658 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:23.746680 16381 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:23.769546 16522 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:23.717139 16461 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:19:23.747391 16398 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.757871 16124 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:23.670706 16473 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:23.736083 16504 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:23.769300 16336 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:19:23.736428 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:23.808311 16352 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:23.808640 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 107444us
I20260709 13:19:23.855278 16515 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:23.852744 16434 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.850803 16348 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.861797 16258 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.808099 16513 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.910126 16514 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.911952 16121 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:23.900576 16536 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:19:23.917394 15786 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.579s	user 0.004s	sys 0.008s
W20260709 13:19:23.896600 16512 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
I20260709 13:19:23.971583 16322 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:23.974104 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.940160 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:23.896804 16358 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.959082 10196 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.13-15.13]
W20260709 13:19:23.968865 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75309.4us
I20260709 13:19:23.940510 16504 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:23.950052 16522 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.983722 16522 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:23.957281 16434 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.984902 15786 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:23.967892 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 338192us
I20260709 13:19:23.935652 16240 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.979421 16538 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:23.984156 16511 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:23.980278 10126 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:23.994992 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 258666us
I20260709 13:19:23.996249 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:23.997860 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:24.002975 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62929.4us
I20260709 13:19:24.004218 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:24.005798 10126 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:23.969703 16381 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:23.942193 16357 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:19:23.984220 16489 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:23.990419 16121 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.989622 16515 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:24.034551 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 283254us
I20260709 13:19:24.036288 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3506) took 1021 ms. Trace:
I20260709 13:19:24.068428 10196 rpcz_store.cc:276] 0709 13:19:23.015176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.015353 (+   177us) service_pool.cc:224] Handling call
0709 13:19:23.035140 (+ 19787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.976088 (+940948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.035834 (+ 59746us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54002de6d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:24.036183 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123724544}
I20260709 13:19:23.991654 16534 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:24.066870 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 207408us
W20260709 13:19:24.076607 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80513.7us
I20260709 13:19:24.076910 10126 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:24.063295 16336 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:24.088898 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.005348 16348 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:24.093586 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 95850.6us
I20260709 13:19:24.096345 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:24.074189 16521 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.290s	user 0.001s	sys 0.001s
I20260709 13:19:24.073737 15786 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:24.098690 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.098860 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.105100 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:24.086094 16534 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:24.120999 16363 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:19:24.123018 16524 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:19:24.109503 16528 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:24.124285 16535 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:24.125099 16535 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:24.125658 16535 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:24.127575 16545 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:19:24.140038 16541 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:19:24.152532 16329 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.002s
W20260709 13:19:24.153432 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.153111 16329 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.075268 16121 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:24.173316 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56522.4us
I20260709 13:19:24.105193 16364 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:24.172874 16336 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:24.068660 16240 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.100804 16348 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:24.185515 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 181432us
I20260709 13:19:24.189364 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.219422 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:24.224575 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 134528us
I20260709 13:19:24.226953 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:24.230602 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 125548us
I20260709 13:19:24.232555 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:24.188365 16536 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:24.239650 10170 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:24.240489 16542 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
W20260709 13:19:24.225744 16409 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.571s	user 0.014s	sys 0.000s
I20260709 13:19:24.243882 16536 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.244951 16336 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.238242 16532 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:24.272755 16532 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.273844 16532 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.238309 16545 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.209725 16458 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:19:24.275561 16545 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:24.280205 16490 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.221s	user 0.012s	sys 0.005s
I20260709 13:19:24.280987 16490 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.282610 10129 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:24.284679 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:24.287017 10894 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:24.288920 16263 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:24.289568 16263 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:24.292685 16322 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:24.293249 16322 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.293588 16121 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:24.293890 16087 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.884s	user 0.016s	sys 0.000s
I20260709 13:19:24.259703 16548 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:19:24.294348 16087 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.296195 16372 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:24.296794 16372 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.296829 16398 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.297585 16536 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.250088 16348 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.251726 16388 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.173369 16515 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.221393 16547 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:24.257673 16534 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.231329 16461 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:24.308446 16461 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.309177 16388 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.241987 16329 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.310114 10898 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:24.259871 16543 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:24.259622 16432 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:19:24.217857 15661 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:24.250209 16538 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:24.243939 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.270318 16517 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:24.272109 16330 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.848s	user 0.016s	sys 0.000s
W20260709 13:19:24.240187 16488 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.658s	user 0.015s	sys 0.000s
I20260709 13:19:24.284782 16490 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.306504 16551 raft_consensus.cc:2764] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:19:24.328711 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86225.9us
W20260709 13:19:24.329541 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97087.5us
W20260709 13:19:24.274557 16167 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:19:24.361045 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:24.304845 16513 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:24.338078 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.374755 10196 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:24.179618 16381 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:24.340636 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.367887 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 141007us
I20260709 13:19:24.294437 16322 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.370658 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.372663 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:24.388990 10201 raft_consensus.cc:606] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.376413 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.389724 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.389948 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:24.376879 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.298276 16372 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:24.353696 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 104816us
I20260709 13:19:24.416076 10020 leader_election.cc:400] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:24.416471 10020 leader_election.cc:403] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:24.313777 16543 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.304140 16553 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:24.370599 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.417634 10127 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:24.354156 16536 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:24.328578 16499 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.905s	user 0.015s	sys 0.000s
I20260709 13:19:24.390496 10050 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:24.378144 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3539) took 1528 ms. Trace:
I20260709 13:19:24.385673 15786 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.349212 16538 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.420890 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.392095 16554 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.429064 10197 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:24.400133 16508 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.837s	user 0.005s	sys 0.000s
I20260709 13:19:24.400818 16545 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:24.345392 16510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.011s	sys 0.000s
I20260709 13:19:24.349627 16488 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.407786 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.439808 10898 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:24.391404 10049 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:24.391404 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.441762 10056 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:24.410199 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.444168 10130 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:24.348986 16547 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:24.419971 10129 rpcz_store.cc:276] 0709 13:19:22.836273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.850006 (+ 13733us) service_pool.cc:224] Handling call
0709 13:19:24.364163 (+1514157us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540015a754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:24.364326 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114640256}
I20260709 13:19:24.377890 16458 raft_consensus.cc:2764] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:24.391381 16527 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:19:24.422129 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:24.296829 16504 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:24.426172 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.427595 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.427479 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:24.458881 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.426492 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.337411 15661 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.445562 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.465209 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:24.349439 16330 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.428963 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:24.471220 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49183.2us
I20260709 13:19:24.471086 10047 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:24.420696 16543 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:24.451259 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.451603 10020 leader_election.cc:400] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:24.474786 10126 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:24.450942 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.415529 16490 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:24.435745 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:24.484581 15661 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:24.452414 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74170.3us
I20260709 13:19:24.450409 16514 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.420881 16553 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:24.459546 10054 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:24.452059 16372 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.373594 16167 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.383193 16522 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:19:24.386297 16363 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:19:24.422824 16398 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:24.450245 16510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.000s
I20260709 13:19:24.309546 16461 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.474844 10020 leader_election.cc:403] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:24.480758 16488 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:24.481093 16504 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.481487 10049 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:19:24.482293 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:24.381752 16513 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.532264 16558 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:24.483410 10128 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:24.532971 16513 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.435534 15786 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:24.535351 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 63141.5us
I20260709 13:19:24.484002 16490 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:24.537025 16490 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.537701 15786 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:24.538340 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:24.429658 16511 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:24.539484 16511 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.539310 16538 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.541941 16511 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.449258 16348 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:24.452134 16554 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:24.543174 16554 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.467963 16322 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.501575 16330 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.503631 16557 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:24.504252 16560 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:24.455440 16121 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:19:24.505033 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:24.550997 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46053.6us
I20260709 13:19:24.496670 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.506995 16522 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:24.492558 16553 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:24.553285 10197 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:24.522635 16379 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.002s
I20260709 13:19:24.515893 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:24.518268 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:24.521130 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 69694us
I20260709 13:19:24.425710 16499 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.528443 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:24.569032 10124 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.528539 10049 pending_rounds.cc:85] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 12
I20260709 13:19:24.496447 16551 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:24.572281 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54115.1us
I20260709 13:19:24.567596 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:24.573599 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.552127 10129 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:24.515082 10196 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:24.512887 16087 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.569967 10049 pending_rounds.cc:107] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:19:24.577770 10049 raft_consensus.cc:2889] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:24.566119 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:24.496208 16514 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.574978 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:24.582823 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.583305 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.581315 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:24.548547 16488 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:24.585757 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99754.6us
I20260709 13:19:24.593947 10197 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:24.585529 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:24.566484 16511 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:24.553506 16522 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:24.600863 10205 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:24.605206 10125 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:24.608258 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:24.610881 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.581936 16124 raft_consensus.cc:2764] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:24.584411 16548 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:24.591524 16349 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:24.616701 10205 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:24.624362 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.632746 16499 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.576768 16379 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.618736 16363 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:19:24.644524 16167 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.567466 16557 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.598908 16558 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:24.646313 16349 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:24.648957 16535 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.004s
I20260709 13:19:24.586658 16547 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:19:24.657321 10197 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.660768 10124 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:24.664963 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:24.656929 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.676671 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.571981 16348 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:24.605854 16511 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.657384 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:24.608970 16330 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:24.712064 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:24.708340 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:24.713867 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.607577 16409 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:19:24.715031 10201 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:19:24.649668 16434 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.000s
I20260709 13:19:24.717041 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:24.719507 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:24.723066 10200 raft_consensus.cc:2368] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:24.652195 16526 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:24.727541 10049 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:24.733222 10047 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.714586 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:24.734403 10048 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:24.730245 10203 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:24.731312 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74044.6us
I20260709 13:19:24.743032 10124 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:24.746063 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.718814 16349 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.757965 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.728152 16511 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:24.759063 10131 raft_consensus.cc:2393] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:24.710646 16533 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:19:24.726732 16330 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.778865 10047 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:24.787981 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.788326 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.789297 10131 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:24.726277 16535 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.810503 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.817517 10124 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:24.758059 16548 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:24.804280 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.784380 16513 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:24.791857 16330 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:24.833524 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.824776 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:24.836581 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.737043 16434 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.836581 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:24.808346 16473 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.889277 16473 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:24.892071 16473 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:24.856156 16263 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.000s
I20260709 13:19:24.893652 16263 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.894302 16557 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:24.815965 16258 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.861972 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.895555 16258 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.896119 10897 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:24.899318 16510 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:24.900509 16499 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:24.900959 16499 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:24.860373 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.856300 10894 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:24.856642 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.857425 10900 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:24.873950 16567 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.859633 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.903884 10895 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:24.874622 16566 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:24.860018 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:24.853449 16489 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:24.860038 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:24.877823 10019 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.508s (SENT)
I20260709 13:19:24.914158 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:24.877969 16330 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:24.878506 16565 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:19:24.898736 16258 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.878907 10197 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:24.864257 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:24.815929 16538 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:24.869768 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 252879us
I20260709 13:19:24.896554 16553 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.851017 16577 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:24.888300 16562 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:19:24.809245 16522 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.892794 16548 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:24.855145 10898 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:24.839260 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.853041 10124 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:24.906802 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:24.942952 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.966334 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 59673.2us
I20260709 13:19:24.837930 16540 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:24.952695 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74925.8us
I20260709 13:19:24.886901 16535 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:24.952602 10896 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:24.955840 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 45311.3us
I20260709 13:19:24.966571 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:24.952003 16541 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:19:24.986301 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.984506 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:24.970479 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 368241us
I20260709 13:19:25.004153 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.005364 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:25.006063 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.009636 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:25.010454 10200 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:25.017117 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.018306 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:25.018525 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:24.948017 16488 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:19:24.983745 16540 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.901921 16499 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.024677 10129 raft_consensus.cc:2368] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:25.026019 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.023284 10130 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:25.028119 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.080994 10900 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:25.024200 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:24.989817 16458 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:24.990451 16258 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:25.083405 16258 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:25.084084 16258 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.091503 16458 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:24.947929 16560 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:19:25.026873 16504 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:25.058135 16543 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.551s	user 0.013s	sys 0.000s
W20260709 13:19:25.023546 16372 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 13:19:25.058869 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:25.050153 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:25.057854 10196 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:25.051666 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:24.962356 16489 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:25.064687 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.061082 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.132709 10131 raft_consensus.cc:2368] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:25.032243 16522 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.057957 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.032202 16540 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.952153 16567 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.104781 16532 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:25.105110 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.150838 16489 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.155718 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.150982 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.162114 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.151386 16532 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.164489 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:25.165251 16532 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:25.165422 16513 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:25.165719 10201 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:25.108944 16528 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:25.115264 16543 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.116994 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:25.119946 16263 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:24.987551 16577 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.120477 16329 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.122231 16588 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.123591 16536 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:25.124993 16573 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:19:24.986605 16538 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.152837 16567 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.103420 16499 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:25.149667 16330 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:25.102371 16546 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:19:25.152292 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.171207 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:25.172719 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:25.152581 16490 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:19:25.131129 10199 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:25.145869 16504 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:25.209744 10128 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:25.180576 16528 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:25.224637 10201 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:25.226132 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.233173 10131 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:25.146149 16534 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:19:25.234776 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 241826us
W20260709 13:19:25.229624 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.102664 16458 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:25.239717 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.250058 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 260882us
W20260709 13:19:25.254801 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.257905 10199 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 12. Current term is 14. Ops: [12.7-12.7]
I20260709 13:19:25.257735 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.201326 16329 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.260624 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:25.260650 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.265039 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.265250 10053 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:25.265769 10056 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.267458 10201 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:19:25.269683 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.270735 10053 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:25.273327 10053 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:25.277199 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.260278 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:25.262776 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.279223 10047 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:25.280789 10050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.020s
W20260709 13:19:25.281944 10050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.004s	sys 0.020s
W20260709 13:19:25.261946 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 159091us
I20260709 13:19:25.283710 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.284142 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.284631 10131 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:25.284795 10048 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20. Re-sending same reply.
I20260709 13:19:25.249171 16532 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.276521 16330 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:25.291335 16330 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.291349 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:25.292111 16330 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:25.259539 16583 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.295617 16583 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.249905 16511 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.300518 16511 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:25.301837 16349 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.009s
W20260709 13:19:25.276512 16554 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.729s	user 0.014s	sys 0.004s
I20260709 13:19:25.303565 16554 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.302500 16330 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.306465 16554 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.302424 16349 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.324558 10047 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:25.326036 10056 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:25.329844 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.330996 10131 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:25.333595 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.260216 16432 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.327366 10203 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:25.315867 10200 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:25.276945 16357 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.337494 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.367641 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.368572 10131 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:25.370035 10197 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:25.351522 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60677.1us
W20260709 13:19:25.337055 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 78799.2us
I20260709 13:19:25.378201 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:25.238554 16490 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.355914 10202 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:25.336115 10124 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:25.360400 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:25.382508 10891 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:25.325948 16330 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.328677 16554 raft_consensus.cc:2764] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
W20260709 13:19:25.392947 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.401230 16559 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:25.599607 16597 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:25.600320 16593 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:19:25.599541 16587 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.317s	user 0.000s	sys 0.009s
I20260709 13:19:25.601775 16593 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:25.602279 16593 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:25.602659 16258 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.012s
I20260709 13:19:25.401541 16580 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
W20260709 13:19:25.403177 16584 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:19:25.603533 16593 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:25.404770 16589 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:19:25.366379 16553 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:25.485569 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.606560 16329 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.365671 16432 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:25.607285 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 329126us
I20260709 13:19:25.608455 16601 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.609124 16601 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.609874 16432 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:25.370563 16528 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.611084 16528 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.603899 16258 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.351796 16532 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.360530 16540 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.370285 16357 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.617364 10198 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:25.615370 16606 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:25.618321 10122 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:25.593189 16567 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:25.619141 16606 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.622445 16357 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.369400 16578 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:25.369792 16588 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:19:25.382303 16504 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.619855 16532 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:25.623436 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:25.667017 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.624295 16606 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:25.628791 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.605283 16593 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.620113 10891 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:25.631580 10204 raft_consensus.cc:2368] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:25.626856 16258 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.632499 16540 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:25.593801 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 209116us
I20260709 13:19:25.682873 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:25.593986 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.614852 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:25.653681 10129 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:25.686488 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.636118 10049 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
W20260709 13:19:25.634442 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.709s (SENT) [suppressed 40 similar messages]
I20260709 13:19:25.660259 10126 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:25.663861 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.692075 10129 raft_consensus.cc:2376] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20. Re-sending same reply.
I20260709 13:19:25.668653 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:25.673044 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:25.675009 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:25.694480 16540 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:25.694686 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:25.695195 16540 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:25.695456 10899 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.671643 10900 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:25.694032 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.697731 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.698534 10900 raft_consensus.cc:2410] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:19:25.698401 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
W20260709 13:19:25.699059 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84286.1us
I20260709 13:19:25.700531 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:25.691339 16499 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:25.706519 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 112635us
I20260709 13:19:25.707530 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.709792 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.714352 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:25.727289 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.717798 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.691289 16514 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.727041 16580 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:25.762008 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:25.761067 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.764966 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.766120 10095 leader_election.cc:400] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:25.767165 10048 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:25.766062 10131 raft_consensus.cc:2410] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:19:25.768554 10891 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:25.768149 10095 leader_election.cc:403] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:19:25.771811 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.772710 10129 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:25.772768 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.773902 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:25.774281 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:25.717331 16522 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:25.714033 16372 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.775422 16522 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.776017 16522 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.777659 16540 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.779515 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.782390 16379 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:25.787613 16594 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.790326 10129 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:25.691612 16543 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:25.790326 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.789803 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:25.800374 10019 leader_election.cc:400] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15.
I20260709 13:19:25.801440 16553 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:25.808751 16553 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:25.771829 16580 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.788977 16593 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.706053 16532 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.813364 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.760576 16567 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:25.816740 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3006) took 1447 ms. Trace:
I20260709 13:19:25.769975 16538 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.819056 10019 leader_election.cc:400] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16.
I20260709 13:19:25.834295 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.790709 16489 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:25.836424 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:25.715408 16167 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 13:19:25.847981 16379 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.824445 10125 rpcz_store.cc:276] 0709 13:19:24.369537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.370372 (+   835us) service_pool.cc:224] Handling call
0709 13:19:25.816484 (+1446112us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001c7d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:25.816653 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491216768}
W20260709 13:19:25.816481 16548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.014s	sys 0.022s
W20260709 13:19:25.857079 16548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.015s	sys 0.022s
I20260709 13:19:25.727531 16601 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:25.846570 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.834617 16349 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:25.850127 16578 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:25.856766 16379 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.840485 16538 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:25.833655 16580 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.825380 16543 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:25.844861 16588 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:25.821692 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:25.831526 16594 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:25.871155 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 236704us
W20260709 13:19:25.878098 10019 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.904s (SENT)
I20260709 13:19:25.879031 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:25.870324 16322 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.320s	user 0.023s	sys 0.006s
I20260709 13:19:25.864706 16532 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:25.864359 16578 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:25.886814 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.887082 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.889967 10053 raft_consensus.cc:2368] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:25.890937 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.891899 10053 raft_consensus.cc:2368] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:25.836126 16567 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:25.895612 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.896847 10053 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:19:25.900156 10055 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.899879 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.850811 16553 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.902164 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:25.893031 16596 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.222s	user 0.001s	sys 0.001s
I20260709 13:19:25.903319 10048 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:19:25.910211 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.913077 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.903069 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.898307 16348 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:25.883507 16490 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.894837 16578 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:25.954125 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.915073 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.950552 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:25.957598 10053 raft_consensus.cc:2368] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:25.886121 16577 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:25.885453 16543 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:25.909492 16604 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:19:25.959543 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.961812 10050 raft_consensus.cc:2393] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:25.963259 10053 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:25.953379 16548 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.950136 16458 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:25.987838 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:25.989522 16124 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:25.990481 16124 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:25.990907 16527 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:25.991518 16124 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.986950 16557 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:25.993944 16557 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:25.994794 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:25.994985 16557 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:25.996459 16557 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:25.996958 16349 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:25.997745 16434 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:25.989398 10123 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:26.002638 10055 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:25.956306 16348 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:25.955523 16322 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.005617 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.003604 16538 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:26.003854 16596 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:25.968868 16510 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.016585 16594 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.017928 16510 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.019196 16488 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:26.019610 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.016181 16348 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.021659 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3517) took 1001 ms. Trace:
I20260709 13:19:26.022063 10204 rpcz_store.cc:276] 0709 13:19:25.020788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.023916 (+  3128us) service_pool.cc:224] Handling call
0709 13:19:26.021416 (+997500us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.021580 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231391616}
I20260709 13:19:26.022778 16510 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.019761 16488 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:26.026322 16372 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:26.015225 16322 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.030113 16322 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.027774 16583 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.023917 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3564) took 1264 ms. Trace:
I20260709 13:19:26.032874 10126 rpcz_store.cc:276] 0709 13:19:24.759596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.155316 (+395720us) service_pool.cc:224] Handling call
0709 13:19:26.023695 (+868379us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54001c8254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.023850 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158022912}
I20260709 13:19:26.034360 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.036132 10126 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.036278 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.037225 10054 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:26.026470 16488 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.038400 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.038823 16488 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.039582 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.020484 10127 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:26.040324 10899 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:19:26.040481 16607 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.243s	user 0.002s	sys 0.000s
I20260709 13:19:26.013360 16538 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.043562 16510 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.044164 16510 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.045429 16538 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:26.111575 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.117951 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:26.120139 10197 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:26.047435 16510 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.042788 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.054479 16363 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.057993 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:26.125304 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67459.9us
I20260709 13:19:26.126461 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.023739 16379 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:26.061096 16567 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.128464 10865 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:26.065116 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 826) took 1048 ms. Trace:
I20260709 13:19:26.064448 16522 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.064471 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3515) took 1046 ms. Trace:
I20260709 13:19:26.065793 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.067008 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:25.999926 16548 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.064574 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.000571 16458 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:26.020341 16608 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:26.073211 16519 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:26.076705 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3025) took 1296 ms. Trace:
I20260709 13:19:26.120285 16372 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.080399 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 994) took 1217 ms. Trace:
I20260709 13:19:26.080587 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:26.052311 16580 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.083246 16434 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.088587 10054 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.089785 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:26.161870 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 72191us
I20260709 13:19:26.162288 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.163687 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:26.093453 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3561) took 1334 ms. Trace:
I20260709 13:19:26.094853 16432 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.100497 16600 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:26.101378 10203 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b09e10904af344988a1a11ae9e576799"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:26.117749 16552 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:19:26.167703 10203 raft_consensus.cc:606] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.134861 16473 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:26.102803 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.150663 10897 rpcz_store.cc:276] 0709 13:19:24.863163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.896962 (+ 33799us) service_pool.cc:224] Handling call
0709 13:19:24.912356 (+ 15394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.912716 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:25.189922 (+277206us) log.cc:844] Serialized 14 byte log entry
0709 13:19:25.201640 (+ 11718us) log.cc:844] Serialized 14 byte log entry
0709 13:19:25.201982 (+   342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:25.202044 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:25.202133 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.202239 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.784572 (+582333us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b1800413920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1
0709 13:19:25.786280 (+  1708us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b1800413920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:26.071138 (+284858us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b1800413920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:26.080146 (+  9008us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b540063ef50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.080315 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13193,"prepare.queue_time_us":862680,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":36345,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":7730,"spinlock_wait_cycles":62163200,"thread_start_us":171803,"threads_started":1}
I20260709 13:19:26.107564 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.108697 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3014) took 1648 ms. Trace:
I20260709 13:19:26.110262 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:26.124222 10197 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.127817 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.131639 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.135237 10198 rpcz_store.cc:276] 0709 13:19:25.017182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.017986 (+   804us) service_pool.cc:224] Handling call
0709 13:19:26.064923 (+1046937us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400228b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.065044 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1375615232}
W20260709 13:19:26.176970 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.103260 10126 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:26.138912 10202 rpcz_store.cc:276] 0709 13:19:25.017605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.018158 (+   553us) service_pool.cc:224] Handling call
0709 13:19:25.355487 (+337329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.381885 (+ 26398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.064229 (+682344us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b540035a7d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.064397 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774830976}
I20260709 13:19:26.172736 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.079177 16489 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.118347 16538 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.153637 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73153.1us
I20260709 13:19:26.143778 10131 rpcz_store.cc:276] 0709 13:19:24.780118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.776527 (+996409us) service_pool.cc:224] Handling call
0709 13:19:25.776905 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.964012 (+187107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.964048 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:25.964093 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:25.964128 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.076289 (+112161us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002ef254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:26.076367 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.076606 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353001728}
W20260709 13:19:26.136487 16528 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.525s	user 0.014s	sys 0.000s
I20260709 13:19:26.157578 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.150053 16602 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:26.103847 10047 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:26.170303 16322 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.221505 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.171607 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.172966 10130 rpcz_store.cc:276] 0709 13:19:24.460824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.463196 (+  2372us) service_pool.cc:224] Handling call
0709 13:19:24.463538 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.080290 (+616752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.108407 (+1028117us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400354154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:26.108620 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1283857792}
W20260709 13:19:26.185420 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82687.1us
I20260709 13:19:26.186657 10052 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:26.188273 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.134475 16363 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:26.199186 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89028.7us
I20260709 13:19:26.143524 16519 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:26.189237 16489 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:26.262012 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.269364 16461 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.271941 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.272823 10129 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.274124 16528 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:19:26.276192 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3031) took 1829 ms (client timeout 2068 ms). Trace:
W20260709 13:19:26.276696 10055 rpcz_store.cc:269] 0709 13:19:24.446374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.481802 (+ 35428us) service_pool.cc:224] Handling call
0709 13:19:26.275885 (+1794083us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540037bc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.276069 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3153694464}
I20260709 13:19:26.278126 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.281894 10054 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:26.282409 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.283320 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:26.219853 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.168123 16379 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:26.118003 16121 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.192690 16610 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.218s	user 0.000s	sys 0.005s
I20260709 13:19:26.225976 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.144294 16167 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 13:19:26.226997 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 351850us
I20260709 13:19:26.171274 16580 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:26.230443 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.164889 10124 rpcz_store.cc:276] 0709 13:19:24.758737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.005378 (+246641us) service_pool.cc:224] Handling call
0709 13:19:26.093145 (+1087767us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.093355 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700899584}
I20260709 13:19:26.229001 16329 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:26.297328 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:26.302205 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76345.4us
I20260709 13:19:26.304486 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:26.192071 16543 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:26.234411 16519 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.235646 16458 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.218179 16538 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.236954 16548 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.237285 16363 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:26.200517 16600 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.186220 16432 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.234411 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.331867 10131 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:26.250581 16488 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:26.283686 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.198391 16527 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:26.216881 10199 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:26.161284 16372 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.190814 16613 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:26.172705 16434 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.221628 16510 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:26.291726 10054 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.293743 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.383235 10052 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.292023 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:26.384615 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92757.2us
W20260709 13:19:26.385141 10019 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.725s (SENT)
I20260709 13:19:26.306124 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:26.386183 10019 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.109s (SENT)
I20260709 13:19:26.314007 10020 leader_election.cc:400] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
I20260709 13:19:26.386785 10020 leader_election.cc:403] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:26.289300 16602 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:26.325305 10129 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:26.391366 10205 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:26.287732 16473 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:26.328246 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.228003 16322 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.330897 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.295396 16552 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.308208 16557 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:26.337494 10204 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:19:26.337965 10053 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:26.309026 16580 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:26.315459 16558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.559s	user 0.020s	sys 0.021s
W20260709 13:19:26.357511 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.312069 16490 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.315721 16398 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:26.386616 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.292398 16379 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.453948 16379 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.459654 16240 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.460340 16240 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.462625 16240 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:26.386641 10124 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:19:26.387890 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74061.3us
I20260709 13:19:26.396435 10054 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:26.397615 10201 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:26.399272 10048 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.469244 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.469930 10054 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:26.375367 16593 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.001s
I20260709 13:19:26.405608 10096 leader_election.cc:400] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:26.407223 16510 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:26.409364 16322 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.409844 10196 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:26.337165 16527 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:26.376427 16588 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.002s
I20260709 13:19:26.414862 10055 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.308131 16121 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.320649 16577 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 13:19:26.373616 16519 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.428326 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.429162 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.432267 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.433545 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.402045 16258 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:26.435403 10128 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3571) took 1674 ms (client timeout 1725 ms). Trace:
W20260709 13:19:26.512267 10128 rpcz_store.cc:269] 0709 13:19:24.761299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.834532 (+ 73233us) service_pool.cc:224] Handling call
0709 13:19:26.435145 (+1600613us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540014bc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.435321 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687711488}
I20260709 13:19:26.437207 16594 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:26.470847 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.518179 10054 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:26.519284 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.520159 10054 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:26.521037 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3023) took 1779 ms (client timeout 2259 ms). Trace:
W20260709 13:19:26.481343 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.487294 10096 leader_election.cc:403] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:26.491935 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:26.519251 10124 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:26.494062 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 269677us
W20260709 13:19:26.427826 16558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.668s	user 0.021s	sys 0.021s
I20260709 13:19:26.496212 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.505308 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.526995 10047 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:26.473111 16548 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 13. Current term is 14.
W20260709 13:19:26.506114 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.511031 10199 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 10.
I20260709 13:19:26.513753 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.532429 10053 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:26.514657 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:26.505276 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119115us
I20260709 13:19:26.472669 16398 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:26.524125 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.524405 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.422600 16580 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.528193 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.494679 16557 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:26.521631 10123 rpcz_store.cc:269] 0709 13:19:24.741709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.745695 (+  3986us) service_pool.cc:224] Handling call
0709 13:19:26.520827 (+1775132us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002efc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.520961 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2856722688}
I20260709 13:19:26.541770 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.521682 10052 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:26.526161 10201 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:26.529974 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.543776 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.530165 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:26.551806 10048 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.535629 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:26.536005 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.530262 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.554868 10050 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:19:26.537335 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.555877 10055 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:26.536583 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.539263 10128 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:26.561517 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.511564 16588 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:26.528383 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:26.539937 10130 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:26.556041 10047 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 13:19:26.556840 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.493088 16593 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:26.553854 10893 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:26.509938 16527 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:26.535871 16540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.024s	sys 0.035s
I20260709 13:19:26.473294 16490 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.529712 16615 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.422875 16552 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.577137 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.577648 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.591432 10130 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:19:26.579305 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3570) took 1818 ms (client timeout 2165 ms). Trace:
I20260709 13:19:26.591020 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:26.530426 16329 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.517724 16594 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:26.582008 10050 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:19:26.584086 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 187491us
I20260709 13:19:26.609938 10096 leader_election.cc:400] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:26.610236 10050 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:26.610488 10096 leader_election.cc:403] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:26.573412 10047 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:26.576941 16540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.025s	sys 0.035s
I20260709 13:19:26.541499 16398 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:26.525903 16558 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:26.529830 16258 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:26.593394 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.626820 10128 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
W20260709 13:19:26.629559 16620 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:19:26.629890 16617 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:19:26.630620 16552 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.632462 16434 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.617993 16626 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:26.634835 16626 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.635665 16625 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:26.636379 16625 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:26.636795 16625 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.638238 16626 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:26.639775 16621 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:19:26.640942 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:26.641930 16616 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:19:26.645310 16526 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:26.646083 16526 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:26.646541 16121 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.647240 16526 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.648499 16623 raft_consensus.cc:993] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:26.648780 16398 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.650187 10131 raft_consensus.cc:2410] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:26.650772 16372 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.651432 16628 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:26.652143 16628 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.652375 16594 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.653751 16594 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.654724 16628 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:26.544801 16522 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.606408 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:26.668411 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58763.4us
I20260709 13:19:26.506392 16543 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:19:26.613350 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.561909 16458 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 14. Current term is 15.
I20260709 13:19:26.676331 16432 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:26.626065 16240 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.600903 16578 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.631s	user 0.014s	sys 0.000s
I20260709 13:19:26.677117 16240 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.625056 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.678272 10123 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:26.679929 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:26.680931 10123 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:26.681995 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.682600 10123 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:26.537545 16167 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:26.592103 10122 rpcz_store.cc:269] 0709 13:19:24.761055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.161765 (+400710us) service_pool.cc:224] Handling call
0709 13:19:26.579046 (+1417281us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b540027d3d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:26.579227 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047520000}
I20260709 13:19:26.597321 10048 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:26.628340 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:26.655284 16510 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.663582 16398 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.634872 16490 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.664701 10051 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:19:26.699271 10051 pending_rounds.cc:85] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:19:26.699651 10051 pending_rounds.cc:107] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:19:26.666574 16258 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.699934 10051 raft_consensus.cc:2889] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:26.602952 16602 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.665643 10049 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:26.702142 10122 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:26.702283 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.668988 10130 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:26.581188 16473 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.674698 16552 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:26.675606 16558 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.707532 10130 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.676056 16489 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.614838 16329 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.715551 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.717216 16489 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.715551 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.625547 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:26.720969 16240 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.722259 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.722713 10204 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:26.649351 16623 raft_consensus.cc:1081] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:26.620712 10050 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:26.729127 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 242759us
I20260709 13:19:26.607947 16615 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:26.729398 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 201097us
I20260709 13:19:26.669863 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:26.704883 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.708370 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 763) took 1414 ms. Trace:
I20260709 13:19:26.710881 16543 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:26.707976 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3027) took 1831 ms. Trace:
I20260709 13:19:26.731526 10122 rpcz_store.cc:276] 0709 13:19:24.876538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.698193 (+1821655us) service_pool.cc:224] Handling call
0709 13:19:26.698578 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.702727 (+  4149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.707907 (+  5180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.732824 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:26.733599 10892 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:26.734772 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.730855 10130 rpcz_store.cc:276] 0709 13:19:25.294326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.706900 (+1412574us) service_pool.cc:224] Handling call
0709 13:19:26.707324 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.708095 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.708314 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.741088 10200 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:26.713927 16379 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.707767 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.677459 16538 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:26.721314 10201 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:26.724560 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.726348 10129 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:26.664304 16600 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:26.685489 16504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.000s	sys 0.031s
I20260709 13:19:26.691537 10056 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.733731 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 765) took 1438 ms. Trace:
I20260709 13:19:26.760952 10122 rpcz_store.cc:276] 0709 13:19:25.294861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.732484 (+1437623us) service_pool.cc:224] Handling call
0709 13:19:26.732915 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.733204 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.733248 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.733336 (+    88us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:26.733391 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.733473 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.733639 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.761744 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.762578 10122 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.763418 10203 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:19:26.764034 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 239684us
I20260709 13:19:26.765615 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.766382 10200 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:26.767575 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 767) took 1435 ms. Trace:
I20260709 13:19:26.768105 10122 rpcz_store.cc:276] 0709 13:19:25.332536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.762063 (+1429527us) service_pool.cc:224] Handling call
0709 13:19:26.762385 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.767286 (+  4901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.767514 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.770295 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 768) took 1437 ms. Trace:
I20260709 13:19:26.770936 10122 rpcz_store.cc:276] 0709 13:19:25.332960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.769343 (+1436383us) service_pool.cc:224] Handling call
0709 13:19:26.769718 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.769920 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.769945 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.769989 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:26.770023 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.770077 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.770227 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.701550 16121 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.752424 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.772509 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:26.752905 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.773115 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.773774 10048 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:26.773860 10199 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:26.774140 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:26.756811 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 766) took 1461 ms. Trace:
I20260709 13:19:26.718032 16329 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:26.757891 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:26.776124 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:26.775753 10130 rpcz_store.cc:276] 0709 13:19:25.310961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.755889 (+1444928us) service_pool.cc:224] Handling call
0709 13:19:26.756244 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.756429 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.756454 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:26.756500 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:26.756530 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.756584 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.756737 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:26.780929 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:26.782126 10130 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:26.782207 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:26.758600 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3026) took 1968 ms. Trace:
I20260709 13:19:26.783658 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:26.759724 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90002.8us
I20260709 13:19:26.784432 10049 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:26.785023 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:26.736733 16543 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:26.701534 16167 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.752432 16534 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:26.783100 10125 rpcz_store.cc:276] 0709 13:19:24.790278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.859681 (+1069403us) service_pool.cc:224] Handling call
0709 13:19:25.860120 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.883091 (+ 22971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:25.883183 (+    92us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:25.883787 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.306022 (+422235us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:26.306087 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.745464 (+439377us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b540015b154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:26.745557 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:26.745671 (+   114us) raft_consensus.cc:1685] finished
0709 13:19:26.745693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:26.758515 (+ 12822us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1147,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":9363,"spinlock_wait_cycles":51483520,"thread_start_us":400211,"threads_started":1,"wal-append.queue_time_us":502597}
I20260709 13:19:26.758224 10123 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:26.677784 16578 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:26.786891 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:26.762677 10891 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:26.787385 10197 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:26.782505 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:26.845566 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.903854 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.840718 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58600.7us
I20260709 13:19:26.788311 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 769) took 1434 ms. Trace:
W20260709 13:19:26.727520 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 193358us
I20260709 13:19:26.783404 16599 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:26.768586 16504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.000s	sys 0.031s
I20260709 13:19:26.749393 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:27.078133 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3046) took 1023 ms. Trace:
I20260709 13:19:27.032968 10898 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:27.026899 10124 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:27.106081 10047 rpcz_store.cc:276] 0709 13:19:26.054319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.612143 (+557824us) service_pool.cc:224] Handling call
0709 13:19:26.612653 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.614010 (+  1357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:26.614055 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:26.614662 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:19:26.881417 (+266755us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:26.881493 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.881610 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.077740 (+196130us) raft_consensus.cc:1685] finished
0709 13:19:27.077777 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.078026 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52390,"raft.queue_time_us":1826,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8612,"thread_start_us":214613,"threads_started":1,"wal-append.queue_time_us":382899}
I20260709 13:19:27.108624 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3587) took 1231 ms. Trace:
I20260709 13:19:27.109368 10124 rpcz_store.cc:276] 0709 13:19:25.876765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.025906 (+1149141us) service_pool.cc:224] Handling call
0709 13:19:27.026498 (+   592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.108164 (+ 81666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.108557 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:27.109154 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 359821us
I20260709 13:19:26.786123 16538 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:27.119949 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 337484us
W20260709 13:19:26.930608 16631 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:26.788722 16490 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:27.128552 10202 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.105623 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3037) took 1089 ms. Trace:
I20260709 13:19:27.130137 10123 rpcz_store.cc:276] 0709 13:19:26.016446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.681638 (+665192us) service_pool.cc:224] Handling call
0709 13:19:27.105510 (+423872us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.111761 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:27.112519 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.113705 10122 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:26.943018 10049 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:27.111670 10130 rpcz_store.cc:276] 0709 13:19:25.353695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.778396 (+1424701us) service_pool.cc:224] Handling call
0709 13:19:26.778909 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.787994 (+  9085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.788234 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.124096 10124 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:27.136775 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3590) took 1259 ms. Trace:
W20260709 13:19:27.090976 16624 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:19:27.126437 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:27.137329 10124 rpcz_store.cc:276] 0709 13:19:25.877656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.123319 (+1245663us) service_pool.cc:224] Handling call
0709 13:19:27.123723 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.136387 (+ 12664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.136696 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.139396 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.053851 10892 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:27.139475 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:26.876778 16626 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.143610 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.144713 10055 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:27.144765 10130 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:19:27.145715 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:27.134194 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 774) took 1437 ms. Trace:
I20260709 13:19:27.146286 10130 pending_rounds.cc:85] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:19:27.147213 10130 pending_rounds.cc:107] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:19:27.147675 10130 raft_consensus.cc:2889] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:27.148566 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:27.148770 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3594) took 1270 ms. Trace:
I20260709 13:19:27.145377 10196 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:27.149387 10130 rpcz_store.cc:276] 0709 13:19:25.878725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.135619 (+1256894us) service_pool.cc:224] Handling call
0709 13:19:27.144373 (+  8754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.148473 (+  4100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.148705 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.150419 10170 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:26.898226 16379 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.154484 10130 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:26.886379 16458 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.172066 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3034) took 1165 ms. Trace:
I20260709 13:19:27.172725 10130 rpcz_store.cc:276] 0709 13:19:26.005753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.151295 (+1145542us) service_pool.cc:224] Handling call
0709 13:19:27.154291 (+  2996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.171086 (+ 16795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.171501 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.174196 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.046360 15786 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.122218 16240 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:27.020020 16167 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.079665 16615 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.088446 16598 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:27.128072 16534 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:27.235641 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2503) took 1064 ms. Trace:
W20260709 13:19:27.199193 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 412351us
I20260709 13:19:27.240990 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:27.153954 16434 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.156049 10122 rpcz_store.cc:276] 0709 13:19:25.696712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.783138 (+1086426us) service_pool.cc:224] Handling call
0709 13:19:26.929525 (+146387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.133783 (+204258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.134108 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.240099 10056 rpcz_store.cc:276] 0709 13:19:26.170748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.640373 (+469625us) service_pool.cc:224] Handling call
0709 13:19:26.691296 (+ 50923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.760951 (+ 69655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.235423 (+474472us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b54001630d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:27.235574 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115580672}
I20260709 13:19:27.175407 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:26.802242 16628 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.368s
I20260709 13:19:26.878535 16329 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:26.780289 16536 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.348s
I20260709 13:19:27.176889 10130 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:27.160099 16522 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:27.146078 10865 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:27.230571 16625 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.003s
I20260709 13:19:26.785493 16552 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.377s
I20260709 13:19:27.116751 16538 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:27.017347 16580 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.191948 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.192430 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.191560 10196 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:27.207091 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2357) took 1089 ms. Trace:
I20260709 13:19:27.236552 16567 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:27.206394 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3591) took 1328 ms. Trace:
I20260709 13:19:27.340723 10205 rpcz_store.cc:276] 0709 13:19:26.117963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.118093 (+   130us) service_pool.cc:224] Handling call
0709 13:19:26.128376 (+ 10283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.398161 (+269785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.206794 (+808633us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:27.207027 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604069120}
I20260709 13:19:27.201244 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.346318 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.347146 10126 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:27.349417 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3035) took 1335 ms. Trace:
I20260709 13:19:27.349995 10126 rpcz_store.cc:276] 0709 13:19:26.013597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.348186 (+1334589us) service_pool.cc:224] Handling call
0709 13:19:27.348647 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.348914 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.348955 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:27.349022 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:27.349071 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.349186 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.349335 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.192991 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.187284 10198 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:19:27.127841 16519 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:27.135514 16600 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:27.067435 16543 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:26.785876 16121 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.019780 16527 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.159729 16409 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:27.158525 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.194521 16504 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 13:19:27.371622 10123 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:27.309151 16167 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:27.253074 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:27.381132 10122 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.180284 16623 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:27.258525 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 112550us
I20260709 13:19:27.387948 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:26.785830 16630 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.373s
W20260709 13:19:27.235064 16594 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.006s
I20260709 13:19:27.259868 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.389976 10130 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
W20260709 13:19:27.261092 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.201098 16578 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:27.200122 16510 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.311870 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 136533us
W20260709 13:19:27.422520 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 181645us
I20260709 13:19:27.379635 16526 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:27.212091  9954 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 14 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:27.165685 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:27.344460 10131 rpcz_store.cc:276] 0709 13:19:25.877923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.512453 (+634530us) service_pool.cc:224] Handling call
0709 13:19:27.206138 (+693685us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:27.206315 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241864448}
I20260709 13:19:27.239704 16598 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:27.377810 16580 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:27.260254 16348 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:27.160624 16634 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.149s
I20260709 13:19:27.263849 16434 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.190764 15786 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.361902 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.362123 16597 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:27.204211 16458 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.361905 16329 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:27.364611 16519 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:19:27.364895 16618 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.958s	user 0.007s	sys 0.008s
I20260709 13:19:27.365869 10054 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:27.463828 10200 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:27.255311 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106823us
I20260709 13:19:27.465081 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:27.440090 16473 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:27.338256 16538 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:27.259835 16588 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:27.382695 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 224311us
I20260709 13:19:27.488869 10170 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.490324 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.309809 16522 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.266989 16625 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:27.310045 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.424214 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:27.249588 16534 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.423334 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:27.429307 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.431514 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:27.502442 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79217.2us
I20260709 13:19:27.432590 10128 raft_consensus.cc:2410] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:26.785676 16461 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.359s
I20260709 13:19:27.360839 16567 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:27.440392 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 242498us
I20260709 13:19:27.446673 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.450842 10131 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:27.527725 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3044) took 1295 ms. Trace:
I20260709 13:19:27.528167 10131 rpcz_store.cc:276] 0709 13:19:26.232346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.431209 (+1198863us) service_pool.cc:224] Handling call
0709 13:19:27.450438 (+ 19229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.527406 (+ 76968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.527659 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.424299 16167 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:27.253935 16240 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.440234 16594 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.472920 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:27.537823 10202 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.445225 16600 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.445586 16363 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.493479 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.451776 16597 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:27.461489 16329 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:27.501307 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77190.1us
I20260709 13:19:27.514884 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3037) took 1463 ms. Trace:
I20260709 13:19:27.503326 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:27.520531 16526 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:27.509989 16458 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.508939 16580 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:27.502900 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71476.1us
I20260709 13:19:27.501875 16434 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.434993 16510 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.496279 16588 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.531441 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.503674 16379 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:27.499261 16599 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:27.558877 16526 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.540318 10201 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:27.547089 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53713.3us
I20260709 13:19:27.549618 10096 leader_election.cc:400] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16.
I20260709 13:19:27.599655 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:27.599917 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50510.3us
I20260709 13:19:27.601012 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:27.522527 16577 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:27.549055 10052 rpcz_store.cc:276] 0709 13:19:26.051711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.565274 (+513563us) service_pool.cc:224] Handling call
0709 13:19:26.601054 (+ 35780us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.601563 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:26.996217 (+394654us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.430268 (+434051us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.430782 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.430927 (+   145us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:27.430959 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.431032 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.514549 (+ 83517us) raft_consensus.cc:1685] finished
0709 13:19:27.514580 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.514787 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":455341,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":7764,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":16448,"thread_start_us":748151,"threads_started":2,"wal-append.queue_time_us":464558}
I20260709 13:19:27.516862 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.525203 16636 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:27.613895 10130 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:19:27.615164 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98370.3us
W20260709 13:19:27.535904 16643 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 13:19:27.557601 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54385us
I20260709 13:19:27.575743 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:27.533042 16654 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:27.537535 16594 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:27.492072 16598 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.557456 16329 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:27.557554 16461 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.535780 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:27.601845 16527 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.563611 16578 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:27.580889 16458 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.548516 16600 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.588762 16588 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.627672 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:27.673877 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46319.4us
W20260709 13:19:27.629277 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.634761 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59096.9us
I20260709 13:19:27.635692 10204 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:27.557137 16363 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.676735 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:27.602062 16526 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:27.597695 16599 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.641801 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3585) took 1765 ms. Trace:
I20260709 13:19:27.708650 10125 rpcz_store.cc:276] 0709 13:19:25.876161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.935628 (+1059467us) service_pool.cc:224] Handling call
0709 13:19:26.936134 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.936444 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:27.435584 (+499140us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.641383 (+205799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:27.641443 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:27.641501 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.641579 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.641729 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":574803,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":2906,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":5107,"thread_start_us":594052,"threads_started":2}
W20260709 13:19:27.644191 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 108524us
I20260709 13:19:27.643095 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:27.596170 10199 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:27.585197 16567 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:27.649021 16647 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:19:27.611961 16558 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:27.680902 16654 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:27.723345 16558 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:27.724005 16558 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.724946 16654 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:27.725634 16654 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:27.727496 16567 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.613337 16379 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.617506 16577 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.657475 10202 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:27.662860 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:27.625149 16630 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:27.723425 16646 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:19:27.638293 16597 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.651144 16519 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.682130 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:27.684324 16636 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:27.684617 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44662.5us
I20260709 13:19:27.685328 16510 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:27.767473 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85430.9us
I20260709 13:19:27.684388 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:27.685427 15661 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:27.687274 16641 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.233s	user 0.000s	sys 0.002s
I20260709 13:19:27.637509 16381 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:27.643019 16240 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:27.691975 16653 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:19:27.695178 10050 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:19:27.695943 16642 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:19:27.701546 16363 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.663751 16121 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.710819 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:27.789399 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3045) took 1493 ms. Trace:
W20260709 13:19:27.678951 16644 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
I20260709 13:19:27.796845 15661 raft_consensus.cc:2764] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:27.694940 16348 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:19:27.719166 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3047) took 1314 ms. Trace:
I20260709 13:19:27.797879 10125 rpcz_store.cc:276] 0709 13:19:26.404745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.710856 (+1306111us) service_pool.cc:224] Handling call
0709 13:19:27.711426 (+   570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.711727 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.711765 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:27.717529 (+  5764us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:27.717640 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.718770 (+  1130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.719023 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:27.719599 10122 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:27.799773 10127 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:27.726805 16654 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.580372 16580 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.736863 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.738731 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3043) took 1546 ms. Trace:
I20260709 13:19:27.714056 16599 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.744865 16600 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.685526 15786 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.748454 16577 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:27.749002 16655 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:19:27.750129 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3593) took 1871 ms. Trace:
I20260709 13:19:27.832681 10123 rpcz_store.cc:276] 0709 13:19:25.878467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.131221 (+1252754us) service_pool.cc:224] Handling call
0709 13:19:27.176406 (+ 45185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.372526 (+196120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.749749 (+377223us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b54002ed1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:27.750046 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448044928}
I20260709 13:19:27.682782 16533 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:27.718369 16594 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.753738 16409 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:27.617568 16167 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.834630 16594 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.674434 16458 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:27.677812 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78245.2us
I20260709 13:19:27.761968 10202 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:19:27.847854 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:27.848042 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.771275 10204 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:27.766366 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:27.850231 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83978.5us
I20260709 13:19:27.850523 10202 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:27.852586 16600 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.853189 16600 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.854197 16600 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.856861 16580 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.861060 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 831) took 1321 ms. Trace:
I20260709 13:19:27.861920 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:27.856971 10056 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:27.861600 10048 rpcz_store.cc:276] 0709 13:19:26.539551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.034223 (+494672us) service_pool.cc:224] Handling call
0709 13:19:27.034674 (+   451us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:27.038812 (+  4138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.038870 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:27.039852 (+   982us) log.cc:844] Serialized 52 byte log entry
0709 13:19:27.450526 (+410674us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:27.450564 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.450630 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.860813 (+410183us) raft_consensus.cc:1685] finished
0709 13:19:27.860836 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.860986 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3618,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9996,"thread_start_us":379882,"threads_started":1,"wal-append.queue_time_us":523261}
I20260709 13:19:27.812201 16663 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.774309 16552 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:27.795157 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84446.3us
I20260709 13:19:27.872072 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:27.872771 10056 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:27.873735 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:27.873466 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.807085 10125 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:27.810534 10130 rpcz_store.cc:276] 0709 13:19:26.192146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.392722 (+1200576us) service_pool.cc:224] Handling call
0709 13:19:27.393201 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.615082 (+221881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.738294 (+123212us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54002d1254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:27.738620 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506451712}
I20260709 13:19:27.756081 16526 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.787479 16630 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:27.920189 16630 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:27.774101 16625 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.601054 16522 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.867461 16548 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:19:27.922405 16349 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.927970 16634 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.927733 16349 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.928570 16634 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.930486 16349 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.834702 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.838799 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3028) took 1839 ms. Trace:
I20260709 13:19:27.736876 16379 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.802559 10122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:19:27.804111 16651 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:27.801565 16534 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.826880 16597 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.786432 10050 pending_rounds.cc:85] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
W20260709 13:19:27.874668 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.882567 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.901196 16329 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.925565 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:27.938851 10050 pending_rounds.cc:107] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:19:27.932533 10199 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:27.940069 10050 raft_consensus.cc:2889] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:27.935514 10122 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:27.884919 16526 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.932871 10124 rpcz_store.cc:276] 0709 13:19:25.999735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.141045 (+1141310us) service_pool.cc:224] Handling call
0709 13:19:27.142239 (+  1194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.177837 (+ 35598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.177869 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.204639 (+ 26770us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.223035 (+ 18396us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b1800216ac0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:27.422020 (+198985us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:27.422069 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.422151 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.838374 (+416223us) raft_consensus.cc:1685] finished
0709 13:19:27.838403 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.838710 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12680,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":25689,"spinlock_wait_cycles":36467968,"thread_start_us":198743,"threads_started":1,"wal-append.queue_time_us":331299}
I20260709 13:19:27.876312 16663 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.895376 16615 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:27.867162 16121 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:27.905092 10130 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:27.870617 16580 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:19:27.912575 16461 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:27.912935 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51131.2us
I20260709 13:19:27.968561 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3029) took 3512 ms. Trace:
I20260709 13:19:27.970615 10123 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:27.914273 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:27.865986 16636 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.931058 16634 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:27.989785 16519 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:27.918918 16409 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:27.919576 15661 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.001461 15661 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:27.792276 16473 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:19:27.943079 10122 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:27.892234 16626 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.722s	user 0.018s	sys 0.000s
I20260709 13:19:27.944473 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.009127 10204 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:28.010627 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:27.945326 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66629.7us
I20260709 13:19:27.921238 16630 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.965982 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:27.951992 16662 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:19:27.981913 16663 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:27.957628 16322 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.966929 16121 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:27.827280 16601 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:19:27.969071 10051 rpcz_store.cc:276] 0709 13:19:24.444098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.485072 (+ 40974us) service_pool.cc:224] Handling call
0709 13:19:24.485434 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:26.700475 (+2215041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.956506 (+1256031us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54001b8854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:27.956824 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4993999616}
I20260709 13:19:27.992077 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.951686 16534 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.994844 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:27.995661 16533 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.997272 16558 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:27.998993 10129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3589) took 2121 ms (client timeout 1980 ms). Trace:
I20260709 13:19:27.999524 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:27.908720 16588 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.028650 10048 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:28.061112 10123 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:28.052136 10199 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:19:28.060588 10129 rpcz_store.cc:269] 0709 13:19:25.877375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.403465 (+526090us) service_pool.cc:224] Handling call
0709 13:19:27.998733 (+1595268us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540014ad54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:27.998900 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327614208}
I20260709 13:19:28.038211 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:28.008366 16490 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.849s	user 0.015s	sys 0.000s
I20260709 13:19:28.035655 10122 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:27.953425 16363 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:19:28.054256 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.030928 10204 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:28.052206 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.068894 10204 raft_consensus.cc:606] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.069381 10204 raft_consensus.cc:614] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:28.024549 16548 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.037510 16626 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:28.070328 10125 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:28.041497 16461 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.073308 15661 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.073411 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.048264 16121 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.072431 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.075084 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:28.079613 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.078230 10053 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:28.079977 10051 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.061179 16409 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:28.060532 16663 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.015697 16664 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:19:28.071216 16490 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:28.075013 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:28.084792 10122 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.085213 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.095609 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.061146 16533 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.097617 10130 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:28.098863 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.073925 16630 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.099936 10130 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.048710 16322 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.062201 16534 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.115738 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.116571 10123 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:28.115032 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:28.094542 16580 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:28.120874 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:28.079020 16548 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.078141 16577 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.080965 16461 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.123948 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.136332 10123 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:28.126102 10893 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:28.109620 16348 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:28.147845 10122 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:28.148116 16348 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.149433 10122 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:19:28.149839 10122 pending_rounds.cc:85] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:19:28.150089 10122 pending_rounds.cc:107] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:28.150713 16348 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.152238 16536 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:28.152704 16588 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.188213 16548 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:28.128242 16677 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:19:28.191166 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.192101 10199 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.153443 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2369) took 1042 ms. Trace:
I20260709 13:19:28.154918 16577 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.133045 16534 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:28.156309 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.145725 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.150334 10122 raft_consensus.cc:2889] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.200773 16522 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:28.176810 16558 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.201885 16522 consensus_queue.cc:1243] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:19:28.202137 16538 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.703s	user 0.014s	sys 0.000s
I20260709 13:19:28.202520 16379 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:28.203054 16379 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:28.204210 16379 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.202716 16538 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.205427 16458 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.205672 16379 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:28.207783 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3542) took 1485 ms. Trace:
I20260709 13:19:28.208393 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2510) took 1828 ms. Trace:
I20260709 13:19:28.208334 10197 rpcz_store.cc:276] 0709 13:19:26.722337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.725387 (+  3050us) service_pool.cc:224] Handling call
0709 13:19:26.725822 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.106153 (+380331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.207471 (+1101318us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b54001d0954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:28.207687 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141049728}
I20260709 13:19:28.208948 10050 rpcz_store.cc:276] 0709 13:19:26.379770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.739978 (+360208us) service_pool.cc:224] Handling call
0709 13:19:26.740589 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.952688 (+1212099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.208113 (+255425us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b54002e9354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:28.208322 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2194251520}
I20260709 13:19:28.210071 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.211645 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.212355 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3030) took 2211 ms. Trace:
I20260709 13:19:28.213039 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.214931 16398 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:28.215402 16398 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:28.216492 16398 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.217160 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.217670 16398 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:28.219554 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3053) took 2082 ms (client timeout 2746 ms). Trace:
I20260709 13:19:28.220191 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.220890 10122 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:28.181480 16329 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.098251 16409 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.101171 16663 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.184084 16461 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.195417 10201 rpcz_store.cc:276] 0709 13:19:27.110693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.110824 (+   131us) service_pool.cc:224] Handling call
0709 13:19:27.111215 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.611105 (+499890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.153138 (+542033us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:28.153368 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978605952}
I20260709 13:19:28.248584 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.214679 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:28.220036 10054 rpcz_store.cc:269] 0709 13:19:26.137223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.529620 (+392397us) service_pool.cc:224] Handling call
0709 13:19:28.219312 (+1689692us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:28.219461 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891024640}
I20260709 13:19:28.141369 16599 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.251459 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:28.252220 10054 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:28.229176 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.212857 10128 rpcz_store.cc:276] 0709 13:19:26.000445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.628007 (+627562us) service_pool.cc:224] Handling call
0709 13:19:28.212126 (+1584119us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400125cd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:28.212283 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704254720}
I20260709 13:19:28.210211 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.256022 10197 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.224234 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:28.238083 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:28.239018 10123 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:19:28.243286 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97624.1us
W20260709 13:19:28.180567 16669 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:28.260556 10122 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:28.195793 16536 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:28.269064 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.295630 10122 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:28.265925 10054 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:28.267763 16490 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:19:28.302812 16558 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:19:28.303823 10048 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:28.258502 16601 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:19:28.314039 16379 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:19:28.311750 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.329524 10052 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:28.311220 16623 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:28.302671 15661 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:19:28.335879 16623 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:28.336999 16623 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.305368 16654 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:19:28.337052 16536 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.313710 16263 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:19:28.302671 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:28.325587 16552 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:19:28.300932 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.236624 16664 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.326628 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.340936 16263 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:28.334594 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.344992 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:28.279896 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.327814 10205 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:19:28.258348 16534 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.257723 16514 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:28.311311 16124 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:19:28.246486 16577 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:28.370577 10128 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:28.372023 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.373482 10205 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 14
I20260709 13:19:28.373950 10205 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:28.374426 10205 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.318598 16329 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:28.241545 16240 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:19:28.311398 16678 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:28.373245 10128 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.390064 10201 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: [15.13-16.14]
I20260709 13:19:28.351955 16552 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.390048 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 87459.7us
I20260709 13:19:28.429698 10199 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:28.435595 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.437330 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.438016 10123 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:28.438951 16654 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:28.305401 16548 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:19:28.439670  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:28.439739 16654 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.341584 16510 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:19:28.365840 16664 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.442764 16654 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.443948 16504 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:28.444461 16504 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:28.391163 16534 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.445976 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3041) took 2428 ms. Trace:
I20260709 13:19:28.446394 10126 rpcz_store.cc:276] 0709 13:19:26.017686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.359433 (+1341747us) service_pool.cc:224] Handling call
0709 13:19:27.370839 (+ 11406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.497947 (+127108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.497987 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.508032 (+ 10045us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.761937 (+253905us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:27.761990 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.445521 (+683531us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540036b854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:28.445575 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.445634 (+    59us) raft_consensus.cc:1685] finished
0709 13:19:28.445648 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.445901 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37773,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":14708,"spinlock_wait_cycles":277489024,"thread_start_us":216429,"threads_started":1,"wal-append.queue_time_us":333107}
I20260709 13:19:28.341228 16657 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:28.445024 16664 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.299693 16668 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:28.373188 16538 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.465492 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.466665 10128 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:28.467617 10126 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:28.468227 10122 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:28.373873 16473 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.486409 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:28.499166 10055 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:28.499774 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.501637 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.502951 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.399585 16580 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:28.466327 16594 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.595s	user 0.013s	sys 0.000s
I20260709 13:19:28.391467 16536 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.467778 16490 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.424115 16625 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 13:19:28.566793 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.576536 10126 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:28.576823 10130 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:28.578218 10130 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:28.457675 16657 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:28.551105 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.470898 16548 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.400522 16615 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 13:19:28.565979 16534 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.377225 16577 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.572310 16663 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.469337 16678 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.580276 10123 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
W20260709 13:19:28.581475 10096 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.034s	sys 0.042s
I20260709 13:19:28.508802 16664 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.583702 10128 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:28.592978 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.592774 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 352059us
W20260709 13:19:28.677748 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 429218us
I20260709 13:19:28.505311 16668 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:28.661960 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:28.462160 10197 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:28.467273 16627 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 13:19:28.671497 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:28.672250 10096 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.035s	sys 0.042s
W20260709 13:19:28.696084 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 750669us
I20260709 13:19:28.673362 10123 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:28.674199 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.678102 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.678771 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:28.457170 16552 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:28.706048 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.703229 10047 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:28.706892 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:28.662257 16538 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:28.659819 16599 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.665880 16558 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:28.510437 16473 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.582892 16490 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.738122 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.741952 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:28.510623 16504 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.678321 16678 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.700289 16654 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:28.705046 16329 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:28.751931 16685 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:28.724203 16458 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:28.734072 16461 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:28.516294 16580 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.748389 10170 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:28.754393 16681 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:28.726519 16583 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:28.740764 16647 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:19:28.770185 10051 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.682s	user 0.009s	sys 0.005s
I20260709 13:19:28.824697 10051 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:28.690647 16626 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:19:28.723618 16409 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.666522 16548 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.834856 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.836170 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:28.836583 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.837196 10051 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:28.756877 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:28.838979 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 82213.9us
I20260709 13:19:28.790201 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.842646 10197 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:28.843152 10197 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:28.792250 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.846654 10198 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:28.739468 16667 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.699595 16664 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:28.825389 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 96073.9us
W20260709 13:19:28.813524 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.854027 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.829675 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.837369 10199 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:28.795006 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:28.851356 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 103070us
I20260709 13:19:28.785856 16599 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:28.743855 16557 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:19:28.734526 16538 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.844215 10197 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.791659 16461 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.864157 10197 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:28.876130 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 167700us
W20260709 13:19:28.832866 16630 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.003s
I20260709 13:19:28.893617 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:28.829857 16533 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.000s
I20260709 13:19:28.739296 16558 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:28.823906 16567 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.096s	user 0.012s	sys 0.005s
I20260709 13:19:28.866365 10055 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:28.849953 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.784611 10128 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:28.879537 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.890710 10197 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:28.897944 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:28.903817 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:28.905810 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.851629 16583 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:28.912161 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119990us
I20260709 13:19:28.861119 16663 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:28.792840 16329 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.909210 10047 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:19:28.909961 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:28.923075 10197 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:28.894150 16461 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.797557 16458 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:28.936399 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.912798 10123 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:19:28.880627 16600 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:28.924626 10893 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
W20260709 13:19:28.925318 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45873.6us
I20260709 13:19:28.940747 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:28.913436 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:28.928153 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:28.929143 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:28.943886 10051 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:28.895011 16258 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:28.787230 16654 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.960125 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:28.965448 16329 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.933431 16567 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:28.787606 16657 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.935995 16510 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:28.924142 16684 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:28.985978 10128 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.977726 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:28.987746 10051 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:28.948774 16258 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:28.865859 16667 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.018682 16258 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.958948 16594 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.025691 16594 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:28.968940 16600 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:29.026907 16121 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.939s	user 0.011s	sys 0.000s
I20260709 13:19:29.033238 16121 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.034178 16538 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.035741 16664 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:28.865688 16668 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.010635 16580 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.012768 16689 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:19:29.040020 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.040889 10124 raft_consensus.cc:2376] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16. Re-sending same reply.
I20260709 13:19:29.041764 16657 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:29.042476 10124 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:29.011232 16583 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.014775 16698 raft_consensus.cc:2764] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:28.993134  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:28.990168 16577 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.979350 16510 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.021930 10051 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:28.949620 16663 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:29.092370 16663 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.100893 16121 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.102239 16398 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.884s	user 0.018s	sys 0.000s
I20260709 13:19:29.103255 16510 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:29.023677 16696 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:29.026119 16558 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.908316 16533 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.114573 16533 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:29.027325 10128 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:28.972738 16557 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.115911 16678 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.117410 10054 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:29.118014 16548 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:28.985044 16692 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.032116 16600 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.033720 16473 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.007682  9954 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:29.013777 16458 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.050454 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2368) took 1940 ms. Trace:
I20260709 13:19:28.976044 16567 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.086468 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:27.794708 10131 rpcz_store.cc:276] 0709 13:19:26.296513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.529023 (+1232510us) service_pool.cc:224] Handling call
0709 13:19:27.545824 (+ 16801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.546059 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:27.574377 (+ 28318us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.770666 (+196289us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.773101 (+  2435us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.788961 (+ 15860us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:27.789034 (+    73us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:27.789088 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.789180 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.789327 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17524,"prepare.queue_time_us":31210,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":34915,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":21056,"spinlock_wait_cycles":2819328,"thread_start_us":195565,"threads_started":2}
I20260709 13:19:29.090821 10130 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:29.106195 16398 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:29.052572 16543 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.119596 10204 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:19:29.121671 10200 rpcz_store.cc:276] 0709 13:19:27.109554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.109726 (+   172us) service_pool.cc:224] Handling call
0709 13:19:27.110179 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.464321 (+354142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.050156 (+1585835us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:29.050367 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812386048}
W20260709 13:19:29.129631 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 188992us
I20260709 13:19:29.135524 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:29.126715 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 213389us
I20260709 13:19:29.137660 16599 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:29.133682 10055 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:29.144925 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 249760us
I20260709 13:19:29.145972 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.139714 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.123534 16692 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.115854 16557 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.119308 16696 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:29.161836 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:29.126583 16567 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:29.127070 16473 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:29.158921 16692 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.169345 10051 raft_consensus.cc:2410] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:19:29.177412 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.164383 16548 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:29.170516 16557 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.174296 16398 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:29.164631 16700 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:29.199040 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:29.171623 16348 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:29.198741 16548 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:29.215502 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.206959 16536 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.629s	user 0.015s	sys 0.000s
I20260709 13:19:29.156601 16599 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.213207 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.209982 16700 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.165331 16698 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:19:29.223636 16557 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:29.199544 16693 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 13:19:29.209563 16704 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:29.259140 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109291us
W20260709 13:19:29.260358 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47332.6us
I20260709 13:19:29.229002 16548 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:29.201323 16702 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:29.262611 16700 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:29.268013 16663 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:29.273065 16536 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.284299 16663 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:29.284329 16536 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.285154 16409 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:29.285868 16409 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:29.285501 16663 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:29.288781 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.209297 16510 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.272630 16704 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.302248 16704 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:29.303479 16409 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.304261 16718 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:19:29.305088 16409 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.267234 16678 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.308241 16699 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:19:29.309245 16699 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:29.323235 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.324644 16657 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.328161 16657 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.336396 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:29.342655 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.347366 16557 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.348147 16557 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:29.362241 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.378672 16611 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:29.379731 16611 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:29.381088 16611 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.350864 16557 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.382781 16611 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:29.384616 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.386013 16678 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.386745 16678 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.389361 16536 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.389611 16678 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.391642 10048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2516) took 1333 ms (client timeout 1602 ms). Trace:
I20260709 13:19:29.392562 16490 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.395318 16432 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.400342 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3059) took 1543 ms. Trace:
I20260709 13:19:29.395576 16552 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.392603 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2517) took 1334 ms (client timeout 1752 ms). Trace:
W20260709 13:19:29.401098 10054 rpcz_store.cc:269] 0709 13:19:28.058504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.344627 (+286123us) service_pool.cc:224] Handling call
0709 13:19:29.392404 (+1047777us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54001649d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.392537 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717613824}
I20260709 13:19:29.391013 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3087) took 1257 ms. Trace:
W20260709 13:19:29.402446 16665 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.599s	user 0.008s	sys 0.026s
I20260709 13:19:29.402773 10051 rpcz_store.cc:276] 0709 13:19:28.133441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.086095 (+952654us) service_pool.cc:224] Handling call
0709 13:19:29.390776 (+304681us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b5400136854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.390938 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102482176}
I20260709 13:19:29.404215 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.405139 10051 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:29.399971 16432 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.405939 16667 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.407403 16432 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:29.410276 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3595) took 2951 ms. Trace:
W20260709 13:19:29.398447 10048 rpcz_store.cc:269] 0709 13:19:28.058193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.061473 (+  3280us) service_pool.cc:224] Handling call
0709 13:19:29.391463 (+1329990us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540010b8d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.391579 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475056640}
I20260709 13:19:29.397269 16625 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:29.410312 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:29.400763 10053 rpcz_store.cc:276] 0709 13:19:27.856441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.857055 (+   614us) service_pool.cc:224] Handling call
0709 13:19:27.857374 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.083348 (+225974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.400031 (+1316683us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54003cfad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:29.400268 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507469056}
I20260709 13:19:29.413362 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.414054 10054 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:29.414252 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.414584 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3077) took 1372 ms. Trace:
I20260709 13:19:29.415058 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3080) took 1316 ms. Trace:
I20260709 13:19:29.415452 10053 rpcz_store.cc:276] 0709 13:19:28.099149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.413973 (+1314824us) service_pool.cc:224] Handling call
0709 13:19:29.414998 (+  1025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.416460 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.414922 10054 rpcz_store.cc:276] 0709 13:19:28.041900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.413005 (+1371105us) service_pool.cc:224] Handling call
0709 13:19:29.414523 (+  1518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.410768 10127 rpcz_store.cc:276] 0709 13:19:26.472454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.788315 (+1315861us) service_pool.cc:224] Handling call
0709 13:19:27.799544 (+ 11229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:27.800280 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.410051 (+1609771us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b540036b350. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.410203 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24952704}
I20260709 13:19:29.417186 10126 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:29.422175 10127 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:29.425289 10127 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:29.427345 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.428557 10053 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:29.419668 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:29.438695 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:29.440878 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.441852 10048 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:29.442468 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2514) took 1385 ms. Trace:
I20260709 13:19:29.442813 10048 rpcz_store.cc:276] 0709 13:19:28.057160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.412701 (+1355541us) service_pool.cc:224] Handling call
0709 13:19:29.442381 (+ 29680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.444404 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.447283 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:29.453235 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.437103 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:29.411414 16659 raft_consensus.cc:2764] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:29.455241 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:29.461725 16683 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.694s	user 0.000s	sys 0.003s
I20260709 13:19:29.463488 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 841) took 1338 ms. Trace:
I20260709 13:19:29.463999 10196 rpcz_store.cc:276] 0709 13:19:28.125129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.125291 (+   162us) service_pool.cc:224] Handling call
0709 13:19:28.166688 (+ 41397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.197169 (+ 30481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.197219 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.197810 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.458910 (+261100us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:28.458957 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.459038 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.460974 (+1001936us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:29.462999 (+  2025us) raft_consensus.cc:1685] finished
0709 13:19:29.463028 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.463396 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1588,"raft.queue_time_us":30861,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"spinlock_wait_cycles":55727488,"thread_start_us":218760,"threads_started":1,"wal-append.queue_time_us":438621}
I20260709 13:19:29.466674 16597 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.458850 16363 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:29.467552 16597 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.468065 16522 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:29.468549 16522 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:29.468964 16522 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.470665 16597 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.458323 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:29.472074 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.472419 16600 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:29.473412 16718 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:29.473899 16718 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.474323 16718 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:29.474498 10055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3079) took 1393 ms (client timeout 1783 ms). Trace:
W20260709 13:19:29.475091 10055 rpcz_store.cc:269] 0709 13:19:28.081461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.562663 (+481202us) service_pool.cc:224] Handling call
0709 13:19:29.474227 (+911564us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54000d9b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.474407 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389581568}
I20260709 13:19:29.476653 16667 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:29.480553 16667 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.481143 16667 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:29.481796 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.484534 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.487155 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.488315 10128 raft_consensus.cc:2393] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:29.489296 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.491576 10128 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:29.494505 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.498564 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:29.499725 10055 raft_consensus.cc:2376] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19. Re-sending same reply.
I20260709 13:19:29.501724 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.502594 10128 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:29.502619 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.503412 10048 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:29.504796 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.460386 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.505807 10125 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:19:29.506220 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.427940 16557 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.429899 16533 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:29.507685 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.510239 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.511025 10128 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:29.520432 16167 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:29.521143 16167 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:29.457647 16087 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:29.524451 16087 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.524938 16087 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:29.525171 16714 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 13:19:29.525794 16522 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:29.527794 16701 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.544s	user 0.003s	sys 0.005s
I20260709 13:19:29.528578 16654 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.528676 16522 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:29.532909 10128 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:29.452554 16678 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:29.537618 10127 raft_consensus.cc:2410] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:29.533494 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:29.496877 16461 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.006s
I20260709 13:19:29.477845 16633 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:29.443800 16625 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.535874 16704 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.541419 16699 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:29.544432 16458 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.543689 10051 raft_consensus.cc:2376] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16. Re-sending same reply.
I20260709 13:19:29.561124 16704 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.549549 16461 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.566123 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.547923 16633 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:29.535463 16557 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.579655 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.544675 16678 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.583750 10051 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:29.588302 16461 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.590651 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3076) took 1153 ms. Trace:
I20260709 13:19:29.593348 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3077) took 1155 ms. Trace:
I20260709 13:19:29.595589 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 844) took 1242 ms. Trace:
I20260709 13:19:29.603850 10201 rpcz_store.cc:276] 0709 13:19:28.353265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.353534 (+   269us) service_pool.cc:224] Handling call
0709 13:19:28.354084 (+   550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:28.433358 (+ 79274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.595219 (+1161861us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b540035b1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:29.595511 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34211072}
I20260709 13:19:29.597918 10131 rpcz_store.cc:276] 0709 13:19:28.437531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.124701 (+687170us) service_pool.cc:224] Handling call
0709 13:19:29.145383 (+ 20682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.145792 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.145821 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.146313 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.453129 (+306816us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:29.453176 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.453256 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.590354 (+137098us) raft_consensus.cc:1685] finished
0709 13:19:29.590383 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.590575 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":768,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":13117,"thread_start_us":281711,"threads_started":1,"wal-append.queue_time_us":406987}
I20260709 13:19:29.602259 10130 rpcz_store.cc:276] 0709 13:19:28.437818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.125484 (+687666us) service_pool.cc:224] Handling call
0709 13:19:29.148699 (+ 23215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.206269 (+ 57570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.206317 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.222246 (+ 15929us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.538554 (+316308us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:29.538595 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.539660 (+  1065us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.593067 (+ 53407us) raft_consensus.cc:1685] finished
0709 13:19:29.593095 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.593271 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17279,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":320487,"threads_started":2,"wal-append.queue_time_us":324764}
I20260709 13:19:29.577723 16578 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.578655 16699 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:29.616354 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:29.618038 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.617924 10051 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.631438 16718 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.633302 10125 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:19:29.631556 16678 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.635883 16667 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.636795 16718 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.638705 16667 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.639780 16543 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.000s
I20260709 13:19:29.640398 16329 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.641168 16329 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.546121 16167 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.654187 16167 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.659524 10051 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:29.660998 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.661721 10051 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
I20260709 13:19:29.662652 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:29.672587 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 214605us
I20260709 13:19:29.676942 10051 raft_consensus.cc:2410] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:19:29.623654 16510 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.678233 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 187708us
I20260709 13:19:29.619238 16578 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.623507 16363 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:29.687700 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:29.688434 10051 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.694801 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.699070 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.670135 16567 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.706562 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.684125 16627 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.710047 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:29.697104 16692 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.714818 16692 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.720607 10124 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:29.724287 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3056) took 1868 ms. Trace:
I20260709 13:19:29.724983 10052 rpcz_store.cc:276] 0709 13:19:27.855485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.858219 (+  2734us) service_pool.cc:224] Handling call
0709 13:19:27.865556 (+  7337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.335600 (+470044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.723837 (+1388237us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54001f9ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:29.724183 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066503680}
I20260709 13:19:29.732432 16625 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:29.734088 16725 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:29.621452 16633 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.717536 16692 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.718830 16510 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:29.750218 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:29.763463 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.763917 10051 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:29.765511 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.765486 16087 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.752120 16719 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.771397 16557 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.771661 16719 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.773069 16087 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:29.751710 16363 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.774231 16719 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.714183 16548 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.766779 10051 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:29.776023 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:29.714423 16627 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.777092 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.776742 10051 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.702057 16699 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.781932 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:29.783927 10049 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:29.784413 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3061) took 1927 ms. Trace:
I20260709 13:19:29.775272 16633 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.787338 16548 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.788407 16626 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:29.788892 16626 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:29.789882 16626 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.784762 10049 rpcz_store.cc:276] 0709 13:19:27.857077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.858220 (+  1143us) service_pool.cc:224] Handling call
0709 13:19:29.781741 (+1923521us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:19:29.784343 (+  2602us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21047168}
I20260709 13:19:29.786296 16578 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.796267 10049 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:29.801510 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3070) took 1923 ms. Trace:
I20260709 13:19:29.802016 10049 rpcz_store.cc:276] 0709 13:19:27.877920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.795641 (+1917721us) service_pool.cc:224] Handling call
0709 13:19:29.796040 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.801184 (+  5144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.801435 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.803288 10049 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:29.803849 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3071) took 1925 ms. Trace:
I20260709 13:19:29.804144 10049 rpcz_store.cc:276] 0709 13:19:27.878182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.802786 (+1924604us) service_pool.cc:224] Handling call
0709 13:19:29.803068 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.803659 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.803805 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.805476 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3072) took 1865 ms. Trace:
I20260709 13:19:29.805819 10049 rpcz_store.cc:276] 0709 13:19:27.940422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.804817 (+1864395us) service_pool.cc:224] Handling call
0709 13:19:29.805053 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.805199 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.805217 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.805249 (+    32us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:29.805269 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.805304 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.805403 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.798725 16322 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.808266 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3073) took 1854 ms. Trace:
I20260709 13:19:29.809566 10052 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:29.824095 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3066) took 1961 ms. Trace:
I20260709 13:19:29.824326 10049 rpcz_store.cc:276] 0709 13:19:27.954026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.806586 (+1852560us) service_pool.cc:224] Handling call
0709 13:19:29.806842 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.806985 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.807001 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.807030 (+    29us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:29.807051 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.807087 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.808197 (+  1110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.824815 10052 rpcz_store.cc:276] 0709 13:19:27.862729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.777922 (+1915193us) service_pool.cc:224] Handling call
0709 13:19:29.809389 (+ 31467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.823703 (+ 14314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.824009 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.828575 16567 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.832448 10052 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:29.833312 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3075) took 1865 ms. Trace:
I20260709 13:19:29.835069 10051 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:29.836876 10052 rpcz_store.cc:276] 0709 13:19:27.968655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.827433 (+1858778us) service_pool.cc:224] Handling call
0709 13:19:29.832244 (+  4811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.832999 (+   755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.833239 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587584}
W20260709 13:19:29.841887 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.842835 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.845484 16692 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.847601 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.848516 10892 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:29.846113 16692 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.827075 16633 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:29.855050 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.821164 16719 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:29.861521 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 830) took 2700 ms (client timeout 2244 ms). Trace:
I20260709 13:19:29.861774 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:29.869086 10048 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:29.869819 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3563) took 1023 ms. Trace:
I20260709 13:19:29.870230 10204 rpcz_store.cc:276] 0709 13:19:28.846760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.952135 (+105375us) service_pool.cc:224] Handling call
0709 13:19:28.976897 (+ 24762us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:29.130045 (+153148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.869470 (+739425us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540039e754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:29.869757 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327028992}
I20260709 13:19:29.864353 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:29.864722 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.846426 16627 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:29.868800 10203 rpcz_store.cc:269] 0709 13:19:27.160980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.161105 (+   125us) service_pool.cc:224] Handling call
0709 13:19:29.861177 (+2700072us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:29.861377 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619020032}
I20260709 13:19:29.871039 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.878695 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.880558 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.881080 16594 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.881362 10896 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:29.882827 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.873903 16633 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.883630 10892 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:29.885497 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.886397 10892 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:29.887236 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.888680 10896 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:29.889470 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.890190 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.890970 10898 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:29.891296 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.892136 10130 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:29.892932 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.894156 16588 raft_consensus.cc:2764] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:29.893018 16458 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:29.895552 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.898908 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.839480 10052 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:29.900447 10051 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:19:29.901039 10051 pending_rounds.cc:85] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:19:29.901396 10051 pending_rounds.cc:107] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:19:29.901768 10051 raft_consensus.cc:2889] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.897763 10096 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.902956 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:29.903201 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.904430 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.905535 10131 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:29.905406 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2513) took 1848 ms. Trace:
I20260709 13:19:29.907735 10052 rpcz_store.cc:276] 0709 13:19:28.056820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.838848 (+1782028us) service_pool.cc:224] Handling call
0709 13:19:29.839299 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.905071 (+ 65772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.905334 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.909828 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:29.910681 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:29.906253 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.847851 16700 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:29.908092 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:29.917475 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.913655 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:29.919605 10096 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:29.914875 16322 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:29.922135 16548 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:29.912487 16588 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:29.915217 16594 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:29.923225 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2515) took 1865 ms. Trace:
I20260709 13:19:29.943819 10051 rpcz_store.cc:276] 0709 13:19:28.057821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.898850 (+1841029us) service_pool.cc:224] Handling call
0709 13:19:29.899946 (+  1096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.922447 (+ 22501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.923161 (+   714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.945716 10051 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:19:29.923540 16718 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:29.947213 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3069) took 2069 ms. Trace:
I20260709 13:19:29.929193 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:29.934099 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:29.938894 10123 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:29.913985 16627 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:29.949656 16627 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.950181 16627 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:29.885542 16633 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:29.923409 16659 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:29.938711 16322 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:29.920883 16599 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:29.876413 16719 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.938954 16519 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:29.957525 10130 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:29.946457 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3084) took 1813 ms. Trace:
I20260709 13:19:29.943660 16588 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.940743 16548 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:29.948268 16718 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.948208 16654 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.976732 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.961802 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:29.968273 10054 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:19:29.982491 10054 pending_rounds.cc:85] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:29.982885 10054 pending_rounds.cc:107] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:19:29.983249 10054 raft_consensus.cc:2889] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.984313 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3085) took 1851 ms. Trace:
I20260709 13:19:29.984737 10054 rpcz_store.cc:276] 0709 13:19:28.132850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.967257 (+1834407us) service_pool.cc:224] Handling call
0709 13:19:29.967730 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.983891 (+ 16161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.984212 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.977222 10051 rpcz_store.cc:276] 0709 13:19:28.132553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.944848 (+1812295us) service_pool.cc:224] Handling call
0709 13:19:29.945376 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.946150 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.946391 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.989668 16167 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.990468 16167 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:29.978637 16588 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:29.990823 16731 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:19:29.963394 16322 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:29.966360 16657 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:29.979461 16659 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:29.993960 10051 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:19:30.000636 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3090) took 1548 ms. Trace:
I20260709 13:19:30.000953 10051 rpcz_store.cc:276] 0709 13:19:28.452401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.992639 (+1540238us) service_pool.cc:224] Handling call
0709 13:19:29.993681 (+  1042us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.000334 (+  6653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.000580 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.962728 16599 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.002386 16599 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:29.975628 16490 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:29.996663 10893 raft_consensus.cc:2410] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:19:29.979990 16548 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:29.981107 16718 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:30.008108 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.992609 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 137222us
I20260709 13:19:29.952426 16594 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:30.007540 16490 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:30.013414 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:30.017722 16363 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:30.018280 16363 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.018676 16363 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:30.020840 16719 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:30.022235 16700 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:30.023492 16692 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:30.025156 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3088) took 1891 ms (client timeout 1775 ms). Trace:
W20260709 13:19:30.025661 10047 rpcz_store.cc:269] 0709 13:19:28.133728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.794659 (+660931us) service_pool.cc:224] Handling call
0709 13:19:30.024881 (+1230222us) spinlock_profiling.cc:243] Waited 50.5 ms on lock 0x7b5400087854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:30.025067 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116088192}
W20260709 13:19:30.029120 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.029625 10048 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:30.030349 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3081) took 1899 ms. Trace:
I20260709 13:19:30.030652 10048 rpcz_store.cc:276] 0709 13:19:28.131475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.029045 (+1897570us) service_pool.cc:224] Handling call
0709 13:19:30.029444 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.030063 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.030293 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:29.996438 16552 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:30.032768 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.034361 10170 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:30.035085 10050 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:19:30.038852 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.039287 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2522) took 1337 ms. Trace:
I20260709 13:19:30.039424 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:30.039845 10050 rpcz_store.cc:276] 0709 13:19:28.702148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.034418 (+1332270us) service_pool.cc:224] Handling call
0709 13:19:30.034721 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.038926 (+  4205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.039215 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.035625 10047 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:30.044920 10130 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:29.948318 10053 rpcz_store.cc:276] 0709 13:19:27.877685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.792104 (+1914419us) service_pool.cc:224] Handling call
0709 13:19:29.792550 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.792790 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:29.914640 (+121850us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.946831 (+ 32191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.946892 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:29.946938 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.947006 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.947153 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2150,"prepare.queue_time_us":147434,"raft.queue_time_us":2200,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":85256,"threads_started":2,"wal-append.queue_time_us":30035}
I20260709 13:19:30.046579 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2521) took 1344 ms. Trace:
I20260709 13:19:30.047142 10047 rpcz_store.cc:276] 0709 13:19:28.701895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.026631 (+1324736us) service_pool.cc:224] Handling call
0709 13:19:30.027002 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.046313 (+ 19311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.046518 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.049851 10050 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:29.991914 16654 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.032855 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.052295 10053 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:30.016752 16490 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:30.052410 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90726.7us
I20260709 13:19:30.118263 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.054076 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 839) took 1064 ms. Trace:
I20260709 13:19:30.067495 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:30.130800 10131 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:30.074621 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:30.111336 10122 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [14.9-16.10]
I20260709 13:19:30.112286 10127 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:30.131901 10050 rpcz_store.cc:276] 0709 13:19:28.992704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.049307 (+1056603us) service_pool.cc:224] Handling call
0709 13:19:30.049618 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.053678 (+  4060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.053986 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.136087 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:30.140765 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.137277 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:30.138104 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63563.1us
I20260709 13:19:30.136523 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 840) took 1143 ms. Trace:
I20260709 13:19:30.136965 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:30.141613 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:30.133982 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66566.4us
I20260709 13:19:30.146948 10127 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:30.077833 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:30.147420 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3082) took 2015 ms. Trace:
I20260709 13:19:30.146016 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.151695 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:30.043817 16533 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:30.151873 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 74172.4us
I20260709 13:19:30.156539 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:30.147773 10053 rpcz_store.cc:276] 0709 13:19:28.995060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.049532 (+1054472us) service_pool.cc:224] Handling call
0709 13:19:30.051790 (+  2258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.136256 (+ 84466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.136460 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.166119 10198 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:30.156293 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.167692 10205 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:30.141166 10050 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:30.168970 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.169173 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 841) took 1139 ms. Trace:
I20260709 13:19:30.160696 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.170094 10050 rpcz_store.cc:276] 0709 13:19:29.050611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.139916 (+1089305us) service_pool.cc:224] Handling call
0709 13:19:30.140708 (+   792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.168479 (+ 27771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.169074 (+   595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.171228 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.160338 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.161537 10130 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:19:30.173650 10130 pending_rounds.cc:85] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 8
I20260709 13:19:30.174036 10130 pending_rounds.cc:107] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:19:30.174361 10130 raft_consensus.cc:2889] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.174481 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.158409 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:30.067960 16087 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:30.033605 16600 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:19:30.048148 16552 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:30.112614 16613 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.215840 10204 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:30.216408 10202 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:30.214466 10048 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:30.204898 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:30.066947 16598 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:30.165031 10122 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:30.118040 16578 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.142702 16633 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:30.076798 16651 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:30.213315 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:30.215240 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.203524 16657 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:30.144840 16409 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:30.065181 16167 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:30.067764 16357 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 13:19:30.075618 16727 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 13:19:30.206310 16087 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.222411 10202 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:19:30.064648 16625 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 13:19:30.203219 10205 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:30.064744 16363 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.209936 16567 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:30.207073 16519 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:30.145027 16692 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
W20260709 13:19:30.220731 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 144964us
W20260709 13:19:30.308838 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95607.8us
W20260709 13:19:30.237208 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.294880 10048 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:19:30.299304 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.294288 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:30.153219 16700 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 13:19:30.302294 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 154790us
I20260709 13:19:30.075524 16594 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.076467 16659 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.220110 16651 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:30.318850 10048 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:30.295078 16087 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:30.227885 16409 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:30.335809 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 154764us
I20260709 13:19:30.297709 16633 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:30.303849 16567 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:30.328845 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.222633 16578 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.224180 16613 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.295401 16167 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:19:30.313614 16742 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:19:30.353894 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 201871us
I20260709 13:19:30.355175 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.316186 16526 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:19:30.397168 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.410712 10196 raft_consensus.cc:2410] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:19:30.332576 16625 consensus_queue.cc:1243] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:30.333412 16409 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:30.411955 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.408825 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.397168 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.317791 16657 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:30.348232 16567 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:30.314640 16519 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.453320 16534 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:30.455674 16578 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:30.456998 16743 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:19:30.458182 16654 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:30.459381 16744 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:19:30.461165 16613 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:30.501993 10168 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:30.504484 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 186103us
I20260709 13:19:30.505820 10899 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:30.408361 16678 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:30.490770 10892 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:30.447855 16746 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:30.526636 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.531009 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.533131 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:30.537997 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 208517us
W20260709 13:19:30.568548 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.479310 16349 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:30.594691 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:30.496035 16578 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.503392 16751 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.610683 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 845) took 1263 ms. Trace:
I20260709 13:19:30.395272 16552 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:30.611241 10048 rpcz_store.cc:276] 0709 13:19:29.347688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.328145 (+980457us) service_pool.cc:224] Handling call
0709 13:19:30.328607 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.328910 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:30.347422 (+ 18512us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.610068 (+262646us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.610280 (+   212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.610331 (+    51us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:30.610382 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.610458 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.610614 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7820,"prepare.queue_time_us":20898,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":67,"thread_start_us":254080,"threads_started":1}
W20260709 13:19:30.536933 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 199564us
I20260709 13:19:30.613871 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.615895 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:30.611258 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:30.617362 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:30.620944 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.532240 16694 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:30.487732 16625 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:19:30.534518 16746 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:30.416997 16742 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.487061 16534 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:30.649477 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109460us
I20260709 13:19:30.600252 16349 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:30.625126 16599 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.002s
I20260709 13:19:30.530685 16533 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:30.672024 10202 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:30.639421 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:30.674636 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 275291us
I20260709 13:19:30.677079 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.597941 16613 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:30.519379 16747 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 13:19:30.680732 10127 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:30.684737 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3091) took 2227 ms. Trace:
I20260709 13:19:30.689188 10051 rpcz_store.cc:276] 0709 13:19:28.457119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.001690 (+1544571us) service_pool.cc:224] Handling call
0709 13:19:30.002084 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.002517 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.002552 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.003189 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:19:30.276098 (+272909us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:30.276169 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.276307 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.684303 (+407996us) raft_consensus.cc:1685] finished
0709 13:19:30.684338 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.684590 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1402,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":272332,"threads_started":1,"wal-append.queue_time_us":314168}
I20260709 13:19:30.691390 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.692062 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:30.695087 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.696168 10202 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:30.692188 10051 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:30.587527 16678 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:30.702307 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.702091 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.520830 16633 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.705817 10048 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:30.660797 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.642s (SENT) [suppressed 14 similar messages]
W20260709 13:19:30.708151 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47415.8us
W20260709 13:19:30.709012 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.653614 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 340096us
I20260709 13:19:30.712976 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:30.724637 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.628269 16578 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:30.730626 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.726248 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:30.750564 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:30.752128 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:30.752815 10051 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:30.759806 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:30.761323 10048 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:30.780261 10202 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:30.787041 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.787590 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:30.725090 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.791991 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3074) took 2827 ms. Trace:
I20260709 13:19:30.800280 10048 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:30.788380 10896 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 14.
W20260709 13:19:30.725368 16600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.018s
I20260709 13:19:30.806125 10196 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:30.806692 16613 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.806609 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:30.809070 16623 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:30.810493 16756 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:30.806279 16749 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:30.801270 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3095) took 1385 ms. Trace:
I20260709 13:19:30.806344 16759 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:30.812438 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.814102 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:30.689335 16363 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:30.598238 16696 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:30.834028 10051 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:30.826289 10205 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:30.728720 16526 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.869791 10129 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:30.792865 10049 rpcz_store.cc:276] 0709 13:19:27.964468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.826694 (+1862226us) service_pool.cc:224] Handling call
0709 13:19:29.831701 (+  5007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.835583 (+  3882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.835623 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.836239 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.914195 (+ 77956us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:30.405331 (+491136us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b18004ad280. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf
0709 13:19:30.405615 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.726435 (+320820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.726532 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.726588 (+    56us) raft_consensus.cc:1685] finished
0709 13:19:30.791619 (+ 65031us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.791895 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8651,"prepare.queue_time_us":35662,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"raft.queue_time_us":207040,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":33633,"spinlock_wait_cycles":4642944,"thread_start_us":403506,"threads_started":3,"wal-append.queue_time_us":110281}
W20260709 13:19:30.858266 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.013s	sys 0.012s
I20260709 13:19:30.681402 16599 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:30.870636 16600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.002s	sys 0.019s
I20260709 13:19:30.878057 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:30.644057 16751 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.884055 10048 rpcz_store.cc:276] 0709 13:19:29.416258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.787464 (+1371206us) service_pool.cc:224] Handling call
0709 13:19:30.787804 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.800895 (+ 13091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.801187 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.883759 16749 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:30.891949 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 358883us
W20260709 13:19:30.875464 16548 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.005s
W20260709 13:19:30.899331 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.013s	sys 0.012s
I20260709 13:19:30.881474 16756 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.890671 16767 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:19:30.921458 10127 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:19:30.845474 16719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.008s	sys 0.011s
W20260709 13:19:30.936102 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.953877 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:30.953357 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 300891us
I20260709 13:19:30.897863 16765 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:30.946519 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 241309us
I20260709 13:19:30.912173 16727 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:30.921926 16600 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:30.880936 16613 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:30.953115 16719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.009s	sys 0.012s
I20260709 13:19:30.816969 16349 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.939469 16363 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.939703 16657 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.950287 16756 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:30.911306 16700 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:19:30.884253 16533 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.922941 16526 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.967912 16727 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:30.913257 16761 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:19:30.801615 16746 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.946516 16752 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.920449 16594 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.997931 16768 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:31.033143 16772 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.974342 16765 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.072728 16742 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.065299 16678 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.106793 16786 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:31.110523 16773 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:19:31.111436 16760 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.238s	user 0.000s	sys 0.002s
I20260709 13:19:31.098768 16752 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.114557 16752 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.043835 16749 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.102049 16746 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.104197 16772 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.096539 16775 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:31.101475 16768 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.142199 16768 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:31.142081 16772 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.177091 10050 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 16. Current term is 18. Ops: []
I20260709 13:19:30.913539 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 846) took 1315 ms. Trace:
I20260709 13:19:31.180588 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 842) took 2123 ms. Trace:
I20260709 13:19:31.181275 10053 rpcz_store.cc:276] 0709 13:19:29.057709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.166090 (+1108381us) service_pool.cc:224] Handling call
0709 13:19:30.204104 (+ 38014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.204397 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.204443 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.210677 (+  6234us) log.cc:844] Serialized 26 byte log entry
0709 13:19:30.537770 (+327093us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:30.537820 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.537898 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.179534 (+641636us) raft_consensus.cc:1685] finished
0709 13:19:31.179641 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.180237 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":213942,"threads_started":1,"wal-append.queue_time_us":361472}
I20260709 13:19:31.179705 10049 rpcz_store.cc:276] 0709 13:19:29.597436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.911956 (+1314520us) service_pool.cc:224] Handling call
0709 13:19:30.912394 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.912645 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.912673 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.912859 (+   186us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:30.912908 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.912982 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.913134 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:31.185850 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:31.187850 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.925388 10048 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:19:31.190797 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3099) took 1505 ms. Trace:
I20260709 13:19:31.191387 10048 rpcz_store.cc:276] 0709 13:19:29.684870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.916604 (+1231734us) service_pool.cc:224] Handling call
0709 13:19:30.917192 (+   588us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.189748 (+272556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.190521 (+   773us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:31.190907 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 236995us
I20260709 13:19:31.187964 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 783) took 2295 ms. Trace:
W20260709 13:19:31.211962 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 398561us
W20260709 13:19:31.213141 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.037s	sys 0.087s
W20260709 13:19:31.213804 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.037s	sys 0.087s
W20260709 13:19:31.214298 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 503559us
W20260709 13:19:31.216058 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.201743 10126 rpcz_store.cc:276] 0709 13:19:28.892677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.420512 (+527835us) service_pool.cc:224] Handling call
0709 13:19:29.453480 (+ 32968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.457144 (+  3664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.457182 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.457727 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.685516 (+227789us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:29.694584 (+  9068us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.697313 (+  2729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.697398 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.111370 (+1413972us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b1800401680. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:31.186866 (+ 75496us) raft_consensus.cc:1685] finished
0709 13:19:31.187152 (+   286us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.187615 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5483,"prepare.queue_time_us":8312,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":5992,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":47638,"spinlock_wait_cycles":19353344,"thread_start_us":143125,"threads_started":1,"wal-append.queue_time_us":203913}
I20260709 13:19:31.221650 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:31.225052 10126 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:19:31.195052 10048 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:31.233106 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3100) took 1542 ms. Trace:
I20260709 13:19:31.233376 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:31.245440 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45205.8us
I20260709 13:19:31.245182 16363 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.233992 10048 rpcz_store.cc:276] 0709 13:19:29.691065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.193041 (+1501976us) service_pool.cc:224] Handling call
0709 13:19:31.194043 (+  1002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.232333 (+ 38290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.232840 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:31.235468 16167 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.002s	sys 0.010s
I20260709 13:19:31.235772 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3088) took 1115 ms. Trace:
I20260709 13:19:31.243610 16349 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:31.249809 16167 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.003s	sys 0.010s
I20260709 13:19:31.250370 16363 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:31.251737 16363 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:31.253764 16167 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:31.249845 10126 rpcz_store.cc:276] 0709 13:19:30.120410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.223911 (+1103501us) service_pool.cc:224] Handling call
0709 13:19:31.224542 (+   631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.235169 (+ 10627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.235629 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.257100 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.257584 10048 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:31.258677 10126 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:19:31.262179 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.260067 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 847) took 1461 ms. Trace:
I20260709 13:19:31.269707 10048 rpcz_store.cc:276] 0709 13:19:29.798929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.256487 (+1457558us) service_pool.cc:224] Handling call
0709 13:19:31.257187 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.259533 (+  2346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.259938 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.994895 16519 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:30.946197 16548 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:31.310024 16409 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.847s	user 0.016s	sys 0.007s
I20260709 13:19:31.310738 16409 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.268718 10126 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:31.291766 16623 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:31.293234 16758 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:31.294416 16526 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:31.293499 16625 consensus_queue.cc:1175] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 16 status { last_received { term: 15 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:31.295892 16594 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:31.302451 16588 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.305s	user 0.014s	sys 0.001s
I20260709 13:19:31.304066 16769 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:31.304764 16534 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.306212 16751 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.307633 16779 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:31.322398 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.327613 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:31.339540 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.332599 16769 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.346276 16769 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.352210 10047 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:31.355218 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92743us
W20260709 13:19:31.355641 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 370102us
I20260709 13:19:31.357398 16783 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:31.346535 16588 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.363456 16779 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.121049 16727 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.082723 16657 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.366457 16779 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.368193 16657 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:31.376806 10019 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.980s (SENT)
I20260709 13:19:31.367422 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.380278 16594 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.381400 16770 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:19:31.383005 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.005s	sys 0.013s
I20260709 13:19:31.376924 16753 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:31.386690 16749 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.366516 16588 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:31.385555 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 136467us
I20260709 13:19:31.397917 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3087) took 1287 ms. Trace:
I20260709 13:19:31.401438 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3082) took 1292 ms. Trace:
I20260709 13:19:31.402524 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.405589 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.405881 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.408324 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.410440 10198 raft_consensus.cc:2410] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:31.410712 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 730) took 1532 ms. Trace:
I20260709 13:19:31.402014 10130 rpcz_store.cc:276] 0709 13:19:30.108845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.108982 (+   137us) service_pool.cc:224] Handling call
0709 13:19:30.109353 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.175013 (+ 65660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.400695 (+1225682us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:31.401231 (+   536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118712576}
I20260709 13:19:31.424279 16770 consensus_queue.cc:1243] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:31.408305 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3078) took 2964 ms. Trace:
I20260709 13:19:31.425503 10123 rpcz_store.cc:276] 0709 13:19:28.444288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.282810 (+838522us) service_pool.cc:224] Handling call
0709 13:19:29.295709 (+ 12899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.949208 (+653499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.407602 (+1458394us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540015b154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:31.408041 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478454016}
I20260709 13:19:31.430778 16548 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.432566 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2534) took 1986 ms. Trace:
I20260709 13:19:31.432334 16792 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:31.433246 16504 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:31.436833 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.437619 16772 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:31.438928 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:31.440454 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.386742 10126 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:31.440820 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.441736 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:31.442328 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.443441 10126 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
W20260709 13:19:31.443513 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66939.4us
I20260709 13:19:31.445312 10126 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:31.445379 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.448051 16626 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:31.451355 16657 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:31.451613 16750 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:31.452168 16657 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:31.452569 16750 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.453289 16750 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.453824 16657 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.413761 10893 rpcz_store.cc:276] 0709 13:19:29.878177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.878329 (+   152us) service_pool.cc:224] Handling call
0709 13:19:31.409864 (+1531535us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:31.410320 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268599040}
I20260709 13:19:31.456658 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:31.398510 10127 rpcz_store.cc:276] 0709 13:19:30.110844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.221791 (+110947us) service_pool.cc:224] Handling call
0709 13:19:31.397320 (+1175529us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:31.397655 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839099648}
I20260709 13:19:31.466238 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:31.467942 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74048.8us
I20260709 13:19:31.471453 16770 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:31.472162 16770 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:31.477195 16770 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.482564 16758 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:31.484050 16747 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:31.484723 16747 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:31.484920 16792 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:31.486084 16600 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:31.486657 16792 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:31.487106 16792 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:31.490865 16792 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.492728 16748 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:31.493546 16748 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:31.433048 10051 rpcz_store.cc:276] 0709 13:19:29.445700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.895512 (+1449812us) service_pool.cc:224] Handling call
0709 13:19:30.895954 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.896400 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.896441 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.897134 (+   693us) log.cc:844] Serialized 26 byte log entry
0709 13:19:31.387308 (+490174us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:31.387516 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.387699 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.432034 (+ 44335us) raft_consensus.cc:1685] finished
0709 13:19:31.432060 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.432338 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":19351,"thread_start_us":119949,"threads_started":1,"wal-append.queue_time_us":404469}
I20260709 13:19:31.492969 16792 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:31.522284 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.523991 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 843) took 2449 ms. Trace:
I20260709 13:19:31.528910 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:31.522826 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:30.128532  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:31.518286 16754 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:31.513936 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.510872 16696 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.520157 10126 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:31.544507 10202 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:31.543285 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.531641  9954 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:31.554194 16765 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.515017 16599 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.536686 16409 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.542819 16754 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:31.557139 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1016) took 1136 ms. Trace:
I20260709 13:19:31.545642 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.528522 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:31.522837 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.537835 10050 rpcz_store.cc:276] 0709 13:19:29.074946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.171534 (+1096588us) service_pool.cc:224] Handling call
0709 13:19:30.171842 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.178786 (+1006944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.523583 (+344797us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540024ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:31.523890 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309832576}
W20260709 13:19:31.594365 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.580555 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:31.558440 10897 rpcz_store.cc:276] 0709 13:19:30.420911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.421210 (+   299us) service_pool.cc:224] Handling call
0709 13:19:31.415622 (+994412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.423512 (+  7890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.423548 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.423613 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:31.423663 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.423773 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.556613 (+132840us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54006ed7d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:31.556817 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2192041728}
I20260709 13:19:31.523986 16750 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.551335 16770 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.618746 10131 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:31.584725 16753 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.546605 16329 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.582990 16613 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.626871 10051 raft_consensus.cc:2376] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19. Re-sending same reply.
I20260709 13:19:31.636097 10130 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
W20260709 13:19:31.636816 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 415502us
I20260709 13:19:31.540774 16769 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.643639 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:31.615527 16349 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:31.536662 16747 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.495079 16748 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:30.155066 10052 rpcz_store.cc:276] 0709 13:19:28.131912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.909572 (+1777660us) service_pool.cc:224] Handling call
0709 13:19:29.910185 (+   613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.910485 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:29.992245 (+ 81760us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.143989 (+151744us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.144057 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:30.144121 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.144199 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.147340 (+  3141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39053,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":537,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4622,"thread_start_us":231510,"threads_started":2}
W20260709 13:19:31.653556 16322 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.612s	user 0.011s	sys 0.020s
I20260709 13:19:31.654748 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:31.663103 16598 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.435s	user 0.007s	sys 0.017s
I20260709 13:19:31.663640 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.665948 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.667380 16558 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.667635 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3073) took 3231 ms. Trace:
I20260709 13:19:31.668119 16558 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.654338 16753 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.669462 10055 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:31.670433 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2518) took 3611 ms. Trace:
I20260709 13:19:31.670454 10056 raft_consensus.cc:2410] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:31.670897 10055 rpcz_store.cc:276] 0709 13:19:28.063672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.660944 (+3597272us) service_pool.cc:224] Handling call
0709 13:19:31.668949 (+  8005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.670078 (+  1129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.670311 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.671571 16775 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.672389 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.672453 16775 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.673552 10055 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:31.668279 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:31.675915 16567 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.165s	user 0.016s	sys 0.006s
I20260709 13:19:31.676766 10050 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:31.676543 16567 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.678170 10205 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:31.675304 16473 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:31.674947 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.683414 16767 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:31.684615 16490 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.533s	user 0.024s	sys 0.003s
I20260709 13:19:31.685204 16490 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.683553 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.687601 10125 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:31.644156 16754 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.687840 16692 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.689046 16692 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.689621 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.689760 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3084) took 1580 ms. Trace:
I20260709 13:19:31.690312 10125 rpcz_store.cc:276] 0709 13:19:30.109433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.109678 (+   245us) service_pool.cc:224] Handling call
0709 13:19:30.110194 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.688180 (+1577986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.689367 (+  1187us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54001c7854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:31.689687 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3623468288}
I20260709 13:19:31.691646 16654 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.697959 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.698616 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.692493 16762 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.691483 10055 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:31.700366 16692 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:31.701648 16651 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.361s	user 0.009s	sys 0.014s
W20260709 13:19:31.702358 16087 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.357s	user 0.013s	sys 0.017s
I20260709 13:19:31.702951 16578 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.660315 16322 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.738075 16567 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.742955 16087 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.675654 16775 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:31.746503 16087 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:31.668164 10124 rpcz_store.cc:276] 0709 13:19:28.436352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.054678 (+618326us) service_pool.cc:224] Handling call
0709 13:19:29.055151 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.723189 (+668038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.654011 (+1930822us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54001f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:31.667399 (+ 13388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529789184}
I20260709 13:19:31.748428 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.749552 10124 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:19:31.750252 10925 heartbeater.cc:646] Failed to heartbeat to 127.8.183.126:40341 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.183.126:40341 timed out after 15.000s (SENT)
W20260709 13:19:31.704690 16627 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.743s	user 0.017s	sys 0.009s
I20260709 13:19:31.751731 16627 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.752307 10899 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:31.753154 16781 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.706293 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.705224 16750 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.707674 10201 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:19:31.707913 16748 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:31.709342 16756 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.012s
I20260709 13:19:31.710597 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.693185 16699 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:31.694000 16458 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:31.772029 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 191599us
I20260709 13:19:31.713783 16598 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:31.694140 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.783481 10123 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:31.784228 10048 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:31.785717 10123 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:31.786008 10048 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:19:31.786361 10048 pending_rounds.cc:85] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 7
I20260709 13:19:31.786628 10048 pending_rounds.cc:107] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:19:31.715973 16753 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:31.694214 16357 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:31.788379 16357 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:31.789680 16357 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.695006 10126 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:31.695960 10127 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:31.694300 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48864.2us
I20260709 13:19:31.696503 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:31.723608 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.723922 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.708715 10047 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:31.728765 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.786909 10048 raft_consensus.cc:2889] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:31.692562 16769 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.739773 10200 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:31.739635 16762 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.741273 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:31.742345 16718 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.734s	user 0.007s	sys 0.016s
I20260709 13:19:31.742035 16651 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.753315 10892 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:31.708714 16694 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.754315 10199 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:31.753786 16781 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.754240 16750 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.757989 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.758584 16322 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.759441 10054 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:31.750876 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.721561 16747 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.763849 16627 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.762583 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:31.764679 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:31.775370 16458 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.776499 16558 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.781518 16654 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.820279 16750 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.821089 16654 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.798159 16613 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:31.787833 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 135362us
I20260709 13:19:31.822269 10201 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:19:31.821990 16613 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.766563 16699 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.823537 16357 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.825520 16699 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.810242 16694 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.783705 16598 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.815897 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:31.822469 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:31.714865 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:31.834177 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:31.835917 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:31.837421 16651 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.839761 10124 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:31.828722 10049 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:31.844548 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:31.848774 10124 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:19:31.850189 10205 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:31.851615 10126 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:31.852639 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:31.860329 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.864089 16753 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:31.864741 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100212us
I20260709 13:19:31.865379 16753 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.865891 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:31.856740 10052 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:31.794764 16756 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.807284 16718 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.877306 10199 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:31.867374 16349 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:31.884765 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:31.886327 16458 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:31.887398 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.887837 16087 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:31.922115 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:31.922365 16087 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:31.923938 16087 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.926141 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:31.928732 10055 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:19:31.932140 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 109883us
I20260709 13:19:31.932426 10050 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:31.933339 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:31.888979 10128 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.890849 16754 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.891198 16654 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.892091 16762 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:31.892560 16746 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.003s
I20260709 13:19:31.895529 16626 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:31.896274 16504 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.897107 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1021) took 1309 ms. Trace:
I20260709 13:19:31.959415 16771 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:19:31.960268 16504 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:31.899787 10048 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:19:31.904894 10201 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:31.962797 16504 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:31.962685 10048 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:19:31.964990 10131 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:31.906137 16756 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:31.907302 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.858742 16490 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:31.684654 16767 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:31.921625 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.959316 16626 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.960176 10899 rpcz_store.cc:276] 0709 13:19:30.588118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.589011 (+   893us) service_pool.cc:224] Handling call
0709 13:19:31.896769 (+1307758us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001e5854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:31.897012 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610277120}
I20260709 13:19:31.898023 16651 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.967062 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:31.971815 10893 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:31.968616 10047 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:31.973613 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.974696 10199 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:31.978796 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:31.966195 16770 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:31.994688 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:31.968648 16758 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:32.004395 16533 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:19:32.013818 16758 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.014123 16533 consensus_queue.cc:1243] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:32.014385 16758 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:32.014894 16533 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:32.015688 10203 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:19:32.015540 16803 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.248s	user 0.000s	sys 0.002s
I20260709 13:19:32.017177 10129 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:31.985875 16363 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:32.018213 16759 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:19:32.019484 10897 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:32.005429 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3089) took 3871 ms. Trace:
I20260709 13:19:31.985507 16557 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.007517 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.007443 16651 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:31.971087 16626 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:32.015548 16533 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:32.021699 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 222608us
I20260709 13:19:32.004443 10201 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:19:32.006600 16613 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:32.030421 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.024122 10054 rpcz_store.cc:276] 0709 13:19:28.134096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.985851 (+1851755us) service_pool.cc:224] Handling call
0709 13:19:29.986307 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.815741 (+1829434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.005087 (+189346us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400199454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:32.005354 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4076607744}
I20260709 13:19:32.045382 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:31.991451 16792 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.047186 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:32.042728 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46648.9us
I20260709 13:19:31.994089 16654 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.054648 16792 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.055797 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.056875 16654 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.057389 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:32.057739 10056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3067) took 4182 ms (client timeout 2103 ms). Trace:
W20260709 13:19:32.058200 10056 rpcz_store.cc:269] 0709 13:19:27.877071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.965585 (+ 88514us) service_pool.cc:224] Handling call
0709 13:19:32.057439 (+4091854us) spinlock_profiling.cc:243] Waited 3.64 s on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:32.057641 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8381164800}
I20260709 13:19:32.058688 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:32.060009 16792 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:32.069939 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 185425us
I20260709 13:19:32.062075 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.062611 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1020) took 1553 ms. Trace:
I20260709 13:19:32.062879 16352 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:32.039610 16694 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.064216 10128 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:32.061055 16727 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.071504 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3086) took 1961 ms. Trace:
I20260709 13:19:32.080816 10047 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:32.073127 10892 rpcz_store.cc:276] 0709 13:19:30.509558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.509851 (+   293us) service_pool.cc:224] Handling call
0709 13:19:30.510300 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.812949 (+1302649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.062309 (+249360us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540063fbd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:32.062537 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2857505536}
I20260709 13:19:32.083050 10129 rpcz_store.cc:276] 0709 13:19:30.110185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.939427 (+829242us) service_pool.cc:224] Handling call
0709 13:19:31.196820 (+257393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.017781 (+820961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.063018 (+ 45237us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400125cd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:32.071375 (+  8357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772304640}
I20260709 13:19:32.065953 16490 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:32.069200 16770 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:32.101099 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:32.091825 16696 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.005s
I20260709 13:19:32.111696 10893 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:32.064673 16357 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:19:32.092305 16781 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.118111 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.125123 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.108449 16792 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:32.110960 16753 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:32.127058 10892 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:32.113943 16756 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:19:32.133219 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:32.133520 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 147372us
I20260709 13:19:32.133945 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.134773 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:32.136161 10047 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:19:32.139725 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77787.2us
I20260709 13:19:32.141459 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3643) took 2237 ms. Trace:
I20260709 13:19:32.148285 16533 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.149610 10170 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:32.149866 16600 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:19:32.154254 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 281222us
I20260709 13:19:32.113647 16598 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.165184 10128 rpcz_store.cc:276] 0709 13:19:29.904256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.904583 (+   327us) service_pool.cc:224] Handling call
0709 13:19:29.906703 (+  2120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.080572 (+2173869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.141027 (+ 60455us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001270d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:32.141365 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4558050048}
I20260709 13:19:32.169966 10054 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:32.122430 16167 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.125452 16627 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:32.125789 16490 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.183992 10897 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:32.133318 16510 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.196362 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.196816 10201 raft_consensus.cc:2410] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:32.200217 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.210685 10047 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:32.214103 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:32.214501 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:32.216260 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.217023 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3582) took 1529 ms. Trace:
I20260709 13:19:32.217233 16728 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:32.218165 16533 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:32.218844 16504 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:32.220211 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:32.188635 16775 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:32.223798 10203 rpcz_store.cc:276] 0709 13:19:30.687857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.701941 (+ 14084us) service_pool.cc:224] Handling call
0709 13:19:32.216643 (+1514702us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540012aa54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:32.216913 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2937715072}
I20260709 13:19:32.221525 16552 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.235554 10047 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:32.241834 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:32.243225 10054 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 23: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:32.243741 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.249475 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.257581 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.257674 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:32.257009 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66872.3us
W20260709 13:19:32.269759 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.245278 16775 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:32.270498 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.271675 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.273051 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:32.273223 16746 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:32.274718 16349 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:32.275068 16694 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.256937 16728 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.275388 16349 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.276180 16349 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:32.276582 16728 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.231926 16490 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.279920 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:32.284423 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.284960 16490 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.262962 16504 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.288130 16504 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:32.254966 16762 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:32.289981 16613 raft_consensus.cc:2764] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:32.291234 16613 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:32.238735 16767 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:32.294483 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:32.308709 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:32.301930 16748 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.503s	user 0.004s	sys 0.006s
W20260709 13:19:32.303342 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83356.8us
I20260709 13:19:32.313076 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.313285 16600 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:32.314172 16600 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.304143 16651 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.315575 16651 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.317068 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.315032 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.317854 10056 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:32.318259 16651 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:32.304816 16758 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.323412 16759 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:32.323995 16759 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.325073 16759 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.328342 16718 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:32.328768 15786 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.328843 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.329676 15786 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.317204 16600 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:32.321375 16613 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.320372 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.273042 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:32.340782 10891 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:32.273643 16756 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:32.335731 10056 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:32.336352 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.202106 16167 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.234443 16625 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:19:32.353392 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:32.349681 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.205639 16510 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:32.344365 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71383.3us
I20260709 13:19:32.345649 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:32.208566 16627 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.326918 16759 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:32.355717 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:32.344020 16747 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.006s
I20260709 13:19:32.369656 10054 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:19:32.375728 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:32.380795 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60488.3us
I20260709 13:19:32.352231 16767 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:32.389823 10127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.007s	sys 0.010s
I20260709 13:19:32.385106 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:32.359505 16600 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:32.359578 16363 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:32.402067 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:32.403028 10054 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:32.402177 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:32.391364 10127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.007s	sys 0.010s
I20260709 13:19:32.392316 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.368755 16258 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:32.374722 16718 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:32.356842 16349 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.417044 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:32.440868 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80264.4us
I20260709 13:19:32.371980 16322 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:32.389801 16770 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:32.441581 10198 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:32.442588 16758 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:32.453238 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.459563 10128 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 11.
I20260709 13:19:32.453873 16613 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:32.360224 16087 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:19:32.453982 10897 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:19:32.454648 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87649.6us
W20260709 13:19:32.455319 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50250.2us
I20260709 13:19:32.393702 16600 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:32.479290 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:32.454836 16627 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:32.461742 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77657.5us
I20260709 13:19:32.482657 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.462170 15786 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:32.401598 16748 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:19:32.464871 10048 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:32.466521 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.401866 16363 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.473078 10892 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:19:32.394526 16756 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:19:32.460335 16727 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:32.401481 16767 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.492646 10201 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:19:32.493860 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.481751 16600 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.503369 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.507546 16533 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:32.507712 16087 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:32.508248 16533 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.508482 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.487735 16613 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:32.492401 16633 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:32.529062 10054 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:32.538266 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.546902 10048 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:32.480079 16770 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:32.552155 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50451.7us
I20260709 13:19:32.554646 10170 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:32.556497 10892 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:32.557730 16811 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:19:32.513150 16748 consensus_queue.cc:1243] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:32.505789 16758 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.560546 16718 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.561201 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:32.540222 16633 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:32.563855 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47534us
I20260709 13:19:32.492159 16793 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.158s
I20260709 13:19:32.574710 10128 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:32.511970 16613 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:32.579913 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 92952.2us
I20260709 13:19:32.493125 16557 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:19:32.603645 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.604707 10893 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:32.557132 16167 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:32.583298 16810 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:19:32.546372 16087 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:19:32.606856 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.546648 16756 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:19:32.630013 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:32.584072 16633 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:32.637351 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.583784 16728 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:32.653039 10130 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:32.654584 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.656847 16557 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:32.656946 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.657469 16557 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:32.657944 16557 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.577140 16533 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.660130 16533 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.661713 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.662928 16767 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:32.663139 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:32.663870 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:32.664716 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.665153 16087 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:32.665478 10125 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:32.665796 16087 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.666363 16087 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.588774 16748 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:19:32.648358 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:32.667492 16748 consensus_queue.cc:1243] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:32.668011 10129 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:32.668047 16727 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:32.668831 16727 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.669997 16727 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.663972 10123 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 15: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:32.673863 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3110) took 1089 ms. Trace:
I20260709 13:19:32.674481 10055 rpcz_store.cc:276] 0709 13:19:31.584026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.929837 (+345811us) service_pool.cc:224] Handling call
0709 13:19:31.930336 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.930934 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.930970 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.940573 (+  9603us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.176763 (+236190us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:32.176843 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.176964 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.673619 (+496655us) raft_consensus.cc:1685] finished
0709 13:19:32.673647 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.673795 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5409,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":123,"thread_start_us":222637,"threads_started":1,"wal-append.queue_time_us":460849}
I20260709 13:19:32.678149 10055 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:32.680927 16718 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:32.686095 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62321.2us
I20260709 13:19:32.687347 16746 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:32.688089 16746 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:32.689433 16746 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.690166 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3108) took 1159 ms. Trace:
I20260709 13:19:32.627445 16363 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.692931 16349 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:32.706486 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 59188.3us
W20260709 13:19:32.706872 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58338.1us
I20260709 13:19:32.708045 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:32.709494 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:32.710527 10893 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:32.690659 10051 rpcz_store.cc:276] 0709 13:19:31.530904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.892826 (+361922us) service_pool.cc:224] Handling call
0709 13:19:31.893590 (+   764us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.894360 (+   770us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:32.119749 (+225389us) log.cc:844] Serialized 14 byte log entry
0709 13:19:32.494139 (+374390us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:32.506669 (+ 12530us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.506882 (+   213us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:32.506934 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.507042 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.689844 (+182802us) raft_consensus.cc:1685] finished
0709 13:19:32.689872 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.690087 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":790,"prepare.queue_time_us":325027,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":2209,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":5186,"thread_start_us":537146,"threads_started":2,"wal-append.queue_time_us":527557}
W20260709 13:19:32.678202 16808 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.334s	user 0.005s	sys 0.000s
I20260709 13:19:32.690536 16678 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:32.753958 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46011.2us
I20260709 13:19:32.709458 16727 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:32.744064 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:32.750051 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:32.761088 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:32.748867 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 274834us
I20260709 13:19:32.767709 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:32.716531 16728 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:32.757886 16817 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:19:32.782233 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:32.775698 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:32.820217 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44640.1us
I20260709 13:19:32.827539 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:32.830513 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 86592.8us
I20260709 13:19:32.757331 16600 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:32.837486 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:32.832510 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:32.843688 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.851647 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 103701us
W20260709 13:19:32.852411 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 70331.2us
I20260709 13:19:32.829226 16167 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:32.838351 10124 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:32.845021 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:32.833577 16552 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:32.841971 16558 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:32.844143 16754 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.904s	user 0.015s	sys 0.000s
I20260709 13:19:32.840550 16748 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:32.889904 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:32.888710 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 327626us
W20260709 13:19:32.891093 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46175.1us
W20260709 13:19:32.898757 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 47404us
I20260709 13:19:32.890233 16558 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:32.910717 16558 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:32.870085 16167 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:32.902689 16822 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.135s	user 0.001s	sys 0.001s
I20260709 13:19:32.934033 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3102) took 1236 ms. Trace:
I20260709 13:19:32.922592 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:32.900028 16775 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:32.920351 16626 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.873s	user 0.018s	sys 0.000s
I20260709 13:19:32.940680 10899 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:19:32.945282 10899 pending_rounds.cc:85] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 13
I20260709 13:19:32.945627 10899 pending_rounds.cc:107] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:19:32.945871 10899 raft_consensus.cc:2889] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:32.934760 10127 rpcz_store.cc:276] 0709 13:19:31.697530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.802777 (+105247us) service_pool.cc:224] Handling call
0709 13:19:31.839322 (+ 36545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.839919 (+   597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.839950 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.840549 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.406383 (+565834us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:32.406450 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.406553 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.933673 (+527120us) raft_consensus.cc:1685] finished
0709 13:19:32.933696 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.933931 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4862,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":13776,"thread_start_us":565360,"threads_started":1,"wal-append.queue_time_us":719574}
I20260709 13:19:32.920406 16167 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:32.962509 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:32.963917 10051 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:32.966596 16828 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:32.968143 16825 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:32.968851 16825 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.969311 16825 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:32.970494 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:32.971568 10056 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:32.976289 16819 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:19:32.979626 10047 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:32.849648 16728 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:32.992332 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 138196us
W20260709 13:19:33.004900 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 172465us
I20260709 13:19:32.984259 16767 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.952661 16626 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:32.953441 16775 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:19:33.076416 16828 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.077414 16828 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.017378 16600 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.079465 16835 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:33.082131 16831 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:19:33.084638 16827 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:19:33.085263 16626 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:33.085783 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 91453.3us
W20260709 13:19:33.087189 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.089429 16770 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.099745 16598 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:33.115633 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.116565 10128 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.123374 10129 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:19:33.136500 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.137334 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.137881 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 86391.4us
I20260709 13:19:33.102771 16598 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:33.141767 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 267447us
I20260709 13:19:32.997408 16651 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:19:33.147821 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:33.148701 10126 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:33.149225 10126 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:33.153868 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.154289 10129 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [16.7-16.7]
I20260709 13:19:33.125669 16793 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.160195 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 205474us
W20260709 13:19:33.151533 16824 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:19:33.162729 10125 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:33.162914 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104991us
I20260709 13:19:33.163237 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:33.166146 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:33.164098 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.151243 16598 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:33.174198 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 251754us
I20260709 13:19:33.155393 10126 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.170439 10125 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:33.060207 16767 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.177238 10201 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:33.162155 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:33.178709 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.191939 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 299779us
I20260709 13:19:33.177175 10126 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.194746 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:33.214084 10051 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:33.215629 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.217562 10122 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.206645 10128 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:33.150144 16654 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:19:33.230451 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:33.233814 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69828.6us
I20260709 13:19:33.193657 16696 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:33.223834 16835 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.249850 16510 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.251442 16860 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:33.251611 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 738) took 1265 ms. Trace:
I20260709 13:19:33.293488 16835 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:33.293905 10899 rpcz_store.cc:276] 0709 13:19:31.996193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.016316 (+ 20123us) service_pool.cc:224] Handling call
0709 13:19:32.016748 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.946495 (+929747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.251275 (+304780us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:33.251527 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047520128}
I20260709 13:19:33.251830 16840 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:33.253743 16833 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:19:33.258078 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.258152 16792 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
W20260709 13:19:33.259899 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.299432 15786 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:33.299799 16747 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.299971 15786 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:33.300380 16747 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.301100 15786 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.301630 16849 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:33.294095 16835 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.302708 16849 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.306201 16849 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.308945 16357 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:33.311682 16654 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:33.312289 16654 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:33.313655 16654 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.315878 16654 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:33.260926 16812 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.872s	user 0.003s	sys 0.002s
I20260709 13:19:33.163477 16775 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 13:19:33.319276 16775 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:33.319768 16775 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:33.320896 16775 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.322563 16775 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:33.327028 16746 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.263397 10125 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.264101 16852 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.279050 16828 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.280637 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:33.289417 16633 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.000s
I20260709 13:19:33.235677 16718 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:33.371433 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 153159us
I20260709 13:19:33.365989 10125 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.302855 16747 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.176563 16844 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.160s
I20260709 13:19:33.239181 16678 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:33.174077 16793 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.307029 16849 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.298115 16600 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:33.372663 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.382805 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:33.361016 16861 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:19:33.339540 16824 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:33.379155 10205 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
W20260709 13:19:33.355899 16557 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.005s
W20260709 13:19:33.329818 16625 consensus_queue.cc:1175] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "db052bef7536453fa2d7f5a00e314a18" responder_term: 21 status { last_received { term: 21 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 21 index: 16 } }
W20260709 13:19:33.375298 16258 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.921s	user 0.011s	sys 0.007s
I20260709 13:19:33.387952 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:33.418000 10020 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.406313 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:33.409199 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:33.420570 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.421633 10051 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:33.415362 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.425282 10048 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:33.412729 10122 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:33.376258 16718 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:33.429329 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.435631 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.360517 16758 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:19:33.438738 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.440608 10053 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:33.431952 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2409) took 1714 ms. Trace:
I20260709 13:19:33.442553 16828 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.407991 16857 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.454474 10047 raft_consensus.cc:2393] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:33.454309 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:33.457418 10128 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:19:33.420030 16533 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.759s	user 0.010s	sys 0.009s
I20260709 13:19:33.460522 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:33.457357 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 318459us
I20260709 13:19:33.461498 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:33.462344 10054 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:33.355438 16852 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.468394 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.431510 16747 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:33.371816 16552 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:19:33.412989 16846 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:33.462788 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.478246 10131 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:33.462636 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:33.458006 16727 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.685s	user 0.015s	sys 0.000s
I20260709 13:19:33.443423 16853 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.485309 10053 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:33.427603 16557 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.441584 16718 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:33.479302 16759 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.092s	user 0.019s	sys 0.002s
I20260709 13:19:33.458437 16600 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.410404 16849 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:33.483398 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65526.8us
I20260709 13:19:33.515827 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.517306 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:33.518283 16558 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
I20260709 13:19:33.383432 16633 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.519423 16767 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.521239 16633 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:33.522269 16793 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.524497 16849 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.485387 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:33.486442 10056 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:33.527838 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.528515 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:33.529443 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.530174 10056 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:33.531282 16852 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.531823 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.532418 10125 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:33.533272 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3137) took 1232 ms. Trace:
I20260709 13:19:33.534652 10053 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:33.466152 16613 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:33.535789 16613 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:33.536716 16613 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:33.481673 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.498284 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3120) took 1730 ms. Trace:
I20260709 13:19:33.453528 16828 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.490054 10122 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:33.539958 16533 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:33.540349 16828 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.497407 16846 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:33.504635 16848 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:19:33.543949 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:33.515045 16863 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:33.547047 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3141) took 1034 ms. Trace:
I20260709 13:19:33.547504 10055 rpcz_store.cc:276] 0709 13:19:32.512671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.708671 (+196000us) service_pool.cc:224] Handling call
0709 13:19:32.745893 (+ 37222us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:32.946943 (+201050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.946990 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:32.988446 (+ 41456us) log.cc:844] Serialized 52 byte log entry
0709 13:19:33.198332 (+209886us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:33.198469 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.198746 (+   277us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.546729 (+347983us) raft_consensus.cc:1685] finished
0709 13:19:33.546756 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.546974 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10222,"raft.queue_time_us":208616,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":47149,"thread_start_us":368214,"threads_started":2,"wal-append.queue_time_us":389080}
I20260709 13:19:33.446195 10202 rpcz_store.cc:276] 0709 13:19:31.698781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.724372 (+ 25591us) service_pool.cc:224] Handling call
0709 13:19:31.756080 (+ 31708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.813754 (+ 57674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.813788 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.869855 (+ 56067us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.188142 (+318287us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:32.188208 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.188329 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.260147 (+1071818us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:33.400072 (+139925us) raft_consensus.cc:1685] finished
0709 13:19:33.400098 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.412888 (+ 12790us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7278,"raft.queue_time_us":1694,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":50712,"spinlock_wait_cycles":8984320,"thread_start_us":219777,"threads_started":1,"wal-append.queue_time_us":429476}
I20260709 13:19:33.539714 10051 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 11: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:33.514395 16167 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
W20260709 13:19:33.502306 16087 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.004s
I20260709 13:19:33.533677 10049 rpcz_store.cc:276] 0709 13:19:32.300857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.653542 (+352685us) service_pool.cc:224] Handling call
0709 13:19:32.712988 (+ 59446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:32.928281 (+215293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.928996 (+   715us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:32.985280 (+ 56284us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.338881 (+353601us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:33.338931 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.339035 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.532996 (+193961us) raft_consensus.cc:1685] finished
0709 13:19:33.533020 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.533199 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40523,"raft.queue_time_us":238463,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":29247,"thread_start_us":392277,"threads_started":2,"wal-append.queue_time_us":438452}
I20260709 13:19:33.484385 10204 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:33.542598 10056 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:33.550031 10050 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:33.504838 15786 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.551216 10201 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:33.554153 10054 raft_consensus.cc:2410] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:19:33.552102 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89540.1us
W20260709 13:19:33.498442 16825 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.002s
I20260709 13:19:33.558248 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.539443 10052 rpcz_store.cc:276] 0709 13:19:31.767986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.959855 (+191869us) service_pool.cc:224] Handling call
0709 13:19:31.960277 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:32.000743 (+ 40466us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:32.337343 (+336600us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.007169 (+669826us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.095589 (+ 88420us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.096479 (+   890us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.096732 (+   253us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:33.096785 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.096915 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.498041 (+401126us) raft_consensus.cc:1685] finished
0709 13:19:33.498068 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.498217 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162615,"prepare.queue_time_us":216992,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":85,"raft.queue_time_us":303102,"raft.run_cpu_time_us":565,"raft.run_wall_time_us":24464,"thread_start_us":888526,"threads_started":3,"wal-append.queue_time_us":595986}
I20260709 13:19:33.521075 16557 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.560596 16087 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.560811 16772 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:33.580062 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 94828.9us
I20260709 13:19:33.581869 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.583040 10129 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:33.571261 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.563982 10055 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:33.583705 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 65023.8us
W20260709 13:19:33.576592 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78934.8us
I20260709 13:19:33.567103 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.578445 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:33.561530 16167 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.542690 16846 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:33.518793 16558 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.587191 10123 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:19:33.592033 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 132120us
W20260709 13:19:33.598279 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48351us
I20260709 13:19:33.580364 16651 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:33.469144 16857 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:33.603682 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.609385 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:33.603955 10122 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:33.540598 16533 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.579289 16600 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:33.576459 16844 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:33.540164 16835 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.591223 16866 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:33.490552 16853 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.625226 10899 raft_consensus.cc:2368] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21.
I20260709 13:19:33.633730 10048 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:33.628890 10126 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:19:33.607645 16753 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:33.608889 16651 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.659518 16753 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:33.624423 16747 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:33.639991 16860 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:33.660298 16835 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:33.643491 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.660893 16860 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:33.660868 16753 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.662425 16860 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.646049 16872 raft_consensus.cc:2764] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:19:33.594548 16772 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.653353 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:33.589700 16598 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.657383 16533 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.659588 16651 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.581387 16825 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.662891 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:33.644409 16510 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:33.664034 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90911.5us
I20260709 13:19:33.638509 10126 pending_rounds.cc:85] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:19:33.626821 16552 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:33.666409 16728 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:19:33.683542 16167 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.662950 16857 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.630920 16866 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:33.702025 10126 pending_rounds.cc:107] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.10
W20260709 13:19:33.689894 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83432.7us
I20260709 13:19:33.703157 10126 raft_consensus.cc:2889] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:33.705796 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.672711 16600 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:19:33.707024 16747 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:33.708686 16868 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:19:33.710575 16087 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.711464 16837 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:33.716293 16840 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:33.716464 16558 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:33.719220 16853 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:33.729336 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66515.7us
I20260709 13:19:33.735976 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:33.745060 16857 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:33.746996 16840 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:33.748183  9954 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 18 to 19, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 19 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:33.751529 10049 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:33.726737 16654 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:33.759558 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.775449 10055 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:33.776686 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.777405 10055 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:33.760412 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.742677 16837 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:33.778393 10054 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:33.763048 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.779646 10051 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:33.710613 16598 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:33.767817 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:33.781894 10050 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:33.762923 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 57283.9us
I20260709 13:19:33.789264 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:33.772997 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49872.4us
I20260709 13:19:33.791105 16825 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:33.741997 16849 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:33.792076 16849 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.792482 16849 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.702615 16866 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.796164 16876 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:33.797107 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:33.798200 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60461.6us
I20260709 13:19:33.799606 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:33.801757 16598 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:33.802335 16363 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:33.808019 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.808597 16775 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:33.811545 16837 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:33.812814 10204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3589) took 3003 ms (client timeout 2633 ms). Trace:
I20260709 13:19:33.813582 16876 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.814165 16876 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.814188 10200 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:33.820815 16857 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:33.796285 16866 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.822576 16558 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:33.823407 16558 consensus_queue.cc:1243] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:33.827788 16876 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:33.813369 10204 rpcz_store.cc:269] 0709 13:19:30.809088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.812049 (+  2961us) service_pool.cc:224] Handling call
0709 13:19:33.812539 (+3000490us) spinlock_profiling.cc:243] Waited 2.54 s on lock 0x7b54001bf654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:33.812707 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5839662976}
I20260709 13:19:33.810375 16775 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.695204 16718 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.843093 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.844630 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.845563 10899 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:33.848448 10169 leader_election.cc:400] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21.
I20260709 13:19:33.849035 10169 leader_election.cc:403] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:19:33.851739 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:33.852790 10893 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:33.833864 16654 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:33.838429 10095 connection.cc:256] RPC call timeout handler was delayed by -1.688880023000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 13:19:33.861317 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1623800230001962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
I20260709 13:19:33.852514 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.866572 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.865971 16860 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.869880 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:33.877960 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.860080 16087 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:33.863569 16876 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:33.863873 16654 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:33.882063 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 246035us
I20260709 13:19:33.884784 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:33.886183 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.843086 16853 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:33.898733 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.899940 10126 raft_consensus.cc:2368] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:33.855432 16352 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:19:33.905829 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.907737 10130 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 11: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:33.872074 16860 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.911096 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.912194 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:33.914466 10128 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:33.867669 16552 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:33.891785 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.923017 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.882553 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:33.890111 10050 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:33.880843 16826 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:33.913297 10054 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:33.925521 10170 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:33.880410 16087 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.875560 16746 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:33.920930 10122 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:33.928171 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.892458 16876 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:33.856802 16558 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:33.868705 16600 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:19:33.923939 16860 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:33.943815 16876 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:33.938730 16357 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:33.959022 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.927094 16552 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:34.003436 16552 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.927736 16826 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.006083 16826 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:34.007252 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.902030 16828 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:33.953594 16613 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:33.943850 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:33.936779 16746 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.937199 16087 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.979702 16718 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:34.015856 16872 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:34.018481 10054 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:34.051614 16872 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:34.020957 10052 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:34.052770 16872 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.054544 16872 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:34.015050 16828 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.020427 16837 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:34.028072 16772 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.062260 16087 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:33.987787 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.067936 10053 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:33.860543 16836 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:33.994699 16633 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:33.972767 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2411) took 2198 ms. Trace:
I20260709 13:19:34.069203 16633 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:34.069392 10200 rpcz_store.cc:276] 0709 13:19:31.774014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.806915 (+ 32901us) service_pool.cc:224] Handling call
0709 13:19:33.813727 (+2006812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.819536 (+  5809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.972381 (+152845us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b54001bf650. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:33.972637 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4592510848}
I20260709 13:19:33.980983  9954 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:33.998445 16651 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.070399 16633 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.071902 16633 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:34.072382 16651 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.074759 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.079707 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.080873 10122 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:34.082317 10131 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 11.
I20260709 13:19:34.033964 10127 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:34.035140 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.035604 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.036087 10124 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:34.035202 10125 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:34.039794 16873 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:34.036744 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.094599 10128 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:34.095425 10123 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:34.025580 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:33.944263 16625 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.104496 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.041852 16748 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.049484 16746 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:34.108337 16087 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.111166 10128 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:19:34.024174 16598 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:34.117668 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:34.119952 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 214480us
I20260709 13:19:34.029426 16849 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.094694 16873 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:34.124378 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.128525 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:34.129890 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 79207.7us
I20260709 13:19:34.130066 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:34.130553 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 110920us
I20260709 13:19:34.047899 16850 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:19:34.134311 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.133703 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.038873 16654 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.146845 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:34.146642 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 121145us
I20260709 13:19:34.114781 16854 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:34.141378 16866 raft_consensus.cc:2764] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:34.142470 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.142561 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:34.143608 10052 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:34.142603 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:34.166796 10054 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.168023 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:34.145081 10197 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:34.144943 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.094013 16840 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.170801 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.132750 10122 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:34.153657 16873 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.197170 16873 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:34.198604 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.136620 16357 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:34.115943 16625 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.200593 16357 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.201781 16625 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.163388 16087 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.202939 16087 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.205227 16087 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.206375 16087 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:34.207333 16756 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:34.213339 16836 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21.
I20260709 13:19:34.214651 10201 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:34.149505 10196 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:34.134924 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.125754 16613 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:34.176239 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:34.178251 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:34.218916 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:34.183709 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:34.184893 10126 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:19:34.186182 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:34.219967 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:34.220679 10893 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:34.122212 16748 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.182766 16718 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:34.187863 16866 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:34.188673 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.191291 10056 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:34.174665 16678 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:34.159528 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.224614 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.225792 10054 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:34.226366 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:34.237812 10892 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:34.238149 16678 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.204850 16552 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.239723 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.240430 10127 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.240819 16678 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:34.240985 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:34.241639 10198 raft_consensus.cc:2410] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:34.243160 10128 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:34.227865 16651 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:34.223948 16613 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.189940 10197 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:19:34.175405 16854 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.236560 10202 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:34.218832 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.244024 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:34.236974 10131 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:34.247862 10196 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:34.247880 10052 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:34.260236 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:34.261152 10893 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
W20260709 13:19:34.261969 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.265986 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.266752 10131 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18. Re-sending same reply.
I20260709 13:19:34.260280 16838 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.265043 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.271093 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.246644 16651 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:34.300424 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 59522.7us
I20260709 13:19:34.303638 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.246147 16775 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.246889 16613 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:34.299170 10128 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:34.318857 10204 raft_consensus.cc:2368] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:34.257373 16718 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:34.324831 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.330657 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.251066 16882 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:34.333415 10020 connection.cc:256] RPC call timeout handler was delayed by -1.0243388010001127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
I20260709 13:19:34.336690 10131 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:34.334376 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.342926 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:34.343068 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.344061 10891 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 15: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:34.345022 10203 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:34.266443 16866 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:19:34.308200 16651 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.364487 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.256960 16884 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:19:34.368235 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.329514 16718 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:34.325711 16891 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:34.295212 16838 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.383581 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:34.359386 16552 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.393875 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.406711 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.415474 16598 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:34.375195 16718 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:34.417765 16837 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:34.418290 16837 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:34.427798 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.428738 10201 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:34.433545 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 99289.6us
I20260709 13:19:34.376690 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.388850 16678 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:34.441370 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.434809 16866 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:19:34.443454 16748 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.443769 16840 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.442407 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.459848 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.460070 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.426203 16654 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.430886 16552 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.464773 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:34.480441 10128 pending_rounds.cc:85] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:19:34.481376 10128 pending_rounds.cc:107] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:34.478276 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.482249 10128 raft_consensus.cc:2889] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:34.488973 16826 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.485317 16842 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:34.492502 16598 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:34.493407 16598 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:34.493079 16853 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:34.495667 16826 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:34.499310 16895 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.079s	user 0.001s	sys 0.001s
I20260709 13:19:34.462037 16840 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:34.463955 16558 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:34.475888 16748 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.441371 10204 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:34.522568 10127 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:34.462296 16678 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.442804 16837 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.510046 16654 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:34.567176 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 234401us
I20260709 13:19:34.509153 16558 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.477288 16552 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.568377 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.578760 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.558699 16872 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:34.519732 16840 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:34.578574 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105889us
W20260709 13:19:34.578456 16860 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.014s
I20260709 13:19:34.569582 16842 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:34.637030 16842 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.643260 16747 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.643997 16747 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.604064 16678 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.587509 16872 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:34.661617 16678 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:34.662001 16872 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.663800 16872 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:34.594763 16854 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:34.577966 16558 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:34.614738 16891 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:34.620537 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.622534 16837 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:34.558281 16748 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.501921 16598 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.623076 16849 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:34.604090 16876 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 13:19:34.624022 10205 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:34.624688 16552 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:34.626168 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.629027 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.627911 10196 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:34.632391 10052 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:34.633610 16838 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.635295 16860 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.651883 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:34.644719 16633 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.002s
I20260709 13:19:34.654971  9954 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 15, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 15 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:34.643252 16902 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:34.662312 16678 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.757633  9954 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:34.759382 16678 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.608237 16840 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.759712  9954 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 18 to 19, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:19:34.760663 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.761317  9954 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:34.667022 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 88402.7us
I20260709 13:19:34.765053 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:34.671445 10197 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:34.768826 10197 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:19:34.761334 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.770745 10197 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:34.642720 16651 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:34.681280 10204 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:34.692950 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:34.700496 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:34.702271 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:34.754957 16747 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.706199 10050 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:34.707332 10170 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:34.781052 16748 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:34.711756 10130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.002s	sys 0.015s
W20260709 13:19:34.781819 10130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.003s	sys 0.016s
I20260709 13:19:34.781901 16860 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:34.705502 16746 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.006s
I20260709 13:19:34.782827 16746 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.784346 16613 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.785480 16746 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:34.735988 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.668296 16836 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.743021 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.749148 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:34.739894 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.740934 10203 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:34.751434 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:34.710968 16891 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:34.754189 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:34.771277 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.673828 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 181932us
I20260709 13:19:34.772564 16882 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:34.773272 16552 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.710757 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.802160 10052 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:34.821264 10051 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:34.809497 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:34.823961 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.824826 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 81920.9us
I20260709 13:19:34.778918 10202 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:34.781172 10199 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:34.787582 10053 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:34.787016 10201 raft_consensus.cc:2368] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:34.774816 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.824167 10020 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.774724  9954 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 13 to 16, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 16 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:34.834118 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.821230 10047 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
W20260709 13:19:34.830441 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.018s	sys 0.064s
W20260709 13:19:34.835604 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.018s	sys 0.064s
W20260709 13:19:34.835961 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 571975us
W20260709 13:19:34.830915 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 123719us
W20260709 13:19:34.837291 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 62619.5us
I20260709 13:19:34.786876 16613 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:34.845841 10205 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [15.13-17.14]
I20260709 13:19:34.848426  9954 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:34.853721 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.841117 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:34.853106 10122 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:34.838685 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:34.779280 10892 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:34.856493 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:34.857244 10131 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:34.827221 16758 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.830035 16912 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:34.827101 16882 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:34.738405 16876 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.839102 16854 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:34.900744 10168 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:34.898418 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.910902 10131 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:34.854665 16633 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:34.838866 16908 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:34.840415 16718 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:34.896641 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 62227.8us
I20260709 13:19:34.905483 10202 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:19:34.907702 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66676.7us
I20260709 13:19:34.908164 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:34.907912 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 69317.2us
I20260709 13:19:34.929278 10198 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:34.946100 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.945463 10122 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:34.946538 10020 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:34.946905 10130 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:34.894717 16772 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:19:34.949474 16884 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:34.949851 16772 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:34.950285 16772 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:34.853044 16905 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:34.952014 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64294.5us
I20260709 13:19:34.953307 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.955082 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.956133 10051 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:34.905395 16882 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:34.951313 16772 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.958329 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:34.861178 16678 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:34.905435 16900 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.832157 16836 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:34.917910  9948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 207) took 61099 ms (client timeout 15000 ms). Trace:
I20260709 13:19:34.944664 16908 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:34.922210 16910 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.218s	user 0.000s	sys 0.003s
I20260709 13:19:35.002135 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:35.059679 10891 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:35.008236 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:35.060674 10124 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.007774 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:34.967978 10892 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:34.959990 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:35.062237 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102352us
I20260709 13:19:34.854836 16552 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.937552 16654 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:34.945653 16633 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:35.028977 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84850.1us
I20260709 13:19:34.913070 16912 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:34.935989 16651 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.048002 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:35.048768 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:35.011314 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53083.7us
I20260709 13:19:35.057044 10196 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:34.958670 16759 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:35.064664 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.084735 10127 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:35.065008 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.065932 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:34.911605 16510 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:34.908528 16758 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:34.959193 16916 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:35.091621 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.091981 16916 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:35.092643 16916 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.092762 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.093626 10052 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:34.929435 16876 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:35.067199 16772 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:35.087177 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.073915 16898 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:35.075489 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.107928 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.109400 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.076393 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.002183 16884 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:35.113144 16884 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.028005 16854 raft_consensus.cc:697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:35.115871 16891 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.116070 16854 consensus_queue.cc:237] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.079360 10201 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:35.089962 16552 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:35.090904 10129 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 22: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:35.090812 16510 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:35.120746 16510 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:35.121323 16510 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:35.121855 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.070080 10056 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.124444 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.060335 16746 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.140139 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53125.3us
I20260709 13:19:35.140650 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.142124 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.067473 16905 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:35.072369 16654 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.078531 16633 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.059062 16908 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:35.147306 16087 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:19:35.151865 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46317.6us
I20260709 13:19:35.153009 16087 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:35.155377 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.156142 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.157289 10052 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:35.158331 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.153362 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:35.161980 10049 raft_consensus.cc:2393] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 23: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.054539 16900 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.070107 16696 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:19:35.171474 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.144325 16759 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:35.170460 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.186156 10050 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:19:35.186362 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.181840 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:35.187992 10125 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:35.175865 10053 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:35.155468 16654 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.156625 16838 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:35.189244 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.194139 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.198055 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.202157 10126 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:35.151049 16363 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:35.204020 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.147095 16746 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:35.205776 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 404537us
I20260709 13:19:35.206909 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.219151 10056 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.180855 16625 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:35.202203 10053 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.202795 10049 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:35.219914 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.190977 16884 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.156117 16905 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.198976 16753 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.244484 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:35.245096 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.245527 10126 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:35.245381 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:35.230551 10019 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.726s (SENT)
I20260709 13:19:35.246843 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.196722 16838 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.195343 16850 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:35.233233 10052 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.220165 16625 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.255304 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.257735 10049 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:35.258272 10049 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:35.233599 16891 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:35.265408 10129 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:35.265367 10125 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:35.253794 16838 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:35.281303 16891 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:35.282176 16891 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.257963 16753 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.284549 16891 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:35.291643 10049 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.293504 10049 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:35.260725 16837 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.000s
I20260709 13:19:35.295212 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:35.262542 16759 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:35.304667 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.311869 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.312558 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.321337 10199 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:35.326236 10124 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:35.328347 10052 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:35.329428 10196 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:35.329797 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.331445 10052 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:35.253176 16850 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.332679 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.335676 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.328847 16837 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:35.329207 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:35.282188 16558 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 13:19:35.275238 16758 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:35.356510 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 164639us
I20260709 13:19:35.359253 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:35.335676 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:35.362174 10891 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:35.342664 16363 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.365765 16363 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.335765 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:35.341893 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.259261 16849 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:35.257331 16625 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.346843 16718 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.330106 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.336197 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.353041 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.354024 10126 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:35.198887 16912 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.337096 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:35.366066 16919 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:19:35.371876 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 141382us
I20260709 13:19:35.341071 10020 leader_election.cc:400] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:35.383399 10020 leader_election.cc:403] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.385773 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:35.368759 10049 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:35.357690 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:35.361338 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:35.373257 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 370277us
I20260709 13:19:35.392007 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:35.392616 10129 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:35.377825 10125 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:19:35.379159 10201 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:19:35.270224 16598 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.352365 16850 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:35.335777 16748 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:35.286964 16753 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:35.377887 10196 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:35.402997 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.404594 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.405369 10127 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:35.368680 10056 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:35.394662 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:35.366144 16826 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:35.396303 16651 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:35.401311 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.416576 10052 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.409412 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.409909 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:35.373634 16696 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.411594 10129 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 13:19:35.413684 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52443.3us
W20260709 13:19:35.415015 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 89705.2us
W20260709 13:19:35.466423 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56603.8us
I20260709 13:19:35.467768 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.468765 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:35.470851 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:35.476547 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:35.480286 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.368999 16558 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.371599 16836 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.433255 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.487787 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.402983 16842 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:35.431468 16718 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:35.482026 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:35.512056 16914 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.389s	user 0.004s	sys 0.000s
I20260709 13:19:35.513659 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3620) took 1072 ms. Trace:
I20260709 13:19:35.415884 16837 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:35.416934 10124 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:35.410756 16826 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:35.489190 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.526157 10052 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:35.527268 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.528122 10052 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:35.438871 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.493539 10200 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:35.472604 16727 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:35.498890 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.501667 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:35.501778 16558 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.478343 16891 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.481968 16884 raft_consensus.cc:2764] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:19:35.545168 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.502138 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.501518 16651 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.504441 16748 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.533718 16087 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:35.521294 16718 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.534938 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.536232 16826 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:35.536588 16928 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:35.402261 16866 consensus_queue.cc:1059] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:19:35.537438 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3618) took 1154 ms. Trace:
I20260709 13:19:35.483027 16902 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:35.502213 16598 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:35.514366 10203 rpcz_store.cc:276] 0709 13:19:34.441388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.844145 (+402757us) service_pool.cc:224] Handling call
0709 13:19:34.888207 (+ 44062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.940556 (+ 52349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.940591 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.941111 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:19:35.257799 (+316688us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:35.257851 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.257939 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.513319 (+255380us) raft_consensus.cc:1685] finished
0709 13:19:35.513355 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.513567 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51884,"raft.queue_time_us":52064,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":316200,"threads_started":1,"wal-append.queue_time_us":455294}
I20260709 13:19:35.587334 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:35.587759 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 257711us
I20260709 13:19:35.502594 10050 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.537343 16905 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.546999 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.548048 10049 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.534662 10197 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:35.523352 16900 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:35.564706 16718 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.548447 16906 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.557006 16748 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.579684 10204 rpcz_store.cc:276] 0709 13:19:34.382663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.777803 (+395140us) service_pool.cc:224] Handling call
0709 13:19:34.778261 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:34.811692 (+ 33431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.811741 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:34.830062 (+ 18321us) log.cc:844] Serialized 52 byte log entry
0709 13:19:35.192878 (+362816us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:35.192925 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.193030 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.537168 (+344138us) raft_consensus.cc:1685] finished
0709 13:19:35.537190 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.537361 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45044,"raft.queue_time_us":33776,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":12959,"thread_start_us":293904,"threads_started":1,"wal-append.queue_time_us":450518}
W20260709 13:19:35.592567 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 97722.8us
I20260709 13:19:35.595489 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.605664 10052 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:35.606986 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.595693 10056 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.608579 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.544051 16836 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:35.565352 16753 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:35.622233 16872 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.958s	user 0.012s	sys 0.002s
I20260709 13:19:35.621889 16633 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.623278 10123 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:19:35.623697 10123 pending_rounds.cc:85] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 10
I20260709 13:19:35.623960 10123 pending_rounds.cc:107] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:19:35.624226 10123 raft_consensus.cc:2889] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:35.598867 16746 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:35.622246 16753 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:35.626830 16758 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:35.627300 16758 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.627763 16753 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:35.629781 16917 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.504s	user 0.003s	sys 0.000s
I20260709 13:19:35.601361 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:35.050642  9948 rpcz_store.cc:269] 0709 13:18:33.817968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.818383 (+   415us) service_pool.cc:224] Handling call
0709 13:19:34.917611 (+61099228us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:22.176626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:22.235943 (+ 59317us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:22.236040 (+    97us) write_op.cc:435] Acquired schema lock
0709 13:19:22.236093 (+    53us) tablet.cc:662] Decoding operations
0709 13:19:22.237839 (+  1746us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:22.238001 (+   162us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:22.238034 (+    33us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:19:22.312498 (+ 74464us) tablet.cc:701] Row locks acquired
0709 13:19:22.312549 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:19:22.312781 (+   232us) write_op.cc:270] Start()
0709 13:19:22.312965 (+   184us) write_op.cc:276] Timestamp: P: 1783603162312708 usec, L: 0
0709 13:19:22.313000 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:19:22.313915 (+   915us) log.cc:844] Serialized 5154 byte log entry
0709 13:19:22.937156 (+623241us) op_driver.cc:464] REPLICATION: finished
0709 13:19:23.195870 (+258714us) write_op.cc:301] APPLY: starting
0709 13:19:23.209321 (+ 13451us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:23.916936 (+707615us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:23.917100 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:24.287529 (+370429us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:24.287943 (+   414us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 13:19:24.288008 (+    65us) write_op.cc:312] APPLY: finished
0709 13:19:24.311253 (+ 23245us) log.cc:844] Serialized 121 byte log entry
0709 13:19:24.520247 (+208994us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:24.550030 (+ 29783us) write_op.cc:454] Released schema lock
0709 13:19:24.572678 (+ 22648us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:24.651373 (+ 78695us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:24.651425 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":65157,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":182598,"apply.run_cpu_time_us":100587,"apply.run_wall_time_us":1440484,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58363,"num_ops":13,"prepare.queue_time_us":507068,"prepare.run_cpu_time_us":21610,"prepare.run_wall_time_us":315479,"raft.queue_time_us":198062,"raft.run_cpu_time_us":13097,"raft.run_wall_time_us":111557,"replication_time_us":623848,"thread_start_us":869701,"threads_started":5,"wal-append.queue_time_us":539303,"wal-append.run_cpu_time_us":46263,"wal-append.run_wall_time_us":2619326}]]}
I20260709 13:19:35.572108 16842 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:19:35.633850 16510 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.001s
I20260709 13:19:35.560374 16363 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:35.627668 16758 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.589695 16852 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:35.585652 16902 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:35.598268 16866 consensus_queue.cc:1243] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:19:35.615221 16911 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.931s	user 0.003s	sys 0.003s
I20260709 13:19:35.602237 16891 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:19:35.621207 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.634447 16510 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.653213 10056 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.626323 16836 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:35.600872 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.609630 10052 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:35.654774 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.633948 10049 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:35.656531 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.657460 10049 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:35.643750 10053 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:35.600345 16884 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:35.646112 10202 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:35.605430 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.649998 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:35.662954 10052 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:19:35.672272 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:35.673171 10052 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.658684 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:35.674826 10049 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:19:35.599375 10865 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.665838 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3616) took 1296 ms. Trace:
I20260709 13:19:35.675571 10200 rpcz_store.cc:276] 0709 13:19:34.369571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.692639 (+323068us) service_pool.cc:224] Handling call
0709 13:19:35.665566 (+972927us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b540006ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:35.665738 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1203777280}
I20260709 13:19:35.666096 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:35.677906 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72579.4us
W20260709 13:19:35.667977 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67210.7us
I20260709 13:19:35.679507 10019 leader_election.cc:400] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:35.679874 10019 leader_election.cc:403] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:35.681164 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.683084 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:35.685084 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 327452us
I20260709 13:19:35.585042 16928 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.686857 10130 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:35.652137 16902 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:35.687978 10056 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:35.693138 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:35.677861 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78601.4us
I20260709 13:19:35.687312 10170 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.694941 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:35.696554 10124 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:35.697229 10865 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:35.702491 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.702881 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.704262 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.705432 10124 raft_consensus.cc:2393] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:35.689512 16916 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.596s	user 0.014s	sys 0.000s
W20260709 13:19:35.714710 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 213136us
I20260709 13:19:35.715941 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:35.717105 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:35.706604 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:35.707370 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:35.661851 16852 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:35.719425 10127 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:35.725353 10049 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:19:35.685242 16866 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:35.731561 10052 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:35.687407 16598 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:35.694082 16891 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:35.611263 16837 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.692317 16746 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.748804 10056 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:35.743634 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.749778 10052 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:19:35.744629 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.697072 16363 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:35.780555 10050 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:19:35.786896 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49051.9us
I20260709 13:19:35.790697 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:35.792618 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113264us
I20260709 13:19:35.706422 16931 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:35.801333 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:35.685241 16884 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.816543 10196 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:35.742965 16882 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.005s
I20260709 13:19:35.818459 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.740592 16866 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:35.738351 16558 raft_consensus.cc:2764] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:35.738580 16934 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:35.750809 16633 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:35.837512 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 136943us
I20260709 13:19:35.752702 16363 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:35.842859 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 155549us
I20260709 13:19:35.813548 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:35.838994 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48441.7us
I20260709 13:19:35.843050 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.854135 10893 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:19:35.855278 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 147991us
I20260709 13:19:35.811726 16891 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:35.874235 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.828504 16842 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:35.817768 16872 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.873845 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3682) took 1633 ms. Trace:
I20260709 13:19:35.738272 16916 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:35.890827 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77394.3us
I20260709 13:19:35.841465 16792 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:35.893610 10128 rpcz_store.cc:276] 0709 13:19:34.240643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.475974 (+235331us) service_pool.cc:224] Handling call
0709 13:19:34.480058 (+  4084us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:34.487811 (+  7753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.487868 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:34.510955 (+ 23087us) log.cc:844] Serialized 52 byte log entry
0709 13:19:34.683914 (+172959us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:34.683970 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.684073 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.432019 (+747946us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:35.840814 (+408795us) raft_consensus.cc:1685] finished
0709 13:19:35.840845 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.873774 (+ 32929us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13125,"raft.queue_time_us":80739,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":13013,"spinlock_wait_cycles":61521280,"thread_start_us":85372,"threads_started":1,"wal-append.queue_time_us":107391}
I20260709 13:19:35.901674 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:35.893607 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:35.840687 16900 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:35.905189 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 208026us
I20260709 13:19:35.824455 16906 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:19:35.833457 16882 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:35.908648 10202 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:35.840447 16905 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.844795 16363 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:35.903986 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60064.3us
I20260709 13:19:35.826042 16746 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.794497 16942 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.142s
I20260709 13:19:35.927982 10051 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:35.891571 16934 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:35.890304 16727 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:35.906992 16792 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:35.927181 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 206774us
I20260709 13:19:35.896574 16552 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.912976 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.933784 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:35.836858 16943 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:19:35.824508 16836 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.890405 16912 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:19:35.933655 16625 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.003s
W20260709 13:19:35.964265 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106382us
I20260709 13:19:35.953269 10122 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:35.966085 10203 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:36.001363 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:36.007424 10897 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 15: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:35.903582 16872 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.939493 16678 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:36.019282 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:36.021221 16936 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:19:36.022604 16916 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:36.023308 16942 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:19:36.023970 16939 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:36.024133 16928 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.028542 16882 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:36.028882 16937 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:19:35.894570 16852 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.029174 16957 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:36.030138 16908 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.866s	user 0.012s	sys 0.001s
I20260709 13:19:36.030644 16908 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.031744 16510 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:35.946164 16934 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.034991 10054 raft_consensus.cc:2393] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 16: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:36.039398 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.036926 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3621) took 1554 ms. Trace:
I20260709 13:19:35.959640 16792 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:35.905821 16946 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:19:36.053087 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:36.043426 16934 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:35.932945 16905 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.056144 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130690us
I20260709 13:19:36.013195 16860 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:36.008491 16900 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.040413 10205 rpcz_store.cc:276] 0709 13:19:34.468980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.850656 (+381676us) service_pool.cc:224] Handling call
0709 13:19:34.898496 (+ 47840us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:34.942177 (+ 43681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.942214 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:34.942852 (+   638us) log.cc:844] Serialized 52 byte log entry
0709 13:19:35.298449 (+355597us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18003d4aa0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:35.561339 (+262890us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:35.561402 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.561517 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.022691 (+461174us) raft_consensus.cc:1685] finished
0709 13:19:36.022723 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.023016 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33281,"raft.queue_time_us":34481,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":141,"spinlock_wait_cycles":27244416,"thread_start_us":262669,"threads_started":1,"wal-append.queue_time_us":750297}
I20260709 13:19:36.072736 16842 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.039639 16912 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:36.030138 16598 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.967619 16552 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:35.960568 16866 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.014931 16625 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.073203 16931 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:36.120040 16957 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.126284 16900 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.127255 16840 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:36.074400 16633 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:36.111570 16654 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.894s	user 0.011s	sys 0.002s
I20260709 13:19:36.159090 16842 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.159881 16654 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.161010 16905 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.162302 16654 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.013557 16891 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.136767 16952 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:19:36.163928 16952 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:36.128237 16943 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:36.164319 16850 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 13:19:36.164710 16943 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:36.164950 16850 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.156414 16840 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.121771 16860 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:36.073820 16964 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:19:36.154868 16958 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:19:36.138048 16959 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:19:36.164361 16952 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:36.166483 10131 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:36.187726 16931 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.189036 16952 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.190539 16952 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:36.191601 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 279301us
W20260709 13:19:36.192119 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 442277us
I20260709 13:19:36.194252 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.155002 16866 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:36.155480 16955 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:36.179870 16949 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:19:36.195636 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:36.186971 16696 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.721s	user 0.003s	sys 0.011s
W20260709 13:19:36.203577 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63515.2us
I20260709 13:19:36.144574 16912 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:19:36.176836 16884 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.228281 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:36.234241 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:36.233451 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:36.235666 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 219219us
I20260709 13:19:36.212635 16908 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:36.211987 16954 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:36.223456 16748 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.003s
I20260709 13:19:36.223206 16696 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:36.227308 16600 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.236145 16842 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:36.249413 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 147928us
W20260709 13:19:36.276520 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:36.315435 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 82090.6us
I20260709 13:19:36.316884 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:36.243906 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.324815 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:36.326937 10020 leader_election.cc:400] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:36.327049 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.327850 10020 leader_election.cc:403] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:36.328459 10205 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 14: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:36.238941 16942 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.245987 16958 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:36.156940 16957 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.325711 16947 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.281s	user 0.000s	sys 0.002s
I20260709 13:19:36.165771 16943 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.340167 16838 consensus_meta.cc:287] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.062s	user 0.014s	sys 0.001s
I20260709 13:19:36.362056 16838 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.368058 16950 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.401s	user 0.004s	sys 0.000s
I20260709 13:19:36.325762 16964 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:36.340313 16633 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:36.343271 10891 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:36.252758 16598 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:36.356959 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.347401 16942 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.326843 16600 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.354707 16840 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.257094 16748 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.249400 16970 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:36.380549 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3179) took 1015 ms. Trace:
W20260709 13:19:36.336838 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88843.2us
I20260709 13:19:36.340093 16967 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:19:36.382651 10199 raft_consensus.cc:2368] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:36.358052 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:36.246429 16960 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.152s
I20260709 13:19:36.219278 16955 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.426335 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.344683 16958 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.406993 10129 rpcz_store.cc:276] 0709 13:19:35.365118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.495994 (+130876us) service_pool.cc:224] Handling call
0709 13:19:35.520649 (+ 24655us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.530427 (+  9778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.530481 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:35.546164 (+ 15683us) log.cc:844] Serialized 26 byte log entry
0709 13:19:35.750425 (+204261us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:35.750478 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.750558 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.182737 (+432179us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:36.380175 (+197438us) raft_consensus.cc:1685] finished
0709 13:19:36.380200 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.380466 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":16049,"spinlock_wait_cycles":73417600,"thread_start_us":155444,"threads_started":1,"wal-append.queue_time_us":326424}
I20260709 13:19:36.245944 16860 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.387404 16956 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:19:36.408674 10019 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.412078 10050 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:36.344854 16842 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.382172 16625 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.447817 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.354103 16654 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.412303 10126 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:36.412686 10202 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:36.382326 16838 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.452651 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:36.382326 16900 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.344099 16898 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.505887 10056 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:36.393162 16598 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.507507 16598 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:36.349326 16908 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
W20260709 13:19:36.495875 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.047s (SENT)
I20260709 13:19:36.444495 16840 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.443701 16884 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.455524 16958 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:36.493924 10205 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:36.399816 16633 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.527295 10056 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:36.501164 16357 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:36.571183 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:36.558578 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 231930us
I20260709 13:19:36.520768 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:36.573590 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77743.5us
I20260709 13:19:36.481570 16957 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.506196 16898 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.563851 16826 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.978s	user 0.015s	sys 0.002s
I20260709 13:19:36.558997 16970 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:36.574839 16696 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:36.588142 10020 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:36.569998 16884 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.608686 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.587383 16957 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:36.610863 10020 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:36.590660 16552 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.601423 16960 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:36.614576 10865 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.564164 16934 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.579991 16357 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.603602 16964 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.625898 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:36.633175 10897 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:36.640727 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:36.641537 16951 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.435s	user 0.003s	sys 0.000s
I20260709 13:19:36.642002 10056 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:19:36.643083 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:36.643446 16966 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.644239 16966 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.647223 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:36.595054 16826 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:36.652235 16981 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:19:36.654531 16982 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:36.641997 16651 raft_consensus.cc:3055] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:36.635473 16973 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:36.612664 16957 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.625274 16696 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:36.617748 16960 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.631812 16985 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:36.679463 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 270836us
I20260709 13:19:36.680184 16942 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.647233 16962 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:36.683794 10200 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:36.680435 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:36.680945 16758 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.026s	user 0.014s	sys 0.003s
I20260709 13:19:36.687345 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:36.639963 16987 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:36.696831 10897 raft_consensus.cc:2368] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
W20260709 13:19:36.684906 16718 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.053s	user 0.013s	sys 0.004s
I20260709 13:19:36.662824 16651 raft_consensus.cc:740] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:36.679714 16934 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.606189 16842 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.671697 16363 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.728s	user 0.015s	sys 0.000s
I20260709 13:19:36.687004 16960 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:36.682229 16696 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:36.680562 16964 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.711254 16891 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.711434 16994 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:36.711733 16898 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.699218 16758 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.721212 16696 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:36.722503 10895 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:36.715502 16985 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.728727 16994 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:36.735958 16994 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:36.731663 16891 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.736057 16985 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.738737 16891 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.705821 16987 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.753809 16987 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:36.725840 16991 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:36.754793 16991 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:36.756027 16991 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.757453 16826 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:36.760696 16758 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.761420 16955 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.762656 16884 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.706326 16966 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:36.730998 16961 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:19:36.765231 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3204) took 1414 ms. Trace:
I20260709 13:19:36.765642 10053 rpcz_store.cc:276] 0709 13:19:35.350994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.661247 (+310253us) service_pool.cc:224] Handling call
0709 13:19:35.690626 (+ 29379us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:35.697506 (+  6880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.697537 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:35.698230 (+   693us) log.cc:844] Serialized 78 byte log entry
0709 13:19:35.934406 (+236176us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:35.934707 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.934812 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.764932 (+830120us) raft_consensus.cc:1685] finished
0709 13:19:36.764956 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.765151 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6238,"raft.queue_time_us":6366,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19191,"thread_start_us":108348,"threads_started":1,"wal-append.queue_time_us":813914}
I20260709 13:19:36.767978 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:36.739620 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:36.769735 10897 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:36.740942 16651 consensus_queue.cc:260] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:36.742406 16983 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:19:36.744043 16977 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 13:19:36.744953 16984 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:19:36.700834 16600 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:36.770812 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:36.755926 16902 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.049s	user 0.015s	sys 0.001s
I20260709 13:19:36.810861 16995 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:36.707278 16718 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.717633 16363 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.819967 16651 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:36.839576 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 152569us
I20260709 13:19:36.844388 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:36.706710 16962 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.828019 16633 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.850502 10897 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:19:36.851413 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 140692us
I20260709 13:19:36.835208 16891 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.834959 16898 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.853478 10020 leader_election.cc:400] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:36.837760 16884 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.858414 10020 leader_election.cc:403] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:36.772076 16957 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.894282 10020 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:36.871310 16985 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:36.884186 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:36.884966 16552 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.887074 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.897835 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:36.898329 10197 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:36.899338 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.888295 16941 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:36.888639 16718 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:36.889237 16357 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:36.891338 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2580) took 1497 ms. Trace:
I20260709 13:19:36.892688 17001 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:36.931874 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:36.932888 10128 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:36.934015 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2445) took 1086 ms. Trace:
I20260709 13:19:36.934178 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:36.893292 16900 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.829947 16902 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.900252 10197 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:36.937698 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:36.901811 10126 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:36.908221 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:36.908747 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:36.910969 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:36.904340 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:36.911001 10169 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.542s (SENT) [suppressed 23 similar messages]
I20260709 13:19:36.927436 10049 rpcz_store.cc:276] 0709 13:19:35.394330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.689493 (+295163us) service_pool.cc:224] Handling call
0709 13:19:35.725082 (+ 35589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.728446 (+  3364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.891057 (+1162611us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b54001b7e50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:36.891261 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80260864}
I20260709 13:19:36.892654 16994 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.929913 17003 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:36.930434 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:36.931099 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:36.934536 10202 rpcz_store.cc:276] 0709 13:19:35.847742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.927029 (+ 79287us) service_pool.cc:224] Handling call
0709 13:19:35.948610 (+ 21581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.494851 (+546241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.933724 (+438873us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540035a7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:36.933942 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066638080}
I20260709 13:19:36.921034 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.000257 10124 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.003834 10129 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:36.861169 16990 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:36.935029 10128 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:37.011312 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:37.012060 10126 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:36.937332 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:36.936694 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.005023 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:36.926590 16952 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.006s
I20260709 13:19:36.896337 16654 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.029773 16654 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.862367 16898 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.007606 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.015793 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:37.019045 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.050886 10202 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:37.051597 10204 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.022307 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:36.859297 16891 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:36.926318 16969 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.026934 10894 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:37.010520 16718 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:36.894265 16884 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.015533 16957 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.028585 16994 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.035763 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:36.895581 16759 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.079595 16759 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:36.936609 16970 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:19:37.037900 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117026us
I20260709 13:19:37.083477 16970 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:37.084559 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3624) took 1771 ms (client timeout 1830 ms). Trace:
I20260709 13:19:37.084743 16970 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:37.085139 10199 rpcz_store.cc:269] 0709 13:19:35.312912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.331775 (+ 18863us) service_pool.cc:224] Handling call
0709 13:19:37.084247 (+1752472us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002184d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.084469 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321564928}
W20260709 13:19:37.086138 16792 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.027s	user 0.013s	sys 0.002s
I20260709 13:19:37.086701 16792 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.087671 16991 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:37.089036 16759 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:37.039347 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 128389us
I20260709 13:19:37.038668 10128 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:19:37.043435 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 134824us
I20260709 13:19:36.935954 16953 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:36.871376 16996 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.166s
I20260709 13:19:36.925985 16633 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 13:19:36.925046 16552 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.062453 10197 raft_consensus.cc:1240] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:19:37.025539 16967 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:37.071199 10126 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:37.072822 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.072926 17011 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:37.068150 16363 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:37.075551 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 748) took 1773 ms. Trace:
I20260709 13:19:37.117470 10893 rpcz_store.cc:276] 0709 13:19:35.302449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.328588 (+ 26139us) service_pool.cc:224] Handling call
0709 13:19:35.339088 (+ 10500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.854768 (+515680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.075235 (+1220467us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:37.075476 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183579392}
W20260709 13:19:37.076721 16930 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.553s	user 0.004s	sys 0.018s
I20260709 13:19:37.078074 16902 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.055068 17003 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:37.056473 10124 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:37.120966 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:37.094185 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:36.927690 16696 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.107779 10204 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:37.126488 10202 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:19:37.128127 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:37.013554 16900 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:37.135576 17004 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:37.137591 16696 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:37.112507 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.139360 16840 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:37.139811 16840 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:37.097708 16866 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.895s	user 0.016s	sys 0.000s
I20260709 13:19:37.073774  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:37.084751 16898 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.089323 16792 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.060304 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.109704 16946 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.134045 10123 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:37.020607 16990 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:37.141417 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 143198us
I20260709 13:19:37.111542 16953 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:37.113180 16967 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:37.092032 16718 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.159679 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.135041 16940 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:37.180711 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52451.4us
I20260709 13:19:37.185025 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.070056 16969 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.195845 10130 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:37.206096 10197 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:37.204473 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.166380 16902 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:37.219854 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.212646 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.220580 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.222317 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.215610 16953 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.216403 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.224210 10056 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:37.217576 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.214591 16891 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.225157 10127 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:37.225371 10048 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:37.226547 10047 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:37.182076 16962 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:37.235623 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.165484 16996 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:19:37.242827 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:37.232731 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 138612us
W20260709 13:19:37.229817 16958 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.004s
I20260709 13:19:37.188632 16946 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.250550 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:37.256544 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.277613 10048 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:19:37.279906 17008 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:19:37.228246 16952 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.255764 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.227321 16854 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:37.262645 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.187736 16840 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:37.286906 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 859) took 2334 ms (client timeout 1703 ms). Trace:
I20260709 13:19:37.234372 16654 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.204430 16940 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:37.268143 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.289831 16973 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.265259 16967 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.294109 16973 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.294792 16967 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:37.295418 16598 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.787s	user 0.006s	sys 0.010s
I20260709 13:19:37.295912 16598 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.296792 16973 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.297441 16996 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:19:37.298117 16598 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.298472 16990 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.299348 16900 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.299883 16840 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:37.303447 16902 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:37.303993 16902 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.304369 16902 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:37.305584 16900 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.306475 16900 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.311827 16900 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.320416 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.325933 16946 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.327008 16854 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.330217 16854 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.338176 16940 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.341224 16718 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.254588 16552 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.224002 16792 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:37.258817 16958 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.288077 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.271742 16953 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.290092 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.369599 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:37.291139 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 886) took 1598 ms. Trace:
I20260709 13:19:37.370407 10200 rpcz_store.cc:276] 0709 13:19:35.692968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.693890 (+   922us) service_pool.cc:224] Handling call
0709 13:19:35.694174 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.693816 (+999642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.290722 (+596906us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54002df0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:37.291014 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275293440}
I20260709 13:19:37.372109 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.373109 10200 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:37.374334 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:37.287647 10054 rpcz_store.cc:269] 0709 13:19:34.952874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.953002 (+   128us) service_pool.cc:224] Handling call
0709 13:19:37.286594 (+2333592us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002364d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.286798 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485404928}
W20260709 13:19:37.358048 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108963us
I20260709 13:19:37.376380 10096 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:37.353797 16969 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:37.369391 10122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 805) took 2301 ms (client timeout 1703 ms). Trace:
W20260709 13:19:37.378765 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 90817.6us
I20260709 13:19:37.391662 10197 raft_consensus.cc:2368] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:37.395136 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.381420 10200 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 23: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:37.402328 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:37.391871 10122 rpcz_store.cc:269] 0709 13:19:35.063450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.064772 (+  1322us) service_pool.cc:224] Handling call
0709 13:19:37.364519 (+2299747us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b540014c154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.364643 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994919808}
I20260709 13:19:37.407598 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.360121 16994 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:37.418164 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:37.420172 10053 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:37.428848 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:37.290845 16952 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.377848 16946 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:37.433811 10096 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:37.443493 16990 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.474946 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:37.479182 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:37.448980 17001 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
I20260709 13:19:37.453385 16850 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:37.457216 17007 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.366s	user 0.006s	sys 0.000s
I20260709 13:19:37.458506 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2449) took 1214 ms. Trace:
I20260709 13:19:37.481726 10205 rpcz_store.cc:276] 0709 13:19:36.244396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.400357 (+155961us) service_pool.cc:224] Handling call
0709 13:19:37.458305 (+1057948us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b54001fe854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.458438 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21928320}
I20260709 13:19:37.458329 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.418597 16994 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:37.463578 16884 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:37.464600 16753 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.467382 16872 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.468318 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:37.468603 17002 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.396s	user 0.003s	sys 0.000s
I20260709 13:19:37.532619 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.534138 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.418938 16552 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:37.419086 16969 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:37.483742 10054 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.461615 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.536749 10048 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:37.539634 10124 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:37.485935 17016 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:37.540763 17016 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.541311 17016 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:37.490847 17001 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:37.487936 16990 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.550086 16990 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.367955 16792 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.551244 16994 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.552753 16994 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:37.553833 16792 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:37.437597 16960 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.007s
I20260709 13:19:37.546432 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.559818 10050 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:37.528054 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.530299 16753 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:37.565598 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 182284us
I20260709 13:19:37.489538 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:37.534945 10055 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:37.477447 16957 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.481792 16718 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.578969 10054 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:37.486382 16972 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.525630 16654 raft_consensus.cc:2764] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:37.589838 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.599828 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.600871 10054 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:37.602169 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.605189 16753 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:37.606889 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 864) took 2517 ms (client timeout 1642 ms). Trace:
W20260709 13:19:37.607443 10051 rpcz_store.cc:269] 0709 13:19:35.089301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.091305 (+  2004us) service_pool.cc:224] Handling call
0709 13:19:37.606669 (+2515364us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540010c2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.606803 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921410304}
I20260709 13:19:37.608820 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.610134 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.613374 10050 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:37.591737 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.614576 10200 raft_consensus.cc:2393] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:37.582892 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93464.9us
I20260709 13:19:37.615841 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.595590 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.618417 10205 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:37.618561 16872 consensus_queue.cc:1059] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:37.619571 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:37.619828 16872 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:37.617446 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.620944 16891 raft_consensus.cc:3055] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:37.621176 10200 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:37.622550 16900 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:37.613477 16968 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.623010 16900 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:37.623611 16968 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.624205 16900 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.596419 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:37.626433 16968 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.621420 16891 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:37.628381 10200 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:37.628218 16891 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.629756 16891 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:37.629667 10203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2443) took 1910 ms (client timeout 2116 ms). Trace:
W20260709 13:19:37.631927 10203 rpcz_store.cc:269] 0709 13:19:35.719081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.828882 (+109801us) service_pool.cc:224] Handling call
0709 13:19:37.629451 (+1800569us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.629596 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276045696}
I20260709 13:19:37.635160 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:37.636955 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3172) took 2700 ms (client timeout 2166 ms). Trace:
I20260709 13:19:37.644196 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:37.637964 10131 rpcz_store.cc:269] 0709 13:19:34.936013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.970550 (+ 34537us) service_pool.cc:224] Handling call
0709 13:19:37.636652 (+2666102us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002d1254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.636859 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2489971072}
I20260709 13:19:37.587729 16884 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:37.587404 16969 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.563725 16960 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.670034 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:37.677575 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.679567 10131 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:37.679890 10020 leader_election.cc:400] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:37.673776 10205 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.680380 10020 leader_election.cc:403] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.680559 10205 raft_consensus.cc:614] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.594750 16957 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.684338 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:37.685495 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.677716 16970 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.688776 10131 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:19:37.632603 16972 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.657677 16952 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.663327 16718 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.710204 16718 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.710880 16718 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.683897 16967 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 13:19:37.691740 10047 raft_consensus.cc:2410] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 16 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
W20260709 13:19:37.687783 16651 consensus_meta.cc:287] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.004s
W20260709 13:19:37.724642 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.684985 16678 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:37.700984 10131 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:19:37.687650 16654 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.753594 10125 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:19:37.713999 16970 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.693372 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:37.755827 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62552.8us
I20260709 13:19:37.757061 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.703078 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:37.760190 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.665076 16969 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.734669 10124 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:37.735325 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.700984 16990 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:37.747669 16908 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:37.668254 16552 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:19:37.767431 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:37.770817 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.771472 10051 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:19:37.771955 10051 pending_rounds.cc:85] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 11
I20260709 13:19:37.772266 10051 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:19:37.772646 10051 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.773128 10051 pending_rounds.cc:107] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:19:37.773514 10051 raft_consensus.cc:2889] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.776600 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:37.762746 16651 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:37.762329 16678 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.786547 10899 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:37.789573 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.790400 10055 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 11.
I20260709 13:19:37.769253 16958 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.779919 16990 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:37.768185 16908 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:37.793510 10048 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:37.808318 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:37.810968 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:37.811998 10203 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:37.800770 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 240090us
I20260709 13:19:37.801874 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.828709 10200 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:37.788475 16994 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:37.830521 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.833966 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.839527 10055 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:37.841471 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:37.841928 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:37.842629 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.792462 16837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.941s	user 0.026s	sys 0.028s
I20260709 13:19:37.845283 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.846650 10048 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:37.810511 16840 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:37.810540 16975 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:37.811441 16990 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:37.870597 16990 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:37.851579 16915 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
W20260709 13:19:37.873096 17026 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:19:37.844367 16837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.993s	user 0.027s	sys 0.029s
I20260709 13:19:37.875226 10199 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:37.788367 16995 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:19:37.876806 10891 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1043) took 2227 ms (client timeout 1542 ms). Trace:
I20260709 13:19:37.877334 16995 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.877300 10891 rpcz_store.cc:269] 0709 13:19:35.649417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.649613 (+   196us) service_pool.cc:224] Handling call
0709 13:19:37.876554 (+2226941us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540063ef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:37.876693 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546195840}
I20260709 13:19:37.877980 16995 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.879057 10201 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:37.881044 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.881814 10051 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:19:37.886149 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.887387 16969 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.888114 16969 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:37.888701 16969 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.891283 16908 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.892432 16908 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:37.835533 17028 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.861271 16860 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:37.919020 16960 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.865116 17031 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:37.869438 16975 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.791846 16902 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.895968 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.843438 16991 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:37.919454 16718 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.856036 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.933419 16964 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.914902  9939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d51ad94710c348889d417f7f00032948 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:19:37.916574 17029 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:37.918108 16970 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.890933 16969 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.935575 10197 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.922376 10203 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:37.938764 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:37.939486 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:37.970240 10127 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:37.930212 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.931677 16957 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.932119 16995 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.928272 17028 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:37.939157 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:37.933888 16746 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:37.882893 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.946151 10049 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:37.981465 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.949798 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:37.945469 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:37.958238 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.972703 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:37.966840 10170 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:37.973835 10048 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:37.976289 10129 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.015756 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.016522 10127 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:38.019356 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.948228 16718 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:37.934293 17031 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:38.022193 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:38.022460 10049 raft_consensus.cc:1240] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:38.044240 10203 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:38.015414 16746 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:37.940093 16972 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.018607 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:37.946960 16902 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:38.056881 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.064119 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.952488 16960 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.073352 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.074411 10205 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:37.963171 16967 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:38.048190 17035 log.cc:927] Time spent T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:19:38.084945 16967 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.089859 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.043743 17031 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:37.954579 17016 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.096858 17031 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.098387 17031 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:38.099361 16972 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:38.099593 17016 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.100365 16965 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.101104 16965 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.102277 10122 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:38.085482 16960 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.104518 16972 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.105970 16960 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.105662 16972 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.047968 16718 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.945631 16991 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:38.094147 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75629.6us
I20260709 13:19:37.953181 16964 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.963927 16748 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.093640 16746 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.018677 16995 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:38.147388 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.147557 16995 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:38.146605 16991 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.148793 16995 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.149446 16991 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:37.945094 16975 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.121551 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:38.136142 10127 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:38.161281 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.162684 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.152068 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3193) took 1229 ms. Trace:
I20260709 13:19:38.136080 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:38.152979 16746 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:37.935961 16678 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.163532 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.174909 10129 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:38.166018 10127 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:38.164131 10130 rpcz_store.cc:276] 0709 13:19:36.922464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.471756 (+549292us) service_pool.cc:224] Handling call
0709 13:19:37.472213 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:37.472697 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:37.472730 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:37.484120 (+ 11390us) log.cc:844] Serialized 26 byte log entry
0709 13:19:37.833901 (+349781us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:37.833972 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.834084 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.151744 (+317660us) raft_consensus.cc:1685] finished
0709 13:19:38.151778 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.151972 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":155,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":10673,"thread_start_us":244908,"threads_started":1,"wal-append.queue_time_us":585240}
W20260709 13:19:38.178015 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56553.8us
I20260709 13:19:38.183043 10170 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:38.185853 16967 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:38.191360 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.017s	sys 0.021s
W20260709 13:19:38.191812 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.017s	sys 0.021s
W20260709 13:19:38.192683 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 509121us
I20260709 13:19:38.193557 10095 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.193976 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.198249 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.199385 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.202597 10170 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:38.204069 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 820) took 1297 ms. Trace:
I20260709 13:19:38.214968 10123 rpcz_store.cc:276] 0709 13:19:36.907150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.907423 (+   273us) service_pool.cc:224] Handling call
0709 13:19:38.203814 (+1296391us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002ef254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.203982 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311481728}
W20260709 13:19:38.189709 17012 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.113s	user 0.008s	sys 0.000s
I20260709 13:19:38.207867 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2600) took 1280 ms. Trace:
I20260709 13:19:38.216624 10053 rpcz_store.cc:276] 0709 13:19:36.919694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.025166 (+105472us) service_pool.cc:224] Handling call
0709 13:19:38.199365 (+1174199us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b5400369f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.199601 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593041280}
I20260709 13:19:38.193692 16600 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:38.223687 10170 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:38.186715 16849 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:38.226207 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.227252 10130 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:38.224566 10127 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:38.138092 16718 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.230001 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.233100 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:38.236199 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.238197 10053 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:38.228825 16836 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:38.241544 16964 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.229755 16849 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.242725 16849 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.207334 16957 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.243686 16957 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.246232 16957 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.228792 16654 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.181469 16678 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.253545 16678 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.254551 10051 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:38.256255 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.207104 16960 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:38.258143 16960 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.234949 16748 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:38.260402 10127 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:38.241765 16900 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.265224 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.257409 10130 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 11.
I20260709 13:19:38.265985 10899 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:38.267247 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.269189 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.274803 10123 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:38.270344 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:38.225646 16940 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.003s	sys 0.041s
W20260709 13:19:38.282541 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.283416 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.288025 10129 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:38.289266 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.292994 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.294286 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.297622 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.297704 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.298669 10130 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:38.274206 16891 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
I20260709 13:19:38.263199 16967 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.260506 17016 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.299634 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:38.298746 17012 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:19:38.301743 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.307386 16967 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:38.309892 16953 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.909s	user 0.014s	sys 0.001s
I20260709 13:19:38.310118 16967 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.262704 16959 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:38.310868 17028 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.260250 16600 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.312247 16600 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.313616 16990 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.315917 16969 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.316493 16969 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.318631 16969 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:38.319279 16651 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:19:38.319746 16651 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:38.320393 16872 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:38.273608 16748 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.321945 16748 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.286582 16902 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.332486 10899 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:19:38.307782 16965 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.286706 16654 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:38.319628 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.290480 16908 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:38.336078 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.281464 16836 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.345451 16678 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:38.346805 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.342633 10049 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:38.346489 16836 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.291882 16552 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.345520 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.319689 17016 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.350872 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:38.343310 16727 consensus_queue.cc:1059] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:19:38.346393 16990 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.352510 10205 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:38.318848 16849 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.341614 16718 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.365788 16727 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.366291 16747 raft_consensus.cc:3055] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:38.366737 16747 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.367863 16747 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.370494 16965 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:38.371556 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:38.372752 10205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:38.369284 16747 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.341348 16654 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.375394 16654 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.355657 16960 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.373349 10205 raft_consensus.cc:606] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:38.377039 10205 raft_consensus.cc:614] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:38.329797 16969 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.344594 16908 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:38.381067 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.384260 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.366483 16727 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.385205 10899 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
W20260709 13:19:38.386132 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.362038 16975 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:38.384747 16908 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:38.359251 16552 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:38.412393 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.012s	sys 0.011s
W20260709 13:19:38.416244 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.012s	sys 0.011s
W20260709 13:19:38.416858 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 581241us
I20260709 13:19:38.425913 10130 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:19:38.427218 10204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.020s
W20260709 13:19:38.427704 10204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.008s	sys 0.020s
W20260709 13:19:38.439996 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.440737 10020 leader_election.cc:400] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:38.441160 10020 leader_election.cc:403] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:38.441184 17001 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.442016 17001 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:38.442659 16860 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.002s
W20260709 13:19:38.443337 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.443650 16727 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.443748 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.444868 10129 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:38.445607 17001 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.445937 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.446907 10129 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:38.447064 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 757) took 1029 ms. Trace:
I20260709 13:19:38.451750 10893 rpcz_store.cc:276] 0709 13:19:37.417673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.417876 (+   203us) service_pool.cc:224] Handling call
0709 13:19:37.442955 (+ 25079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:37.657233 (+214278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:37.657644 (+   411us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:37.681806 (+ 24162us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.178603 (+496797us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:38.178673 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.178789 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.446729 (+267940us) raft_consensus.cc:1685] finished
0709 13:19:38.446764 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.446979 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154006,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":38068,"thread_start_us":682148,"threads_started":2,"wal-append.queue_time_us":756573}
I20260709 13:19:38.453908 10205 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:19:38.454591 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.447736 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.450421 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3628) took 1817 ms. Trace:
I20260709 13:19:38.455879 10202 rpcz_store.cc:276] 0709 13:19:36.632621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.055354 (+422733us) service_pool.cc:224] Handling call
0709 13:19:37.126069 (+ 70715us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:37.134942 (+  8873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.450078 (+1315136us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b54001d0450. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.450320 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115018624}
I20260709 13:19:38.456934 10203 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:38.456633 16960 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.451085 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:38.473194 10124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.016s	sys 0.016s
W20260709 13:19:38.473647 10124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.016s	sys 0.016s
W20260709 13:19:38.475855 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.013s	sys 0.013s
I20260709 13:19:38.406140 16975 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.450457 16990 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:38.480338 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.013s	sys 0.013s
W20260709 13:19:38.481330 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 668972us
I20260709 13:19:38.482442 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:38.477838 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.458890 16960 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.472936 16994 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:38.484540 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
W20260709 13:19:38.485565 10196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.015s	sys 0.011s
W20260709 13:19:38.487870 10196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.015s	sys 0.011s
I20260709 13:19:38.474223 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:38.488273 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:38.485889 10865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.015s	sys 0.011s
W20260709 13:19:38.488987 10865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.015s	sys 0.011s
I20260709 13:19:38.489030 10127 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:38.487497 10202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:38.490199 10202 raft_consensus.cc:606] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:38.490605 10202 raft_consensus.cc:614] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:38.490648 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:38.491324 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.492880 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3185) took 1572 ms. Trace:
I20260709 13:19:38.493351 10126 rpcz_store.cc:276] 0709 13:19:36.920046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.114318 (+194272us) service_pool.cc:224] Handling call
0709 13:19:38.204250 (+1089932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.204520 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.204548 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:38.204591 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:38.204620 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.204696 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.492595 (+287899us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002ef250. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.492784 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2505795840}
W20260709 13:19:38.496201 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3626) took 3182 ms (client timeout 1956 ms). Trace:
I20260709 13:19:38.497820 10198 raft_consensus.cc:1240] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:19:38.490046 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:38.496826 10201 rpcz_store.cc:269] 0709 13:19:35.313550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.435554 (+122004us) service_pool.cc:224] Handling call
0709 13:19:38.495850 (+3060296us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54001bf654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.496002 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5468261760}
I20260709 13:19:38.505008 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:38.507223 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:38.508764 10200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.024s	sys 0.060s
W20260709 13:19:38.509292 10200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.025s	sys 0.061s
I20260709 13:19:38.469885 16753 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:19:38.481392 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.514775 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.525324 10130 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:38.507457 10124 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:38.531590 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.533104 10129 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:38.535643 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53321.1us
W20260709 13:19:38.537007 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47032.1us
I20260709 13:19:38.542583 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:38.545045 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.486752 16994 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:38.552026 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.022s	sys 0.053s
W20260709 13:19:38.562989 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.022s	sys 0.053s
I20260709 13:19:38.551286 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.508742 16916 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:38.564819 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.508018 17001 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:38.511528 16967 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:38.512362 17031 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.578629 16957 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:38.579895 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.009s	sys 0.033s
I20260709 13:19:38.579596 16957 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:38.580358 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.009s	sys 0.033s
I20260709 13:19:38.582607 16957 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.583664 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2596) took 1665 ms. Trace:
I20260709 13:19:38.584316 17032 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.584925 17032 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.589470 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.590514 10054 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:19:38.593499 16967 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:38.594002 16967 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:38.597066 16968 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:38.600817 16968 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:38.600996 16849 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.601384 16968 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:38.603425 10131 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:38.603785 10899 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:38.604339 16552 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.605638 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3701) took 1403 ms. Trace:
I20260709 13:19:38.606169 10131 rpcz_store.cc:276] 0709 13:19:37.202260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.730380 (+528120us) service_pool.cc:224] Handling call
0709 13:19:37.730878 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.603972 (+873094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.605336 (+  1364us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54001c8254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:38.605567 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005966080}
I20260709 13:19:38.607561 16908 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.610515 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:38.614123 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.590952 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3231) took 1128 ms. Trace:
I20260709 13:19:38.616951 10054 rpcz_store.cc:276] 0709 13:19:37.462552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.589054 (+1126502us) service_pool.cc:224] Handling call
0709 13:19:38.590884 (+  1830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:38.615615 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:38.619568 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.622278 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:38.546026 16953 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:38.623581 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:38.584142 10047 rpcz_store.cc:276] 0709 13:19:36.918243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.247442 (+329199us) service_pool.cc:224] Handling call
0709 13:19:38.583432 (+1335990us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54002e8e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.583591 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934234368}
I20260709 13:19:38.624305 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:38.508096 16991 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:19:38.631965 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.638075 10130 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:38.639467 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:38.641690 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:38.625494 10891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.027s	sys 0.033s
I20260709 13:19:38.620345 10131 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:38.551561 17028 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:38.642596 10891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.027s	sys 0.034s
I20260709 13:19:38.643515 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.644892 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.645510 10131 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:38.646390 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.628509 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.625231 10129 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:38.570263 16916 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.653909 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.568220 16994 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:38.649541 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:38.647012 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:38.656571 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.660477 10897 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:38.657856 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.649077 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.630221 16849 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:38.566900 17033 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:38.664821 10131 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:38.665934 17032 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.663883 17037 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:38.648689 17036 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.684552 17036 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.685240 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.686199 16955 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:38.686640 16955 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.611011 16908 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.687014 16955 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.602567 16968 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.631444 16953 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.691911 16953 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:38.667387 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:38.706394 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:38.706650 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:38.682225 16727 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:38.683862 17037 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.652415 17001 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.657907 16852 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:38.712687 16727 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.704954 10893 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:38.675242 10203 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:38.716445 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3696) took 1803 ms. Trace:
I20260709 13:19:38.716915 10125 rpcz_store.cc:276] 0709 13:19:36.913227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.007187 (+ 93960us) service_pool.cc:224] Handling call
0709 13:19:38.716202 (+1709015us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:38.716374 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616935168}
I20260709 13:19:38.707378 10131 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 12.
I20260709 13:19:38.634382 16860 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:38.657970 16884 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:38.709424 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:38.661662 16916 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:38.655211 16959 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:38.718675 10128 raft_consensus.cc:1275] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:38.695987 17028 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:38.668968 17033 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.741115 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:38.741150 10125 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:38.707275 16849 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:38.748011 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3629) took 1749 ms. Trace:
W20260709 13:19:38.750595 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.759891 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:38.759967 10199 rpcz_store.cc:276] 0709 13:19:36.998699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.086308 (+ 87609us) service_pool.cc:224] Handling call
0709 13:19:37.086738 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.875761 (+789023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.747603 (+871842us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54001ff754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:38.747910 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812671744}
I20260709 13:19:38.711582 17037 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.713425 16852 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:38.751199 16968 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:38.786440 16908 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.791985 16908 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.796170 10130 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:19:38.799088 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3229) took 1341 ms. Trace:
I20260709 13:19:38.786790 16964 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.798439 16955 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.799642 10052 rpcz_store.cc:276] 0709 13:19:37.457825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.959593 (+501768us) service_pool.cc:224] Handling call
0709 13:19:37.959948 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:37.960663 (+   715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:37.960704 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:37.961415 (+   711us) log.cc:844] Serialized 52 byte log entry
0709 13:19:38.563549 (+602134us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:38.563594 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.563867 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.798833 (+234966us) raft_consensus.cc:1685] finished
0709 13:19:38.798854 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.799014 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3682,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"thread_start_us":601665,"threads_started":1,"wal-append.queue_time_us":835300}
I20260709 13:19:38.800817 16964 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.723032 16960 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.716928 16990 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:38.712766 16696 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:38.767198 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.824869 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.723465 16884 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.804795 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.806777 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.841993 10050 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:38.841538 10052 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:38.755750 10131 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:38.808594 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:38.809441 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:38.810226 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.812122 10053 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:38.845376 10056 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:38.743860 16971 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:38.758796 17033 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:38.817452 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.825083 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.020s	sys 0.044s
W20260709 13:19:38.876464 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.020s	sys 0.045s
W20260709 13:19:38.860503 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.739825 16975 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:38.715409 16727 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.715761 17001 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.751675 16991 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.897635 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.773850 16852 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:38.866313 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.753648 16959 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:38.928687 10051 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
W20260709 13:19:38.877734 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 576058us
I20260709 13:19:38.930685 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.932528 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:38.934664 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:38.826836 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:38.846498 10054 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:38.846606 10047 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: [15.9-16.10]
I20260709 13:19:38.848412 16884 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.879614 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:38.827450 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86408.3us
W20260709 13:19:38.944039 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64500.5us
I20260709 13:19:38.944607 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:38.824038 16696 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:38.866595 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:38.895605 10896 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.083s
W20260709 13:19:38.948488 10896 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.011s	sys 0.083s
I20260709 13:19:38.787016 17036 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:38.770391 16957 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.907826 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:38.866303 16990 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:38.915293 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.919936 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.848804 16952 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:38.854405 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:38.933667 16900 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:38.937458 16552 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:38.938879 16975 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:38.939656 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 112986us
I20260709 13:19:38.936930 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:38.949973 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.972827 10895 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:38.978488 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.980403 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:38.991618 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:39.002945 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.018072 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:39.024154 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:39.024537 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 87698.8us
I20260709 13:19:39.035264 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:39.029582 10898 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:39.030184 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.039422 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 759) took 1156 ms. Trace:
W20260709 13:19:39.028491 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 50115.8us
I20260709 13:19:39.040002 10891 rpcz_store.cc:276] 0709 13:19:37.883491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.915792 (+ 32301us) service_pool.cc:224] Handling call
0709 13:19:37.916268 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.022345 (+106077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.022379 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.072966 (+ 50587us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.643150 (+570184us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:38.643196 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.012719 (+369523us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54003f2ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:39.012806 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:39.012892 (+    86us) raft_consensus.cc:1685] finished
0709 13:19:39.012914 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.039349 (+ 26435us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":658,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":32116,"spinlock_wait_cycles":242973440,"thread_start_us":547150,"threads_started":1,"wal-append.queue_time_us":842334}
I20260709 13:19:39.042182 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.971868 16696 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:38.969334 16995 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.034s	sys 0.074s
I20260709 13:19:39.041011 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:38.991163 17036 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:38.976846 16840 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.062166 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:38.982631 16955 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:38.993005 16971 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:38.931923 16991 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:39.058491 16747 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.685s	user 0.014s	sys 0.000s
W20260709 13:19:39.071563 16995 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.034s	sys 0.075s
W20260709 13:19:39.077626 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.993602 16952 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:19:39.084702 10196 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.011s	sys 0.014s
I20260709 13:19:39.087641 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:39.115147 10196 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.012s	sys 0.014s
W20260709 13:19:39.087649 10126 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.001s	sys 0.025s
I20260709 13:19:39.015223 16908 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:39.045454 16940 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:38.946871 16849 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:39.116631 10126 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.001s	sys 0.025s
I20260709 13:19:38.992799 16957 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.099859 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:39.057241 10892 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:39.061558 16552 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:38.934109 17028 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:38.993961 16990 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.086241 17036 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:39.096818 16678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.019s	sys 0.008s
W20260709 13:19:39.123350 10127 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.006s	sys 0.025s
W20260709 13:19:39.225697 10127 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.006s	sys 0.026s
I20260709 13:19:39.127562 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3642) took 1343 ms. Trace:
I20260709 13:19:39.094748 16971 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.131199 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:39.061244 16900 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:19:39.223493 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:39.109543 17032 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:39.235682 10048 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:39.128963 16965 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:39.119355 16952 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:39.230639 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2446) took 3357 ms. Trace:
W20260709 13:19:39.218263 16600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.017s	sys 0.022s
W20260709 13:19:39.227444 10205 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.021s	sys 0.031s
I20260709 13:19:39.093243 16994 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:39.292248 10205 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.021s	sys 0.031s
W20260709 13:19:39.228351 16678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.020s	sys 0.008s
I20260709 13:19:39.177376 16995 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.123952 16940 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.265226 10047 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:39.289857 10198 rpcz_store.cc:276] 0709 13:19:35.873534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.044588 (+171054us) service_pool.cc:224] Handling call
0709 13:19:38.497489 (+2452901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.498361 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.230352 (+731991us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b54001bf650. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:39.230558 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5640078592}
I20260709 13:19:39.095665 16840 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.120189 16955 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:19:39.305840 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:39.234582 10122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.018s	sys 0.037s
W20260709 13:19:39.307206 10122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.026s	sys 0.042s
W20260709 13:19:39.262807 16792 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.012s	sys 0.032s
I20260709 13:19:39.226378 17036 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:39.313290 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:39.320817 16968 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.005s
I20260709 13:19:39.322783 10898 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:19:39.297817 17040 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:19:39.227005 10196 rpcz_store.cc:276] 0709 13:19:37.784275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.793035 (+  8760us) service_pool.cc:224] Handling call
0709 13:19:37.793560 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.793857 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:37.831779 (+ 37922us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.115934 (+1284155us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.126826 (+ 10892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:39.126917 (+    91us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:39.127003 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.127268 (+   265us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.127471 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41219,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":48,"raft.queue_time_us":129,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4142,"thread_start_us":1283122,"threads_started":2,"wal-append.queue_time_us":904059}
I20260709 13:19:39.234014 16971 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:39.292078 16600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.018s	sys 0.022s
W20260709 13:19:39.312923 16792 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.014s	sys 0.041s
I20260709 13:19:39.256615 16748 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:39.331846 10049 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:39.306742 10892 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:39.262584 17051 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:39.257479 16972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.017s	sys 0.026s
I20260709 13:19:39.294118 16363 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:39.256850 17045 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:39.304450 16727 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:39.307214 16940 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:39.341930 10022 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.003s	sys 0.010s
W20260709 13:19:39.397708 10022 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.003s	sys 0.010s
I20260709 13:19:39.360610 16959 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:39.366887 16990 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:39.351523 16552 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.366945 16991 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:39.370105 16957 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:39.371816 17052 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:19:39.373773 16994 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:39.376250 17050 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.261s	user 0.004s	sys 0.000s
I20260709 13:19:39.376603 16840 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:39.290940 17032 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.257133 17042 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.257s
I20260709 13:19:39.311094 16995 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.409090 16727 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:39.412014 17045 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:39.355543 16972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.017s	sys 0.027s
W20260709 13:19:39.412575 16953 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.720s	user 0.013s	sys 0.000s
W20260709 13:19:39.414593 17033 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:19:39.416555 17001 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:39.429399 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 484540us
I20260709 13:19:39.504745 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:39.506053 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:39.431536 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:39.507859 10056 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
I20260709 13:19:39.434103 10053 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:39.435365 10052 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:39.448735 16967 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.854s	user 0.009s	sys 0.005s
I20260709 13:19:39.441531 10895 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:39.511840 10052 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:19:39.449473 17037 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.000s
W20260709 13:19:39.449254 16654 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.073s	user 0.016s	sys 0.000s
I20260709 13:19:39.438131 10048 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
W20260709 13:19:39.529994 10202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.031s	sys 0.028s
W20260709 13:19:39.561380 10202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.032s	sys 0.028s
W20260709 13:19:39.530642 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 495511us
W20260709 13:19:39.500020 16969 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.103s	user 0.016s	sys 0.000s
I20260709 13:19:39.563174 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:39.570533 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:39.566084 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:39.571977 10056 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.567221 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:39.508395 16916 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.005s
I20260709 13:19:39.507503 16727 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:39.620606 10171 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.015s	sys 0.036s
W20260709 13:19:39.621322 10171 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.015s	sys 0.036s
W20260709 13:19:39.624385 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.018s	sys 0.033s
W20260709 13:19:39.626886 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.018s	sys 0.034s
W20260709 13:19:39.630520 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 535406us
I20260709 13:19:39.650012 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:39.497604 17036 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.605249 16969 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.451685 16852 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.603710 16978 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:19:39.678026 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 492843us
I20260709 13:19:39.626598 17045 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.522275 17033 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.521488 16953 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.563058 16967 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.570269 16654 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.684823 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:39.668680 16860 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.947s	user 0.014s	sys 0.000s
I20260709 13:19:39.613687 16916 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:39.671094 10056 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:39.569993 17037 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.685758 17045 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:39.701499 17066 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:19:39.703938 10899 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:39.709848 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.728139 10891 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: []
W20260709 13:19:39.690188 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.692171 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 894) took 1056 ms. Trace:
I20260709 13:19:39.688237 10123 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:39.685920 10170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.017s
W20260709 13:19:39.733399 10170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.005s	sys 0.017s
I20260709 13:19:39.740268 10201 raft_consensus.cc:606] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:39.741752 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.015s	sys 0.017s
W20260709 13:19:39.742295 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.015s	sys 0.017s
I20260709 13:19:39.694109 16969 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.751412 10021 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.013s	sys 0.016s
W20260709 13:19:39.752596 10126 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.011s	sys 0.017s
W20260709 13:19:39.758237 10126 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.011s	sys 0.017s
W20260709 13:19:39.757503 10021 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.013s	sys 0.016s
I20260709 13:19:39.704763 16860 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:39.762281 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 631213us
I20260709 13:19:39.768400 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:39.770771 10051 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:39.762959 17037 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:39.737679 17001 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.727797 16908 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:39.785597 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.745951 17068 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:39.786110 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:39.802161 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:39.802747 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.806772 10051 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:39.809957 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3205) took 1860 ms. Trace:
W20260709 13:19:39.812671 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 242225us
I20260709 13:19:39.813957 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:39.767562 16994 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:39.766551 16969 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:39.816010 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 310044us
I20260709 13:19:39.846513 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:39.892261 10055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.003s	sys 0.030s
W20260709 13:19:39.892722 10055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.003s	sys 0.030s
I20260709 13:19:39.733135 10205 rpcz_store.cc:276] 0709 13:19:38.635368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.635580 (+   212us) service_pool.cc:224] Handling call
0709 13:19:38.636075 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.636383 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:39.294729 (+658346us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.691529 (+396800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:39.691631 (+   102us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:39.691725 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.691848 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.692068 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2985,"prepare.queue_time_us":736009,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":3780,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":7441,"thread_start_us":1050281,"threads_started":2}
I20260709 13:19:39.810834 10126 rpcz_store.cc:276] 0709 13:19:37.949082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.494579 (+545497us) service_pool.cc:224] Handling call
0709 13:19:38.494931 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.507557 (+ 12626us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:39.124477 (+616920us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.804514 (+680037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:39.804630 (+   116us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:39.804736 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.804866 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.809689 (+  4823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73275,"prepare.queue_time_us":997627,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"thread_start_us":1191616,"threads_started":2,"wal-append.queue_time_us":738753}
I20260709 13:19:39.791224 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:39.812355 16900 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:40.069604 17067 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:40.040387 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 253267us
I20260709 13:19:40.071053 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:40.073423 10169 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:40.073879 16965 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.002s
W20260709 13:19:40.027812 17058 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.346s	user 0.001s	sys 0.001s
I20260709 13:19:39.807897 16908 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.801631 17001 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:40.067945 17070 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:19:40.028532 16994 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:39.802196 17037 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:39.813031 17068 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.196841 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2617) took 1388 ms. Trace:
I20260709 13:19:40.197288 10048 rpcz_store.cc:276] 0709 13:19:38.808507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.809076 (+   569us) service_pool.cc:224] Handling call
0709 13:19:40.196625 (+1387549us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400086454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.196772 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":660208512}
I20260709 13:19:39.806437 17045 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.200029 17045 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.200559 17045 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:40.203852 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.076679 17033 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.076856 16836 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:40.079401 17075 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:40.128484 16965 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:40.085260 17090 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:40.086401  9939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d51ad94710c348889d417f7f00032948 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:40.087476 17036 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:40.064503 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 218090us
W20260709 13:19:40.088521 17069 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
I20260709 13:19:40.089033 16852 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:40.091708 17053 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.424s	user 0.001s	sys 0.000s
I20260709 13:19:40.088436 16727 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:40.104485 17077 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:19:39.815933 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85203.6us
W20260709 13:19:40.120486 10899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 767) took 1506 ms (client timeout 1802 ms). Trace:
W20260709 13:19:40.235355 10899 rpcz_store.cc:269] 0709 13:19:38.614268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.614548 (+   280us) service_pool.cc:224] Handling call
0709 13:19:40.120193 (+1505645us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54006ed7d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.120405 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474196352}
I20260709 13:19:40.137151 10900 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:39.871570 16552 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:40.155077 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3188) took 3234 ms. Trace:
I20260709 13:19:40.220027 16953 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.177886 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36318 (request call id 49) took 1555 ms. Trace:
I20260709 13:19:40.180404 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1057) took 1105 ms. Trace:
I20260709 13:19:40.202898 17045 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.183895 16978 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:40.186733 10898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1054) took 1212 ms. Trace:
I20260709 13:19:40.190527 16991 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.192946 17065 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.506s	user 0.000s	sys 0.001s
I20260709 13:19:40.193519 16678 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.194298 16990 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.320720 17001 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.320842 16990 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.263612 17096 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.323387 16748 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:40.324070 16748 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.323804 17096 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.327625 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.328591 16969 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.329237 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.329775 16994 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.329910 10196 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:40.331410 17099 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.332053 17099 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:40.195325 17054 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.859s	user 0.002s	sys 0.013s
I20260709 13:19:40.039494 16746 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:40.337064 17063 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.452s	user 0.001s	sys 0.002s
I20260709 13:19:40.075816 17094 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.206015 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:40.208832 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.339677 10896 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 764) took 1978 ms. Trace:
I20260709 13:19:40.339874 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 898) took 1660 ms. Trace:
I20260709 13:19:40.340749 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:40.340445 10202 rpcz_store.cc:276] 0709 13:19:38.679243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.679398 (+   155us) service_pool.cc:224] Handling call
0709 13:19:38.679773 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.680180 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.680214 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.680733 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:19:39.562037 (+881304us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:39.562091 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.562203 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.336008 (+773805us) spinlock_profiling.cc:243] Waited 6.19 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:40.339533 (+  3525us) raft_consensus.cc:1685] finished
0709 13:19:40.339555 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.339805 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1261,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":14247168,"thread_start_us":880821,"threads_started":1,"wal-append.queue_time_us":1124859}
I20260709 13:19:39.856817 16792 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.284s
I20260709 13:19:40.107870 16995 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.344898 16995 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.345479 16995 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.084455 17078 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:40.338929 17094 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.347944 16990 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.220827 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3204) took 2272 ms. Trace:
W20260709 13:19:40.213816 16975 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.167s	user 0.011s	sys 0.003s
W20260709 13:19:40.229784 16696 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.172s	user 0.016s	sys 0.000s
I20260709 13:19:40.348739 10127 rpcz_store.cc:276] 0709 13:19:37.948814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.489869 (+541055us) service_pool.cc:224] Handling call
0709 13:19:38.490200 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.490427 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:39.226258 (+735831us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.687931 (+461673us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.902784 (+214853us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.220349 (+317565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.220424 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.220492 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.220588 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.220752 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22049,"prepare.queue_time_us":1078045,"prepare.run_cpu_time_us":193,"prepare.run_wall_time_us":6765,"raft.queue_time_us":1094257,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":285,"thread_start_us":1649671,"threads_started":4,"wal-append.queue_time_us":615738}
I20260709 13:19:40.350126 16696 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:40.222345 16884 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.240s	user 0.014s	sys 0.000s
W20260709 13:19:40.156455 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2618) took 1347 ms (client timeout 1653 ms). Trace:
W20260709 13:19:40.239517 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:40.353842 10049 rpcz_store.cc:269] 0709 13:19:38.808820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.809194 (+   374us) service_pool.cc:224] Handling call
0709 13:19:40.156157 (+1346963us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54002e9354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.156366 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1119265024}
I20260709 13:19:40.219877 17075 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.358033 17075 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.359498 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:40.360299 10129 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:40.239750 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 425913us
I20260709 13:19:40.238515 10128 rpcz_store.cc:276] 0709 13:19:36.920967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.135935 (+214968us) service_pool.cc:224] Handling call
0709 13:19:38.718208 (+1582273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.743594 (+ 25386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.154643 (+1411049us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54000a0850. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.154958 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3637764352}
I20260709 13:19:40.241816 10891 rpcz_store.cc:276] 0709 13:19:39.074655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.084830 (+ 10175us) service_pool.cc:224] Handling call
0709 13:19:39.085287 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.728862 (+643575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.179989 (+451127us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540064e1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:40.180311 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477768064}
I20260709 13:19:40.181715 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 894) took 1334 ms. Trace:
I20260709 13:19:40.229939 16957 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.241354 10201 rpcz_store.cc:276] 0709 13:19:38.622183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.623403 (+  1220us) service_pool.cc:224] Handling call
0709 13:19:40.177577 (+1554174us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540035a2d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffe853 00007fa27cfcc60a 00007fa2765527d4 00007fa276552762 00007fa276552691 00007fa27655260c 00007fa2765518c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.177727 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2554875264}
I20260709 13:19:40.225354 16836 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:40.248472 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.249954 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 891) took 1688 ms. Trace:
I20260709 13:19:40.302668 16960 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.233042 16916 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:40.255563 17082 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:19:40.256083 17085 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:40.256469 17087 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:19:40.237015 16654 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.230895 17072 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:40.233444  9939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 147117us
I20260709 13:19:40.243343 17036 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:40.269400 10052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.011s	sys 0.039s
I20260709 13:19:40.264781 10894 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
W20260709 13:19:40.439167 10052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.012s	sys 0.039s
I20260709 13:19:40.440510 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3256) took 1005 ms. Trace:
I20260709 13:19:40.441030 10052 rpcz_store.cc:276] 0709 13:19:39.435052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.525349 (+ 90297us) service_pool.cc:224] Handling call
0709 13:19:39.525888 (+   539us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:39.526444 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:39.526489 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:39.544828 (+ 18339us) log.cc:844] Serialized 52 byte log entry
0709 13:19:40.439916 (+895088us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.439990 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.440114 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.440194 (+    80us) raft_consensus.cc:1685] finished
0709 13:19:40.440211 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.440421 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7828,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":9408,"thread_start_us":875067,"threads_started":1,"wal-append.queue_time_us":772971}
I20260709 13:19:40.267846 10898 rpcz_store.cc:276] 0709 13:19:38.974174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.058532 (+ 84358us) service_pool.cc:224] Handling call
0709 13:19:40.186458 (+1127926us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540064f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.186638 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477880704}
W20260709 13:19:40.276576 17081 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:19:40.276902 16952 consensus_meta.cc:287] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.981s	user 0.014s	sys 0.000s
I20260709 13:19:40.391624 17098 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.281476 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:40.294411 16837 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:40.271390 16978 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:40.295291 16854 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.295542 16964 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.290278 16991 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.298774 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.267656 17067 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:40.301761 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.303143 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.304817 10204 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:40.392948 17033 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:19:40.306218 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.438645 17036 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:40.309022 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:40.309826 17080 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 13:19:40.310482 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.316883 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.319295 16678 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.279733 16860 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:40.351812 10897 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:40.354974 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.355523 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.331051 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.501076 10196 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:40.363960 10047 rpcz_store.cc:276] 0709 13:19:38.847748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.973919 (+126171us) service_pool.cc:224] Handling call
0709 13:19:40.181431 (+1207512us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540037b754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.181631 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533254528}
I20260709 13:19:40.365891 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:40.503230 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 137496us
I20260709 13:19:40.246007 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3211) took 2286 ms. Trace:
I20260709 13:19:40.507234 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3261) took 1024 ms. Trace:
I20260709 13:19:40.508077 10053 rpcz_store.cc:276] 0709 13:19:39.502786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.195441 (+692655us) service_pool.cc:224] Handling call
0709 13:19:40.195877 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.219918 (+ 24041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.219951 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.220000 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.220032 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.506775 (+286743us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b5400369054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:40.506864 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.507100 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54701312}
I20260709 13:19:40.368325 10055 rpcz_store.cc:276] 0709 13:19:38.561320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.561525 (+   205us) service_pool.cc:224] Handling call
0709 13:19:38.605078 (+ 43553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.605334 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:39.103445 (+498111us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.249510 (+1146065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.249576 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:40.249633 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.249735 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.249883 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":305277,"prepare.queue_time_us":785160,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":326921,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":1715,"thread_start_us":1337942,"threads_started":3,"wal-append.queue_time_us":973816}
I20260709 13:19:40.340225 10896 rpcz_store.cc:276] 0709 13:19:38.361000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.361154 (+   154us) service_pool.cc:224] Handling call
0709 13:19:38.364276 (+  3122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.364744 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.364776 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.365336 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.949008 (+583672us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:38.949053 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.949158 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.194511 (+1245353us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:40.339235 (+144724us) raft_consensus.cc:1685] finished
0709 13:19:40.339262 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.339587 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17990,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":5433,"spinlock_wait_cycles":17820544,"thread_start_us":583199,"threads_started":1,"wal-append.queue_time_us":892709}
W20260709 13:19:40.380431 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.349843 17094 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.376531 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:40.349846 16975 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:40.516185 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3197) took 2572 ms (client timeout 2150 ms). Trace:
I20260709 13:19:40.392616 10201 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:19:40.394459 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:40.516723 10123 rpcz_store.cc:269] 0709 13:19:37.944182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.292411 (+348229us) service_pool.cc:224] Handling call
0709 13:19:40.515969 (+2223558us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540036b354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.516100 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3095587712}
I20260709 13:19:40.403083 17106 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.403635 17104 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.405153 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 126168us
I20260709 13:19:40.518064 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.314888 16908 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.349453 16990 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:40.326893 17096 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.416603 10128 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:19:40.491395 16964 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.335003 17099 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:40.539943 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2779871609998281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:19:40.540457 10095 connection.cc:256] RPC call timeout handler was delayed by -2.2581871609997961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
I20260709 13:19:40.374207 17100 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.374112 17107 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:40.277163 16971 consensus_meta.cc:287] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.923s	user 0.012s	sys 0.002s
I20260709 13:19:40.373117 16957 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.358062 16884 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.468286 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3248) took 1067 ms. Trace:
I20260709 13:19:40.565523 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.565676 10054 rpcz_store.cc:276] 0709 13:19:39.400973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.401216 (+   243us) service_pool.cc:224] Handling call
0709 13:19:39.401568 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.401803 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:39.411408 (+  9605us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.766539 (+355131us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.768586 (+  2047us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.190120 (+421534us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b1800014980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:19:40.331784 (+141664us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b1800014980. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:40.467548 (+135764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.467646 (+    98us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.467725 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.467833 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.468189 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":403691,"prepare.queue_time_us":46876,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":137,"raft.queue_time_us":441450,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":20317,"spinlock_wait_cycles":151268224,"thread_start_us":617362,"threads_started":2,"wal-append.queue_time_us":668183}
I20260709 13:19:40.475971 10895 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:19:40.457657 17079 consensus_queue.cc:1048] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:19:40.568454 17079 consensus_queue.cc:1243] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:40.569339 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.305758 16727 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.570087 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3259) took 1129 ms. Trace:
I20260709 13:19:40.307433 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.570540 10051 rpcz_store.cc:276] 0709 13:19:39.440155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.808203 (+368048us) service_pool.cc:224] Handling call
0709 13:19:39.850931 (+ 42728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.467455 (+616524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.467494 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:40.468046 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:19:40.556192 (+ 88146us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:40.556251 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.556356 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.569800 (+ 13444us) raft_consensus.cc:1685] finished
0709 13:19:40.569829 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.570003 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":343368,"raft.queue_time_us":352697,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":7746,"thread_start_us":360181,"threads_started":2,"wal-append.queue_time_us":99477}
I20260709 13:19:40.393420 16960 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:40.488828 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95746us
I20260709 13:19:40.572171 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.570103 10895 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:40.427636 17073 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:19:40.490005 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:40.583249 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:40.493744 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:40.587924 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:40.468760 16940 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.021s	user 0.014s	sys 0.000s
W20260709 13:19:40.511768 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.484728 17067 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:40.327926 17068 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.519526 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.520123 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.521222 10131 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.507429 10122 rpcz_store.cc:276] 0709 13:19:37.959737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.502193 (+542456us) service_pool.cc:224] Handling call
0709 13:19:38.502509 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.503058 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.503101 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.508530 (+  5429us) log.cc:844] Serialized 26 byte log entry
0709 13:19:39.309627 (+801097us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:39.830277 (+520650us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.830631 (+   354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.830756 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.245746 (+414990us) raft_consensus.cc:1685] finished
0709 13:19:40.245771 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.245936 (+   165us) inbound_call.cc:177] Queueing success response
0709 13:19:40.368285 (+122349us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b1800012ca0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
Metrics: {"prepare.queue_time_us":275761,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":253905,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":26222,"spinlock_wait_cycles":11430144,"thread_start_us":1140929,"threads_started":2,"wal-append.queue_time_us":1017033}
I20260709 13:19:40.612388 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.489902 16363 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:40.425287 17085 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.425279 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:40.529675 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 248297us
I20260709 13:19:40.483353 16854 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.469651 17098 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.532416 16696 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.625262 17085 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.533928 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1058) took 1451 ms. Trace:
I20260709 13:19:40.626458 10131 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:40.626345 10900 rpcz_store.cc:276] 0709 13:19:39.082808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.314608 (+231800us) service_pool.cc:224] Handling call
0709 13:19:40.126192 (+811584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.241298 (+115106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.533671 (+292373us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54006ed7d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:40.533847 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852119296}
I20260709 13:19:40.535494 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.464839 16952 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.543802 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:40.628631 10123 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.628808 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.630117 16952 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:40.630360 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.546998 16908 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:40.625100 17098 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.483783 16792 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:19:40.549757 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:40.632366 10128 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.632572 16363 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:40.633785 10128 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:19:40.550052 17036 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.522308 16995 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:40.551997 10095 connection.cc:256] RPC call timeout handler was delayed by -1.6059621189999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
W20260709 13:19:40.636191 10095 connection.cc:256] RPC call timeout handler was delayed by -1.5273453049999262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
I20260709 13:19:40.637555 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:40.552495 17045 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.642632 17045 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.644132 16900 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.644693 16900 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.488363 16836 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.552968 17099 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.555364 16746 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:40.555343 17106 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.525028 16990 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.557104 16964 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.544672 16654 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.575883 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.577539 16884 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.577661 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.579741 10124 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 12: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.669195 10895 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:19:40.609822 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.613126 10122 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:40.672405 10122 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:40.673832 10122 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:19:40.616451 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 239984us
I20260709 13:19:40.618147 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.620791 10170 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:40.677249 10199 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:19:40.677452 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56748.6us
I20260709 13:19:40.678303 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.565886 17107 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.678653 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:40.627948 10131 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:40.631524 16908 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.632370 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.638998 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:40.640633 10128 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:19:40.640848 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:40.647816 10123 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:40.572767 17101 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.683379 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:40.683265 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:40.684545 10053 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:40.684790 10048 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:40.648430 17098 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:40.621357 17051 consensus_meta.cc:287] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.238s	user 0.015s	sys 0.002s
I20260709 13:19:40.572924 16747 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:19:40.659348 10203 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
W20260709 13:19:40.661823 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.665927 10898 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:40.667433 10197 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:40.697909 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.664867 10196 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:19:40.647836 17110 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:19:40.687285 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.537497 17104 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.690549 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:40.571422 16971 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.692128 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:40.699585 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:40.658843 16990 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:40.700631 10047 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:40.765863 10200 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:40.765578 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.703436 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.773038 10204 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:40.688665 17079 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.619632 16940 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.711545 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.564549 17100 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.666164 17045 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.689236 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.709156 16746 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:40.769425 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:40.687846 16908 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.775635 10200 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:40.792577 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 100542us
I20260709 13:19:40.776075 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:40.793654 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:40.721580 10205 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: [15.13-18.14]
I20260709 13:19:40.692740 17094 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:40.766384 17031 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.656901 17099 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.770853 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:40.797919 10055 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:40.798663 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 108209us
I20260709 13:19:40.687080 16654 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.801862 10892 raft_consensus.cc:2393] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 13: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:40.800439 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:40.806708 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:40.708477 17051 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:40.808637 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:40.700419 16552 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:19:40.817766 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.781332 17107 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.795449 16916 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:40.791576 17079 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:40.835878 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.694621 16837 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:19:40.824515 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:40.821609 10205 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:40.843386 10123 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:40.825642 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.805168 17085 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.844610 10128 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.693799 17101 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.836804 10898 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:40.825932 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:40.790810 17106 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.831656 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.806473 17098 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:40.827215 17045 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:40.806875 16991 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:40.797439 16884 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:40.860204 10131 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.839267 17114 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:40.864199 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.841357 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.830091 17094 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:40.873632 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.944042 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:40.866357 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 69935.8us
W20260709 13:19:40.907302 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140797us
I20260709 13:19:40.913028 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.947078 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.947861 10205 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:40.915949 10129 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:40.913910 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.927497 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.934317 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.936055 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:40.952171 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:40.942368 10199 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:40.942174 10128 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:40.954265 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.956373 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.955614 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.957513 10128 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.958715 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.960170 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.961937 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.962807 10204 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:40.947422 16960 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:40.964663 10896 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:40.951460 10131 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.804926 17031 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:40.843657 17113 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 13:19:40.941607 16884 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:40.943878 17094 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.926107 17098 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:40.972115 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.984687 10128 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:40.974565 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:40.985790 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.986552 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:40.973369 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.988004 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.988917 10128 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:40.944936 10055 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:19:40.993888 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49601.1us
I20260709 13:19:40.994347 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:40.995061 10123 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:40.996543 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:40.867287 17085 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:40.996758 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:41.031592 10898 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:40.998422 17130 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:41.033077 17085 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.033401 17130 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:41.034842 17100 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.040660 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.041968 10123 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:41.045130 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.002146 16678 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.007436 16552 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.015473 17105 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:41.055666 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.057092 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:40.948772 17114 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.058565 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.060519 17114 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.017427 17117 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:41.061779 17117 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.062400 17117 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:40.967263 17119 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:19:41.018229 16991 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:41.015568 16964 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:41.071925 16748 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.000s
I20260709 13:19:41.071359 16552 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.072606 16748 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.073982 10205 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:40.979579 17079 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.075387 16748 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:40.979709 16884 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:41.023888 16654 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:41.089941 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 290369us
I20260709 13:19:40.960960 16916 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:40.985189 17094 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.027885 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:41.046830 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.001026 17031 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.032521 17099 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.047374 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:41.051085 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76916.4us
I20260709 13:19:40.989935 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.034600 17130 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.052557 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.052604 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.022848 16940 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.034844 16746 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:41.117759 10047 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:41.049335 17123 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:41.057814 10204 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:41.016786 17104 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.020798 17098 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.074685 16552 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.148214 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:41.099536 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.101444 16678 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:41.111725 10128 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:41.149559 10047 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:41.150954 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:41.119431 10055 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:41.106635 17105 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:41.120370 16746 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:41.115548 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 68330.2us
I20260709 13:19:41.138113 16991 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:41.121791 17116 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:41.121681 17123 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:41.138839 16978 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.003s
I20260709 13:19:41.158479 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.166216 10128 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:41.137511 17078 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.790s	user 0.001s	sys 0.014s
I20260709 13:19:41.161083 10048 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:19:41.134924 17075 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.007s
I20260709 13:19:41.169704 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:41.172667 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:41.171669 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.111442 17085 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:19:41.156733 17105 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:41.165354 16991 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.113629 16900 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.171578 16978 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.173419 17123 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:41.177881 10047 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:19:41.179570 10048 pending_rounds.cc:85] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:19:41.174186 17078 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:41.206318 10048 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:19:41.206709 10048 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:41.182720 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.207747 10128 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:41.195871 10170 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:41.242350 10123 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:41.174181 17116 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:41.245617 17093 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.118s	user 0.000s	sys 0.004s
I20260709 13:19:41.209281 16363 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:19:41.190618 17105 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.247853 16990 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.202863 16991 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:41.175779 17075 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.254714 16908 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.279497 10053 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:41.243274 16978 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.285147 17116 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.286456 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:41.256841 10168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.008s	sys 0.012s
I20260709 13:19:41.286638 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:41.309089 10898 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:41.308437 10168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.009s	sys 0.013s
I20260709 13:19:41.209952 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.289417 17114 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:41.290998 17116 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:41.293114 16991 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.296020 10198 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:41.298591 16990 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.300655 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:41.300863 17126 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:41.293792 17075 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.250952 17117 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:41.310703 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 693024us
W20260709 13:19:41.289206 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93509.2us
I20260709 13:19:41.316445 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:41.304448  9954 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 122) took 24584 ms (client timeout 15000 ms). Trace:
I20260709 13:19:41.349359 10196 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:41.359180 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.315483 17114 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.362259 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:41.373075 17104 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:41.331769 17126 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:41.375708  9951 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 8 to 9, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 9 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.354192 17116 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.384227 16991 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.364626 17114 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.384649 17072 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:41.386049 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:41.386421 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:41.434688 10898 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:41.303121 16884 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.388749 16852 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:41.391098 17117 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.391353 17079 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:41.361127 17094 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.392213 17132 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:41.386163 16908 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.456528 17132 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:41.410004 10896 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:19:41.457007 17132 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:41.457890 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:41.457979 17132 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.458554 10896 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.411245 10900 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: [21.12-23.13]
I20260709 13:19:41.460132 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:41.460492 16884 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.461645 10900 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:41.415812 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3266) took 1910 ms. Trace:
I20260709 13:19:41.417588 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3666) took 1111 ms. Trace:
I20260709 13:19:41.463855 16748 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:41.464207 10198 rpcz_store.cc:276] 0709 13:19:40.305811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.305943 (+   132us) service_pool.cc:224] Handling call
0709 13:19:41.417348 (+1111405us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54001d1354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.417513 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860815232}
I20260709 13:19:41.423151 17104 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:41.460394 10204 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:41.465588 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.466269 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.419518 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.419598 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:41.420557 10128 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.398895 16746 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:41.433542 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46808.6us
I20260709 13:19:41.472638 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.438097 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:41.425905 17123 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.437731 17072 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.419445 17032 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.405643 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:41.415562 16900 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.466099 17104 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.430001 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:41.468591 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.466885 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.474038 10899 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.477370 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:41.476219 10203 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:41.478406 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.479324 10898 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 24.
I20260709 13:19:41.480644 10204 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:41.491326 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61476.6us
I20260709 13:19:41.481863 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:41.492416 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:41.496582 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:41.489821 10198 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:41.497464 10131 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:41.499055 16860 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.500440 17073 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:41.500048 16860 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.501127 17073 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:41.460492 17116 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.502211 17073 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.503356 16940 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:41.480445 17132 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.503937 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:41.509784 17114 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:41.510767 10900 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:41.512238 17123 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:41.513201 16860 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.514348 16746 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.515373 10196 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:41.517014 16940 consensus_queue.cc:1243] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:41.523772 16940 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:41.524411 16940 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:41.525945 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:41.526932 10900 raft_consensus.cc:2368] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24.
I20260709 13:19:41.527338 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1047) took 2713 ms. Trace:
I20260709 13:19:41.528200 10169 leader_election.cc:400] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24.
I20260709 13:19:41.528656 10169 leader_election.cc:403] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.525900 16940 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.529665 16940 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:41.463670 16852 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 13:19:41.519232 17101 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:41.527791 10897 rpcz_store.cc:276] 0709 13:19:38.813472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.813773 (+   301us) service_pool.cc:224] Handling call
0709 13:19:38.814100 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.499509 (+1685409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.526945 (+1027436us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54004650d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:41.527227 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3535918848}
I20260709 13:19:41.537664 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:41.554515 10196 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:41.467816 16971 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:41.564706 10896 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:41.567176 10895 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 18. Ops: []
I20260709 13:19:41.571089 10202 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
W20260709 13:19:41.526010 17111 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.975s	user 0.004s	sys 0.015s
I20260709 13:19:41.496150 17032 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:41.593047 17109 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.016s	sys 0.012s
I20260709 13:19:41.548204 17119 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:19:41.593794 17109 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.016s	sys 0.012s
I20260709 13:19:41.602581 17071 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:41.605005 17071 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:41.606135 17071 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.579011 17136 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
W20260709 13:19:41.607048 17135 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
W20260709 13:19:41.619180 17112 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.067s	user 0.007s	sys 0.015s
W20260709 13:19:41.623302 17143 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:19:41.633102 17120 raft_consensus.cc:3055] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:41.633929 17120 raft_consensus.cc:740] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:41.639874 17137 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:19:41.641428 17139 log.cc:927] Time spent T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:19:41.680357 17149 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:19:41.388422  9939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d51ad94710c348889d417f7f00032948 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:19:41.716857  9939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 328281us
W20260709 13:19:41.341470  9954 rpcz_store.cc:269] 0709 13:19:16.720049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.720223 (+   174us) service_pool.cc:224] Handling call
0709 13:19:41.304300 (+24584077us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:35.375966 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:35.507981 (+132015us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:35.508059 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:19:35.508109 (+    50us) tablet.cc:662] Decoding operations
0709 13:19:35.509596 (+  1487us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:35.509739 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:35.509772 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:19:35.510963 (+  1191us) tablet.cc:701] Row locks acquired
0709 13:19:35.511009 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:19:35.536657 (+ 25648us) write_op.cc:270] Start()
0709 13:19:35.536868 (+   211us) write_op.cc:276] Timestamp: P: 1783603175526034 usec, L: 0
0709 13:19:35.536910 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:19:35.561451 (+ 24541us) log.cc:844] Serialized 4450 byte log entry
0709 13:19:36.727927 (+1166476us) op_driver.cc:464] REPLICATION: finished
0709 13:19:37.097445 (+369518us) write_op.cc:301] APPLY: starting
0709 13:19:37.110849 (+ 13404us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:37.313862 (+203013us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:37.314029 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:37.558255 (+244226us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:37.558756 (+   501us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:19:37.558857 (+   101us) write_op.cc:312] APPLY: finished
0709 13:19:37.610321 (+ 51464us) log.cc:844] Serialized 105 byte log entry
0709 13:19:37.610596 (+   275us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:37.610767 (+   171us) write_op.cc:454] Released schema lock
0709 13:19:37.610838 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:37.611520 (+   682us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:37.611551 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7133016,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":291722,"apply.run_cpu_time_us":23960,"apply.run_wall_time_us":555638,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50019,"num_ops":11,"prepare.queue_time_us":323621,"prepare.run_cpu_time_us":21521,"prepare.run_wall_time_us":614720,"raft.queue_time_us":84547,"raft.run_cpu_time_us":10163,"raft.run_wall_time_us":119744,"replication_time_us":1190804,"thread_start_us":563265,"threads_started":4,"wal-append.queue_time_us":509551,"wal-append.run_cpu_time_us":17772,"wal-append.run_wall_time_us":2512280}]]}
I20260709 13:19:41.716763  9951 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 8 to 9, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 9 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.458467 10019 leader_election.cc:400] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18.
I20260709 13:19:41.448072 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:41.463653 10049 rpcz_store.cc:276] 0709 13:19:39.505324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.360218 (+854894us) service_pool.cc:224] Handling call
0709 13:19:40.360506 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.625946 (+265440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.625979 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.626028 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.626060 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.415536 (+789476us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54002350d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:41.415588 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.415734 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609923328}
I20260709 13:19:41.733745 10047 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:41.736801 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:41.738991 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.731086  9951 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 9 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 10 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.745261 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:41.441669 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:41.748107 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.748663 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3260) took 2308 ms. Trace:
W20260709 13:19:41.750638 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 228430us
W20260709 13:19:41.752308 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 224105us
I20260709 13:19:41.752966 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46788 (request call id 2624) took 1054 ms. Trace:
W20260709 13:19:41.755930 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.492213 16363 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 13:19:41.769747 16727 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:42.018816 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:41.789230 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:42.018042 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 559565us
I20260709 13:19:41.752445 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3230) took 1421 ms. Trace:
W20260709 13:19:42.120786 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.804443 10891 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:41.796636 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3269) took 2121 ms. Trace:
I20260709 13:19:41.798148 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57108 (request call id 3670) took 1488 ms. Trace:
I20260709 13:19:41.776022 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3258) took 2335 ms. Trace:
W20260709 13:19:41.985097 10095 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.000s	sys 0.000s
I20260709 13:19:41.780220 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3233) took 1425 ms. Trace:
I20260709 13:19:41.776566 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.239353 10131 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:41.778234 10124 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3229) took 3156 ms (client timeout 2366 ms). Trace:
I20260709 13:19:42.240392 16957 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.241160 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.241331 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:42.242434 10131 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:42.245316 10095 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.004s	sys 0.007s
W20260709 13:19:41.798518 10892 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 769) took 1593 ms (client timeout 1711 ms). Trace:
I20260709 13:19:42.247768 17067 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.190907 10053 rpcz_store.cc:276] 0709 13:19:40.698138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.699280 (+  1142us) service_pool.cc:224] Handling call
0709 13:19:41.752256 (+1052976us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54002369d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.752636 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161460608}
W20260709 13:19:42.240216 10124 rpcz_store.cc:269] 0709 13:19:38.621260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.654586 (+ 33326us) service_pool.cc:224] Handling call
0709 13:19:41.777561 (+3122975us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.777946 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4305114368}
W20260709 13:19:41.803069 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 517) took 3258 ms (client timeout 1834 ms). Trace:
I20260709 13:19:42.227595 10050 rpcz_store.cc:276] 0709 13:19:39.439928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.782081 (+342153us) service_pool.cc:224] Handling call
0709 13:19:39.782532 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.215800 (+433268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.215836 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.215881 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.215930 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.746847 (+1530917us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54000d8c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:41.746985 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.775778 (+ 28793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995834752}
I20260709 13:19:42.258976 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.252974 17067 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:42.219146 10127 rpcz_store.cc:276] 0709 13:19:40.351708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.353972 (+  2264us) service_pool.cc:224] Handling call
0709 13:19:40.354339 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.354610 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:40.381119 (+ 26509us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.618583 (+1237464us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:41.750507 (+131924us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.751084 (+   577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:41.751207 (+   123us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:41.751396 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.751595 (+   199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.752031 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2384758,"prepare.queue_time_us":1500903,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":18343,"raft.run_cpu_time_us":632,"raft.run_wall_time_us":5839,"spinlock_wait_cycles":15895424,"thread_start_us":101757,"threads_started":1,"wal-append.queue_time_us":170920}
I20260709 13:19:42.260314 17067 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.260401 10129 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.261914 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:42.259364 10863 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.528s (SENT) [suppressed 27 similar messages]
I20260709 13:19:42.273576 17101 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.933777 16854 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:42.274708 17101 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.278522 17101 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:42.279520 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 807747us
I20260709 13:19:42.283547 17036 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.283735 16968 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:42.284516 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.327401 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3692) took 1158 ms. Trace:
I20260709 13:19:42.328284 10124 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:42.328689 16968 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:42.329404 16968 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:42.329591 10054 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:42.329902 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.225750 10202 rpcz_store.cc:276] 0709 13:19:40.309926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.342953 (+ 33027us) service_pool.cc:224] Handling call
0709 13:19:41.797465 (+1454512us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002fc1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.797828 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350712704}
I20260709 13:19:42.289258 10053 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:19:42.289994 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:42.180557 10056 rpcz_store.cc:276] 0709 13:19:39.482358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.174997 (+692639us) service_pool.cc:224] Handling call
0709 13:19:40.285924 (+110927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.286296 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.286341 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:40.287033 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 13:19:40.457071 (+170038us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.457127 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.457213 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.525264 (+1068051us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:41.747575 (+222311us) raft_consensus.cc:1685] finished
0709 13:19:41.747647 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.748361 (+   714us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18770304,"thread_start_us":126310,"threads_started":1,"wal-append.queue_time_us":191794}
I20260709 13:19:42.291345 17085 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:42.330894 10124 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:42.230964 10125 rpcz_store.cc:276] 0709 13:19:40.354367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.374450 (+ 20083us) service_pool.cc:224] Handling call
0709 13:19:40.374817 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.814041 (+439224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.814079 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.814132 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:40.814178 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.779635 (+965457us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540014ad54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:41.779718 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.780029 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009521536}
I20260709 13:19:42.295914 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1046) took 3482 ms. Trace:
W20260709 13:19:42.302065 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61031.4us
I20260709 13:19:42.305038 17104 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.309626 16967 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:42.309610 10051 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:42.314661 17126 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:41.613476 17071 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.347090 17079 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:42.347915 16854 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:42.348156 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:42.114476 17114 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:42.177141 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:42.349681 17114 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:42.350684 10198 raft_consensus.cc:606] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:42.335029 16840 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:42.349792 10128 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.882s	user 0.014s	sys 0.005s
I20260709 13:19:42.351682 16840 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.351941 10128 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:42.352213 16840 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:42.324724 17072 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.848s	user 0.016s	sys 0.003s
I20260709 13:19:42.353168 17072 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.302340  9951 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 8 to 9, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 9 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:42.335158 10895 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:19:42.358739 16940 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.828s	user 0.015s	sys 0.006s
I20260709 13:19:41.770035 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:42.335891 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46153.8us
W20260709 13:19:42.256814 10894 rpcz_store.cc:269] 0709 13:19:38.544322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.544578 (+   256us) service_pool.cc:224] Handling call
0709 13:19:41.802462 (+3257884us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400953b54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.802767 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3939448576}
I20260709 13:19:42.336777 10893 rpcz_store.cc:276] 0709 13:19:38.813124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.841851 (+ 28727us) service_pool.cc:224] Handling call
0709 13:19:42.239770 (+3397919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.240175 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.240203 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.240285 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:42.240355 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.240459 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.295452 (+ 54993us) spinlock_profiling.cc:243] Waited 3.39 s on lock 0x7b5400953b50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:42.295682 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7807659264}
W20260709 13:19:42.251089 10892 rpcz_store.cc:269] 0709 13:19:40.205342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.205670 (+   328us) service_pool.cc:224] Handling call
0709 13:19:41.797761 (+1592091us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54006f2f54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:41.798192 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062762240}
I20260709 13:19:42.348399 17079 consensus_queue.cc:1243] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:42.350771 17114 consensus_queue.cc:1243] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer e9a849ac7eda426abfca97cf8b15da0f log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:19:42.333619 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.344599 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3746) took 1106 ms. Trace:
I20260709 13:19:42.362566 10124 rpcz_store.cc:276] 0709 13:19:41.230174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.329370 (+1099196us) service_pool.cc:224] Handling call
0709 13:19:42.336301 (+  6931us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:42.337204 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50275.7us
I20260709 13:19:42.224520 10052 rpcz_store.cc:276] 0709 13:19:39.675042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.446714 (+771672us) service_pool.cc:224] Handling call
0709 13:19:40.447242 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.804615 (+357373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.804660 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.804733 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:40.804784 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.795923 (+991139us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400136354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:41.796036 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.796373 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6158,"raft.queue_time_us":583,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":36460,"spinlock_wait_cycles":820592000}
I20260709 13:19:42.331036 16968 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.364812 17072 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.365218 17105 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:42.365849 10052 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.366348 16968 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.369516 10047 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.370885 17067 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.367415 10055 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:42.373936 17067 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:42.388410 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.397480 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49535.6us
I20260709 13:19:42.399626 17067 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:42.400508 17067 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:42.401836 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.401991 17067 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.414076 17114 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:42.416328 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.417430 10124 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:42.417630 17043 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:42.418418 17043 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.418969 17043 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:42.420197 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.421380 10125 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:42.385457 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:42.418877 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:42.423914 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 654377us
I20260709 13:19:42.328136 10199 rpcz_store.cc:276] 0709 13:19:41.169206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.169375 (+   169us) service_pool.cc:224] Handling call
0709 13:19:41.172099 (+  2724us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:41.204790 (+ 32691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.204829 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:41.263552 (+ 58723us) log.cc:844] Serialized 78 byte log entry
0709 13:19:41.419960 (+156408us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:41.420011 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.420097 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.766164 (+346067us) raft_consensus.cc:1685] finished
0709 13:19:41.766238 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.327077 (+560839us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35971,"raft.queue_time_us":33140,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":29920,"thread_start_us":142574,"threads_started":1,"wal-append.queue_time_us":288348}
I20260709 13:19:42.430851 10128 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:42.434846 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3752) took 1019 ms. Trace:
W20260709 13:19:42.431471 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 170779us
I20260709 13:19:42.435472 10125 rpcz_store.cc:276] 0709 13:19:41.410822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.419781 (+1008959us) service_pool.cc:224] Handling call
0709 13:19:42.430563 (+ 10782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.430699 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1061) took 2194 ms. Trace:
I20260709 13:19:42.438047 10891 rpcz_store.cc:276] 0709 13:19:40.371564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.377762 (+  6198us) service_pool.cc:224] Handling call
0709 13:19:41.803450 (+1425688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.219695 (+416245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.382852 (+163157us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400953b50. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:42.424708 (+ 41856us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3277339520}
I20260709 13:19:42.446298 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3753) took 1026 ms. Trace:
I20260709 13:19:42.451140 10124 rpcz_store.cc:276] 0709 13:19:41.411223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.418422 (+1007199us) service_pool.cc:224] Handling call
0709 13:19:42.437433 (+ 19011us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.453027 17126 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:42.453588 17112 log.cc:927] Time spent T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:42.454341 16969 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:42.459821 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.460623 10047 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:42.461308 10128 raft_consensus.cc:2393] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:42.462356 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3737) took 1364 ms. Trace:
I20260709 13:19:42.463402 10054 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.461496 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 901) took 1039 ms. Trace:
I20260709 13:19:42.468243 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:42.469635 17072 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:42.471657 17072 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.472400 17072 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.472731 16840 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.474946 17072 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:42.462805 10128 rpcz_store.cc:276] 0709 13:19:41.097953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.459251 (+1361298us) service_pool.cc:224] Handling call
0709 13:19:42.462264 (+  3013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.466835 10047 rpcz_store.cc:276] 0709 13:19:41.422153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.735970 (+313817us) service_pool.cc:224] Handling call
0709 13:19:41.744827 (+  8857us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:19:42.461337 (+716510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4075392}
I20260709 13:19:42.493153 10131 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:42.494886 10051 raft_consensus.cc:2410] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:42.496012 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 898) took 1074 ms. Trace:
I20260709 13:19:42.496531 10051 rpcz_store.cc:276] 0709 13:19:41.421189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.788484 (+367295us) service_pool.cc:224] Handling call
0709 13:19:42.495926 (+707442us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:42.497964 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43794.6us
I20260709 13:19:42.499010 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3259) took 1137 ms. Trace:
I20260709 13:19:42.499938 10129 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:42.505946 10052 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:42.506951 10055 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:42.506099 10056 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:42.500644 10131 rpcz_store.cc:276] 0709 13:19:41.357064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.240498 (+883434us) service_pool.cc:224] Handling call
0709 13:19:42.494034 (+253536us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.508633 16940 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.509423 16940 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.508687 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:42.511857 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 903) took 1085 ms. Trace:
I20260709 13:19:42.512377 10055 rpcz_store.cc:276] 0709 13:19:41.426198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.434272 (+  8074us) service_pool.cc:224] Handling call
0709 13:19:42.507612 (+1073340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.514870 10123 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:42.511875 10051 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:42.501730 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3260) took 1144 ms. Trace:
I20260709 13:19:42.525075 10129 rpcz_store.cc:276] 0709 13:19:41.357315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.017646 (+660331us) service_pool.cc:224] Handling call
0709 13:19:42.501636 (+483990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.530417 10054 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:42.543196 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.544497 10131 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:42.545061 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3731) took 1581 ms. Trace:
I20260709 13:19:42.545446 10131 rpcz_store.cc:276] 0709 13:19:40.963879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.517812 (+1553933us) service_pool.cc:224] Handling call
0709 13:19:42.544975 (+ 27163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.547132 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3236) took 2190 ms. Trace:
I20260709 13:19:42.548270 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3285) took 1264 ms. Trace:
I20260709 13:19:42.548946 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.550069 10124 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:42.548835 10048 rpcz_store.cc:276] 0709 13:19:41.167396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.381095 (+213699us) service_pool.cc:224] Handling call
0709 13:19:41.381461 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.381689 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:41.732903 (+351214us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.247385 (+514482us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.344048 (+ 96663us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:42.386809 (+ 42761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.386917 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.431903 (+ 44986us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4077,"prepare.queue_time_us":613,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":85007,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":4701,"thread_start_us":763878,"threads_started":2,"wal-append.queue_time_us":634103}
I20260709 13:19:42.547636 10130 rpcz_store.cc:276] 0709 13:19:40.356449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.356635 (+   186us) service_pool.cc:224] Handling call
0709 13:19:40.358284 (+  1649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.373941 (+ 15657us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:41.533435 (+1159494us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18004583e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:19:41.533798 (+   363us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.004787 (+470989us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.445784 (+440997us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.455285 (+  9501us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:42.455346 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.455459 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:42.546466 (+ 91007us) raft_consensus.cc:1685] finished
0709 13:19:42.546599 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.546858 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21236,"prepare.queue_time_us":1191243,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":4908,"raft.queue_time_us":13083,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":13957,"spinlock_wait_cycles":39769984,"thread_start_us":460644,"threads_started":1,"wal-append.queue_time_us":790129}
I20260709 13:19:42.552456 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.553479 10125 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:42.554139 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3715) took 1621 ms. Trace:
I20260709 13:19:42.554514 10125 rpcz_store.cc:276] 0709 13:19:40.932230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.523478 (+1591248us) service_pool.cc:224] Handling call
0709 13:19:42.554060 (+ 30582us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.543298 17094 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:42.562217 16727 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24.
I20260709 13:19:42.550711 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3254) took 1211 ms. Trace:
I20260709 13:19:42.564218 10124 rpcz_store.cc:276] 0709 13:19:41.339702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.527105 (+1187403us) service_pool.cc:224] Handling call
0709 13:19:42.550631 (+ 23526us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:42.564713 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49182us
W20260709 13:19:42.584220 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 325089us
I20260709 13:19:42.564951 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:42.598088 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.595920 16968 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:19:42.614897 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 122079us
I20260709 13:19:42.618049 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.619444 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.600826  9951 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 9 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 10 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:42.625480 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:42.634095 16840 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:42.634362 17104 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:42.635357 17104 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:42.637032 17104 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.640378 16940 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.641515 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.642894 10124 raft_consensus.cc:2393] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:42.643236 10122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.029s	sys 0.052s
W20260709 13:19:42.643743 10122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.029s	sys 0.053s
I20260709 13:19:42.643569 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3759) took 1230 ms. Trace:
I20260709 13:19:42.644241 10124 rpcz_store.cc:276] 0709 13:19:41.413066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.597329 (+1184263us) service_pool.cc:224] Handling call
0709 13:19:42.643471 (+ 46142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.655722 17094 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:42.656071 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.656478 10126 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:19:42.646793  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:42.645965 16968 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.656443 17094 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.657028 10126 pending_rounds.cc:85] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:19:42.660795 10124 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:42.661991 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3764) took 1247 ms. Trace:
I20260709 13:19:42.662376 10124 rpcz_store.cc:276] 0709 13:19:41.417516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.655512 (+1237996us) service_pool.cc:224] Handling call
0709 13:19:42.661898 (+  6386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.664604 10129 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:42.662650 10128 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:42.675576 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3741) took 1546 ms. Trace:
I20260709 13:19:42.676227 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3745) took 1468 ms. Trace:
I20260709 13:19:42.676795 10128 rpcz_store.cc:276] 0709 13:19:41.205599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.585907 (+1380308us) service_pool.cc:224] Handling call
0709 13:19:42.674074 (+ 88167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.676257 10129 rpcz_store.cc:276] 0709 13:19:41.119541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.576494 (+1456953us) service_pool.cc:224] Handling call
0709 13:19:42.665538 (+ 89044us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.665091 10126 pending_rounds.cc:107] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:19:42.681653 10126 raft_consensus.cc:2889] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:42.685222 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.685714 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.686669 17036 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:42.687789 17036 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:42.686762 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55432.8us
I20260709 13:19:42.691932 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:42.692415 17036 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:42.692756 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 45941.2us
I20260709 13:19:42.693830 17036 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:42.694648 17036 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.696293 17096 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.697033 17096 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.697510 17036 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.701519 16968 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:42.702280 16968 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:42.707417 10169 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:42.660801 17094 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:42.685941 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 60714.4us
I20260709 13:19:42.686825 10128 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.764797 16840 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:42.691963 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:42.703022 16968 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.778575 16968 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.779223 16852 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:42.780241 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.791940 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:42.771556 10096 connection.cc:256] RPC call timeout handler was delayed by -2.2990946410000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 13:19:42.764992 17119 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:19:42.794361  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:42.796643 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:42.801777 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3760) took 1388 ms. Trace:
I20260709 13:19:42.802464 10129 rpcz_store.cc:276] 0709 13:19:41.413431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.680454 (+1267023us) service_pool.cc:224] Handling call
0709 13:19:42.801669 (+121215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.800292 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:42.787190 16852 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:42.787473 16840 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:42.830600 10049 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:42.774602 17043 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.836117 10894 raft_consensus.cc:2368] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
W20260709 13:19:42.839053 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.840353 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.845575 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 153776us
I20260709 13:19:42.846395 17109 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.815548 17126 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:19:42.849202 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 475651us
I20260709 13:19:42.846906 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:42.837632 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:42.853511 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62236.7us
I20260709 13:19:42.854171 17136 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 13:19:42.854723 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:42.857196 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:42.858052 10129 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:42.855036 17071 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.004s
I20260709 13:19:42.859879 17071 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:42.859699 16836 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 13:19:42.861294 10127 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.865953 17130 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:42.871518 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49037.9us
I20260709 13:19:42.874094 17109 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.875370 10892 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:42.875494 17036 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:42.876338 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:42.876917 17109 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:42.877367 10894 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:42.856642 17101 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:42.878382 17157 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.878207 16840 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.879344 10124 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:42.881402 16792 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:42.877497 17036 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:42.831823 16969 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.864671 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3763) took 1450 ms. Trace:
I20260709 13:19:42.892524 10129 rpcz_store.cc:276] 0709 13:19:41.414400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.807556 (+1393156us) service_pool.cc:224] Handling call
0709 13:19:42.864588 (+ 57032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.814911 16968 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:42.889339 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3258) took 1532 ms. Trace:
I20260709 13:19:42.893908 10124 rpcz_store.cc:276] 0709 13:19:41.356826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.663180 (+1306354us) service_pool.cc:224] Handling call
0709 13:19:42.889254 (+226074us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.894419 17126 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.895004 10893 raft_consensus.cc:1240] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: [21.12-23.13]
I20260709 13:19:42.895272 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.897336 17126 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:42.883873 16727 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:42.896106 10124 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:42.900141 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3748) took 1637 ms. Trace:
I20260709 13:19:42.900556 10124 rpcz_store.cc:276] 0709 13:19:41.262626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.894773 (+1632147us) service_pool.cc:224] Handling call
0709 13:19:42.900069 (+  5296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.911732 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:42.913548 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3761) took 1499 ms. Trace:
I20260709 13:19:42.914191 10128 rpcz_store.cc:276] 0709 13:19:41.413742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.685160 (+1271418us) service_pool.cc:224] Handling call
0709 13:19:42.913436 (+228276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.918356 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:42.922942 16852 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.924386 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:42.898511 17130 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:42.926614 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:42.938666 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:42.940846 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.942421 10129 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:42.893375 17109 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.945025 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.904517 16792 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:42.949998 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.954241 10128 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:42.957688 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3766) took 1539 ms. Trace:
I20260709 13:19:42.958214 10129 rpcz_store.cc:276] 0709 13:19:41.418222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.893464 (+1475242us) service_pool.cc:224] Handling call
0709 13:19:42.957604 (+ 64140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.959399 10125 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:42.961405 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:42.924856 16852 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:42.918699 17085 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:42.964159 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3757) took 1551 ms (client timeout 1961 ms). Trace:
W20260709 13:19:42.973306 10125 rpcz_store.cc:269] 0709 13:19:41.412464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.925128 (+1512664us) service_pool.cc:224] Handling call
0709 13:19:42.964066 (+ 38938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.902262 17157 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:42.974406 10891 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:42.955039 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3767) took 1536 ms. Trace:
I20260709 13:19:42.971900 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.918295 16727 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:42.970655 17031 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:42.940194 16940 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:42.976125 10128 rpcz_store.cc:276] 0709 13:19:41.418538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.920426 (+1501888us) service_pool.cc:224] Handling call
0709 13:19:42.954951 (+ 34525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.983772 10124 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:19:42.971308 17036 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.992653 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:42.994405 16940 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:42.994643 17036 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:42.951473 17119 consensus_queue.cc:1059] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:19:42.995869 16940 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.999465 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3717) took 2066 ms. Trace:
I20260709 13:19:42.999946 10124 rpcz_store.cc:276] 0709 13:19:40.932713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.917580 (+1984867us) service_pool.cc:224] Handling call
0709 13:19:42.999380 (+ 81800us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.001283 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.002319 10124 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:42.945152 16969 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.005057 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.005985 10128 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:43.006474 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3750) took 1636 ms. Trace:
I20260709 13:19:43.006610 16727 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.007603 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:42.991905 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.004407 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.008456 10129 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:42.991663 17031 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:43.002789 10019 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.590s (SENT)
I20260709 13:19:43.006855 10128 rpcz_store.cc:276] 0709 13:19:41.369843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.983652 (+1613809us) service_pool.cc:224] Handling call
0709 13:19:43.006402 (+ 22750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.002803 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3724) took 2059 ms. Trace:
I20260709 13:19:42.976384 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.010520 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:42.994931 17157 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.010782 10124 rpcz_store.cc:276] 0709 13:19:40.961590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.000779 (+2039189us) service_pool.cc:224] Handling call
0709 13:19:43.002721 (+  1942us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.009639 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3749) took 1683 ms. Trace:
I20260709 13:19:43.022056 10129 rpcz_store.cc:276] 0709 13:19:41.326461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.959000 (+1632539us) service_pool.cc:224] Handling call
0709 13:19:43.009537 (+ 50537us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.028167 10054 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:19:43.039276 10053 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:42.983285 17105 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:19:43.040028 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2246303089998491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 13:19:43.051154 10096 connection.cc:256] RPC call timeout handler was delayed by -1.2210111439999309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 13:19:43.051678 10096 connection.cc:256] RPC call timeout handler was delayed by -1.1850303089997851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
W20260709 13:19:43.052068 10096 connection.cc:256] RPC call timeout handler was delayed by -1.174230308999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18080000000000002s for a response
I20260709 13:19:43.052299 10052 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:43.049525 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.054489 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.007825 16792 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:42.978196 17109 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.006894 16969 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.069072 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.099519 10128 raft_consensus.cc:2410] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:19:43.075839 10125 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:19:43.052461 10050 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:43.101533 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:43.107366 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.108999 10127 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:43.038355 17157 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.136780 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.137130 17157 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.137830 10056 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:43.027475 16967 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:43.139058 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.140120 10056 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:43.142903 17043 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:43.143391 10125 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3722) took 2204 ms (client timeout 2911 ms). Trace:
I20260709 13:19:43.144150 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3234) took 2789 ms. Trace:
I20260709 13:19:43.144593 10126 rpcz_store.cc:276] 0709 13:19:40.354673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.354879 (+   206us) service_pool.cc:224] Handling call
0709 13:19:40.355252 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.695687 (+2340435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.143752 (+448065us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54003895d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:43.144054 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5275685248}
I20260709 13:19:43.145309 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.146194 10050 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:43.146538 10126 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:19:43.147229 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.147377 17119 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:43.110834 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:43.150089 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:43.151454 16969 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:43.152112 16969 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:43.152287 16837 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:43.152920 16837 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:43.153247 16969 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.153981 16837 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.048003 17085 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.155433 17071 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.114179 17126 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:43.157025 17126 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.157546 17085 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.158321 17157 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.114638 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.159782 10124 raft_consensus.cc:2410] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:19:43.116187 17109 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.115648 17130 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.116815 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.168304 10201 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:43.097797 17114 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.122181 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:43.122488 10049 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 900) took 1700 ms (client timeout 1976 ms). Trace:
I20260709 13:19:43.124071 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.056574 17101 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.125006 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.126061 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.127089 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:43.127374 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 240615us
I20260709 13:19:43.127420 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:43.127833 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3755) took 1716 ms (client timeout 1507 ms). Trace:
I20260709 13:19:43.128152 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:43.173857 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.128710 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.129312 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:43.131469 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128745us
I20260709 13:19:43.131625 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.132092 17096 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:43.134781 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.051872 17067 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:43.057718 17036 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:43.148084 10050 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 12: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:43.143963 10125 rpcz_store.cc:269] 0709 13:19:40.938556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.974398 (+2035842us) service_pool.cc:224] Handling call
0709 13:19:43.143281 (+168883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.111435 10048 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:19:43.163691 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.183993 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:43.184141 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.184921 10894 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.185055 10131 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:43.185508 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:43.185781 10131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3756) took 1773 ms (client timeout 1960 ms). Trace:
W20260709 13:19:43.186163 10131 rpcz_store.cc:269] 0709 13:19:41.412139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.183700 (+1771561us) service_pool.cc:224] Handling call
0709 13:19:43.185690 (+  1990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.186964 10204 raft_consensus.cc:2393] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:43.187336 10202 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:43.187618 17079 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:19:43.188548 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.189162 17114 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.189982 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.190670 17096 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:43.189272 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.191550 17126 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.191401 17096 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.192016 10048 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:43.193010 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.193360 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.167695 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3747) took 1912 ms. Trace:
I20260709 13:19:43.193766 10048 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:43.149029 17120 consensus_queue.cc:260] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:43.170756 10049 rpcz_store.cc:269] 0709 13:19:41.421833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.735797 (+313964us) service_pool.cc:224] Handling call
0709 13:19:43.122269 (+1386472us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b540024ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:43.122412 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2024626560}
I20260709 13:19:43.110265 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3771) took 1664 ms. Trace:
I20260709 13:19:43.195708 10128 rpcz_store.cc:276] 0709 13:19:41.446061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.021640 (+1575579us) service_pool.cc:224] Handling call
0709 13:19:43.110167 (+ 88527us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:43.172919 10123 rpcz_store.cc:269] 0709 13:19:41.411848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.775561 (+363713us) service_pool.cc:224] Handling call
0709 13:19:43.127586 (+1352025us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540015ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:43.127720 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521865472}
I20260709 13:19:43.193369 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:43.197495 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:43.198233 17157 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.199813 17109 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:43.201025 17109 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:43.200721 17036 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.201421 17109 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:43.201897 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3251) took 2598 ms. Trace:
I20260709 13:19:43.189738 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:43.176080 10130 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:43.202526 10122 rpcz_store.cc:276] 0709 13:19:40.603258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.674522 (+ 71264us) service_pool.cc:224] Handling call
0709 13:19:40.674852 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.842440 (+167588us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:41.757109 (+914669us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.644460 (+887351us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.645238 (+   778us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.645400 (+   162us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:42.645439 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.201285 (+555846us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540036b854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:43.201365 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.201436 (+    71us) raft_consensus.cc:1685] finished
0709 13:19:43.201591 (+   155us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.201798 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":922594,"prepare.queue_time_us":1137025,"prepare.run_cpu_time_us":12286,"prepare.run_wall_time_us":238644,"raft.queue_time_us":9338,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":332,"spinlock_wait_cycles":364254464,"thread_start_us":886581,"threads_started":1,"wal-append.queue_time_us":1141223}
W20260709 13:19:43.205029 10130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3758) took 1792 ms (client timeout 1982 ms). Trace:
I20260709 13:19:43.205426 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:43.207355 17085 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.196077 10049 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
W20260709 13:19:43.205538 10130 rpcz_store.cc:269] 0709 13:19:41.412762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.116379 (+1703617us) service_pool.cc:224] Handling call
0709 13:19:43.204931 (+ 88552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.033566 17031 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.210348 10129 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:19:43.170092 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3713) took 2228 ms. Trace:
I20260709 13:19:43.220471 10126 rpcz_store.cc:276] 0709 13:19:40.931697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.145670 (+2213973us) service_pool.cc:224] Handling call
0709 13:19:43.146167 (+   497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.146901 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.160403 (+ 13502us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.208199 10055 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.218048 10131 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:43.200130 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:43.208319 10049 pending_rounds.cc:85] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:19:43.227502 10049 pending_rounds.cc:107] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:19:43.227943 10049 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.236833 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3720) took 2284 ms. Trace:
I20260709 13:19:43.237504 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3754) took 1826 ms. Trace:
I20260709 13:19:43.237627 10131 rpcz_store.cc:276] 0709 13:19:40.938047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.186857 (+2248810us) service_pool.cc:224] Handling call
0709 13:19:43.217729 (+ 30872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.222112 (+  4383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.222442 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.202566 17101 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.253751 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3723) took 2314 ms. Trace:
I20260709 13:19:43.255280 16967 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.202440 17109 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.256668 17109 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:43.256912 10125 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:43.257858 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3714) took 2325 ms. Trace:
I20260709 13:19:43.258185 10125 rpcz_store.cc:276] 0709 13:19:40.931965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.186713 (+2254748us) service_pool.cc:224] Handling call
0709 13:19:43.256253 (+ 69540us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfe9c 00007fa274edfa5b 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:43.256507 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.257505 (+   998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.257762 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75528064}
I20260709 13:19:43.254339 10123 rpcz_store.cc:276] 0709 13:19:40.938796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.196748 (+2257952us) service_pool.cc:224] Handling call
0709 13:19:43.252473 (+ 55725us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:43.252776 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.253300 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:43.253334 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:43.253384 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:43.253412 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.253481 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.253650 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52324608}
I20260709 13:19:43.260066 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.148936 17119 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.238665 10129 rpcz_store.cc:276] 0709 13:19:41.411551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.045818 (+1634267us) service_pool.cc:224] Handling call
0709 13:19:43.237403 (+191585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.188426 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.275807 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.194082 10124 rpcz_store.cc:276] 0709 13:19:41.255586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.048522 (+1792936us) service_pool.cc:224] Handling call
0709 13:19:43.167480 (+118958us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:43.276350 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76347.2us
I20260709 13:19:43.263275 10126 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:43.193676 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.279834 10048 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:43.282925 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.286101 10050 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:43.290664 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.291666 10049 raft_consensus.cc:2368] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18.
I20260709 13:19:43.293236 10892 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:43.295814 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102571us
I20260709 13:19:43.291584 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.292834 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3257) took 1936 ms. Trace:
I20260709 13:19:43.302990 10123 rpcz_store.cc:276] 0709 13:19:41.356564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.260876 (+1904312us) service_pool.cc:224] Handling call
0709 13:19:43.292163 (+ 31287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.292431 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.292465 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:43.292518 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:43.292556 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.292620 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.292742 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.202822 17067 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:43.309217 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 137602us
I20260709 13:19:43.310141 10055 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:43.311457 10051 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:43.315901 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:43.317299 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 369716us
I20260709 13:19:43.315366 10204 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:19:43.341825 10124 raft_consensus.cc:2368] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18.
W20260709 13:19:43.342758 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 45869.5us
I20260709 13:19:43.353811 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:43.356406 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:43.363413 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 158123us
I20260709 13:19:43.313972 17136 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.384057 17136 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.385972 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.386716 10129 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:19:43.387602 17136 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.387799 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3261) took 2030 ms. Trace:
I20260709 13:19:43.388399 10129 rpcz_store.cc:276] 0709 13:19:41.357566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.275251 (+1917685us) service_pool.cc:224] Handling call
0709 13:19:43.386339 (+111088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.387399 (+  1060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.387717 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:43.370512 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3769) took 1950 ms (client timeout 1551 ms). Trace:
W20260709 13:19:43.389747 10127 rpcz_store.cc:269] 0709 13:19:41.420700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.261366 (+840666us) service_pool.cc:224] Handling call
0709 13:19:43.370266 (+1108900us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540014b254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:43.370431 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376464640}
I20260709 13:19:43.371817 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.372627 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:43.375892 10128 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:43.376271 17031 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.386929 10202 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:43.392936 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3733) took 2412 ms. Trace:
I20260709 13:19:43.393416 10128 rpcz_store.cc:276] 0709 13:19:40.980530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.197794 (+2217264us) service_pool.cc:224] Handling call
0709 13:19:43.294653 (+ 96859us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:43.375251 (+ 80598us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b540036b1a8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cfa8ebc 00007fa27cfde08c 00007fa27cfc893b 00007fa27cfc83d2 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c
0709 13:19:43.375417 (+   166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.392493 (+ 17076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.392839 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75063808}
I20260709 13:19:43.395236 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.379878 17036 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.388814 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3255) took 2048 ms. Trace:
I20260709 13:19:43.359468 10122 pending_rounds.cc:85] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:19:43.399348 10122 pending_rounds.cc:107] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:19:43.391593 10126 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:43.396298 10124 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
W20260709 13:19:43.369076 17094 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.007s
I20260709 13:19:43.404495 10123 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.399672 10122 raft_consensus.cc:2889] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.405771 10131 rpcz_store.cc:276] 0709 13:19:41.339970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.239647 (+1899677us) service_pool.cc:224] Handling call
0709 13:19:43.348142 (+108495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.388417 (+ 40275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.388449 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:43.388513 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:43.388548 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.388607 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.388722 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.401502 17136 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.411592 17101 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.412467 10127 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:43.413511 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3762) took 1999 ms. Trace:
I20260709 13:19:43.413697 17074 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.413954 10127 rpcz_store.cc:276] 0709 13:19:41.414088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.400551 (+1986463us) service_pool.cc:224] Handling call
0709 13:19:43.412184 (+ 11633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.413108 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.413414 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.414928 17074 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.415908 10127 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:43.418488 17074 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:43.419009 10126 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:43.411614 17136 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:43.427627 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.424638 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:43.437783 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 44343.3us
W20260709 13:19:43.437858 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 68861.1us
I20260709 13:19:43.442750 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:43.444240 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 102592us
I20260709 13:19:43.397720 17157 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:43.450775 10131 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:19:43.451584 10124 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:43.447351 10894 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:43.497021 10047 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 18. Ops: []
I20260709 13:19:43.433862 17105 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:43.521692 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:43.522758 10131 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.525569 10048 raft_consensus.cc:3060] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.526335 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:43.529657 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:43.434314 17094 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.491232 17101 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:19:43.552486 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3295) took 1012 ms. Trace:
I20260709 13:19:43.518101 17157 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:43.557942 10051 rpcz_store.cc:276] 0709 13:19:42.539939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.291246 (+751307us) service_pool.cc:224] Handling call
0709 13:19:43.311207 (+ 19961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.311995 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.552094 (+240099us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b54003cfad0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:43.552367 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44670976}
W20260709 13:19:43.562958 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 128627us
I20260709 13:19:43.549425 17072 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.537395 17074 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:43.578714 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76629.2us
I20260709 13:19:43.558314 16840 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:43.593638 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:43.599980 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 235260us
I20260709 13:19:43.602638 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:43.609051 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:43.610486 10196 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:43.573216 16837 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:43.571729 16968 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.618131 10056 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:43.619136 10052 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:43.620703 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.571209 17074 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:43.627847 10055 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:43.630702 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.636075 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.593302 16840 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:43.592159 17114 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:43.645586 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:43.625768 17074 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.660907 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.626067 16940 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.671916 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.637787 17101 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:43.634859 17119 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:43.679729 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 188141us
I20260709 13:19:43.651265 16840 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:43.684962 10131 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:43.685025 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 191235us
I20260709 13:19:43.651584 17171 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:43.668779 17104 raft_consensus.cc:2764] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:19:43.687256 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:43.687561 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 97815.5us
I20260709 13:19:43.688529 10896 raft_consensus.cc:1240] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:19:43.696319 10131 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.706861 10055 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:19:43.708364 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 215594us
W20260709 13:19:43.710760 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65290.2us
I20260709 13:19:43.655072 16968 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.635667 17105 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.739880 17094 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.741572 17136 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.706081 17157 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.743628 10891 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:43.745981 17170 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.695999 17171 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.721961 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3256) took 2365 ms. Trace:
I20260709 13:19:43.723099 17120 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:43.724351 17172 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:19:43.727226 17186 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.119s
W20260709 13:19:43.728824 17174 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:19:43.773526 16968 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.731714 16969 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:43.774302 16969 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:43.735961 17168 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 13:19:43.775070 10894 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
W20260709 13:19:43.736481  9939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:43.778015 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3253) took 2438 ms. Trace:
I20260709 13:19:43.716043 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:43.755285 10125 rpcz_store.cc:276] 0709 13:19:41.356256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.260811 (+1904555us) service_pool.cc:224] Handling call
0709 13:19:43.276500 (+ 15689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.283277 (+  6777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.283313 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.284511 (+  1198us) log.cc:844] Serialized 494 byte log entry
0709 13:19:43.531458 (+246947us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:43.531522 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.531654 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.721623 (+189969us) raft_consensus.cc:1685] finished
0709 13:19:43.721648 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.721833 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":232122,"threads_started":1,"wal-append.queue_time_us":374219}
I20260709 13:19:43.716979 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:43.746774 17170 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:43.791764 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.757882 10123 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:43.683234 16940 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.768115 16854 consensus_queue.cc:1059] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:43.769191 17119 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:43.770336 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.778582 10130 rpcz_store.cc:276] 0709 13:19:41.339441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.218621 (+1879180us) service_pool.cc:224] Handling call
0709 13:19:43.249684 (+ 31063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.267398 (+ 17714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.267454 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.283085 (+ 15631us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.576886 (+293801us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:43.576929 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.577009 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.777608 (+200599us) raft_consensus.cc:1685] finished
0709 13:19:43.777653 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.777876 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":196,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":4727,"thread_start_us":293297,"threads_started":1,"wal-append.queue_time_us":425063}
I20260709 13:19:43.721194 17067 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.779207 17096 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:43.795040 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.796619 10048 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:43.803663 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 94078.6us
I20260709 13:19:43.788257 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.801767 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.821431 17170 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:43.825285 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.759169 17171 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:43.806020 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:43.830312 10125 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:43.836042 10123 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:43.837110 17104 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:43.839653 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 129832us
I20260709 13:19:43.840780 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:43.856499 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.818852 16940 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:43.857227 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.865931 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.866978 10048 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:43.870939 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.828040 17094 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:43.881577 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.882432 10050 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:43.887344 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:43.884493 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.892771 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.888509 10125 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:43.842172 16854 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:19:43.895870 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:43.901672 10131 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:43.880556 17105 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:43.917006 10131 raft_consensus.cc:1217] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Deduplicated request from leader. Original: 11.6->[12.7-16.9]   Dedup: 12.7->[14.8-16.9]
I20260709 13:19:43.919638 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:43.873984 17074 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:43.924527 10125 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:43.889951 17094 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:43.922333 10052 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:43.918231 10131 pending_rounds.cc:85] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 7
I20260709 13:19:43.930317 10131 pending_rounds.cc:107] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:43.930939 10131 raft_consensus.cc:2889] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:43.935956 17188 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:19:43.943367 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.944144 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.947613 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:43.951821 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3252) took 2612 ms. Trace:
I20260709 13:19:43.952589 16836 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:43.957585 17166 raft_consensus.cc:3055] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:43.952617 10122 rpcz_store.cc:276] 0709 13:19:41.339085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.205691 (+1866606us) service_pool.cc:224] Handling call
0709 13:19:43.355768 (+150077us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b5800150998. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:43.356101 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:43.502149 (+146048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.502187 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:43.522174 (+ 19987us) log.cc:844] Serialized 52 byte log entry
0709 13:19:43.645712 (+123538us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:43.841930 (+196218us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.842580 (+   650us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.842819 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.842947 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.951338 (+108391us) raft_consensus.cc:1685] finished
0709 13:19:43.951386 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.951693 (+   307us) inbound_call.cc:177] Queueing success response
0709 13:19:43.955814 (+  4121us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b18003f4960. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ee52a6 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f
0709 13:19:43.956358 (+   544us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
Metrics: {"mutex_wait_us":1311,"prepare.queue_time_us":226468,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":39370,"raft.run_cpu_time_us":2721,"raft.run_wall_time_us":128187,"spinlock_wait_cycles":73643008,"thread_start_us":228321,"threads_started":2,"wal-append.queue_time_us":336796}
I20260709 13:19:43.958231 17166 raft_consensus.cc:740] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:43.959645 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:43.934492 17085 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.727s	user 0.016s	sys 0.000s
I20260709 13:19:43.961172 10123 pending_rounds.cc:85] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 6
I20260709 13:19:43.965906 10123 pending_rounds.cc:107] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:19:43.966455 10123 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.909489 17105 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:43.965832 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:43.962371 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 121754us
I20260709 13:19:43.966890 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:43.969136 10122 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.973392 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:43.961632 17192 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:43.978507 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.979754 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.979590 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:43.979754 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.976300 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.982800 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.979534 16968 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:43.983603 16968 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:43.983683 10197 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:19:43.984367 16968 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:43.968751 17105 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.987673 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.988569 10200 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:43.991214 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:43.993058 17094 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:43.994185 10897 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:44.002871 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:43.971366 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.013852 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:44.014955 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.015679 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.016000 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:43.885639 16969 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.017707 10125 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:43.968926 17085 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:44.042393 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.043574 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 263485us
W20260709 13:19:44.045461 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 81516us
I20260709 13:19:44.047695 10126 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:44.047593 17136 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:44.048703 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:44.034920 10892 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 5 to 10, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:44.032145 16940 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:44.050704 16940 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.051146 16940 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:44.051529 16969 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.049420 10126 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:43.983265 17170 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:19:44.063642 17170 consensus_queue.cc:1243] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:19:44.056159 17067 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.065546 17067 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:44.068116 17205 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:19:44.047693 17176 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:44.035012 17202 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:44.060920 10168 leader_election.cc:400] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18.
I20260709 13:19:44.061437 17079 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.069351 10897 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:19:44.077296 17201 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
I20260709 13:19:44.045509 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:43.959671 17166 consensus_queue.cc:260] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.070526 17079 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.061992 17119 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:44.069805 17176 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.102797 17176 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.104861 17079 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:44.070223 10168 leader_election.cc:403] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:19:44.079208 10897 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:44.107246 17203 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:44.107935 17203 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.108215 10897 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.109759 10897 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.085589 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:44.086050 10126 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:19:44.088268 17109 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.831s	user 0.006s	sys 0.014s
W20260709 13:19:44.111995 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66691.5us
I20260709 13:19:44.112258 17074 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:44.112686 17109 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:44.113075 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.091470 17166 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:44.113231 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:44.114413 10126 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:44.090898 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:44.115285 17196 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:19:44.115521 17173 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:44.072940 17067 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:44.118124 10126 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:44.116472 17173 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:44.125119 10047 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:19:44.125659 10047 pending_rounds.cc:85] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:44.125906 10047 pending_rounds.cc:107] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:19:44.126159 10047 raft_consensus.cc:2889] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.128700 10198 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
W20260709 13:19:44.138528 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 77846.2us
I20260709 13:19:44.128961 10169 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:44.113305 17109 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:44.148016 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57192.5us
I20260709 13:19:44.108300 17203 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.141889 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.148028 17136 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18.
I20260709 13:19:44.167157 16968 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.170079 16968 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:44.148447 17109 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:44.172864 10170 leader_election.cc:400] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Vote denied by peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347) with higher term. Message: Invalid argument: T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18.
I20260709 13:19:44.173981 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:44.180135 17119 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:44.181597 17119 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:44.182508 17104 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:44.183617 16854 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:44.184423 10128 raft_consensus.cc:2410] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:19:44.186151 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.120440 17105 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.182641 16727 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:44.186676 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:44.187698 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.206089 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.189409 16840 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.199970 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:44.193825 10168 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:44.219940 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:44.232769 10125 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:44.253027 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 66973.4us
W20260709 13:19:44.257666 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63944.6us
I20260709 13:19:44.161321 17067 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:19:44.205471 17172 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 13:19:44.261787 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 75212.4us
I20260709 13:19:44.194813 17173 raft_consensus.cc:2764] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:19:44.261696 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 69211.3us
W20260709 13:19:44.272249 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.275377 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65535.1us
I20260709 13:19:44.276868 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:44.263585 17067 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.288519 17067 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:44.289176 10198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2494) took 1165 ms. Trace:
I20260709 13:19:44.289608 10198 rpcz_store.cc:276] 0709 13:19:43.124145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.124741 (+   596us) service_pool.cc:224] Handling call
0709 13:19:43.197291 (+ 72550us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5800170d98. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:19:44.288901 (+1091610us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:44.289068 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073379456}
I20260709 13:19:44.208591 16727 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:44.292124 16727 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.293954 16727 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.297262 17176 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.203615 16854 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:44.303907 10125 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:19:44.305627 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:44.306752 10125 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:44.314875 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.316485 17085 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:44.317364 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.312143 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:44.321944 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.213292 16837 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:19:44.186794 17171 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.269644 17210 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:44.336079 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67640.4us
I20260709 13:19:44.336211 17096 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:44.354466 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:44.340296 17126 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 5 to 10, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:44.341593  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:44.341848 17202 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:44.343258 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 64226.3us
I20260709 13:19:44.345716 17105 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:44.346513 17109 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.350083 16940 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.352381 16840 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:44.269335 17071 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:44.337495 17198 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.390s	user 0.002s	sys 0.001s
I20260709 13:19:44.374228 10897 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:44.362413 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.376933 10125 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:44.319975 17173 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:44.420356 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.528s (SENT)
W20260709 13:19:44.421883 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.424192 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 51430.1us
I20260709 13:19:44.425243 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:44.432689 10128 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19. Re-sending same reply.
I20260709 13:19:44.426357 10891 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:44.353896 17085 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:44.435657 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.362051 17096 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.441819 10125 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:44.361479 17210 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.441586 10122 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13.
I20260709 13:19:44.444136 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:44.442199 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90894.5us
W20260709 13:19:44.442811 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80494.4us
I20260709 13:19:44.445065 10128 raft_consensus.cc:2410] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:19:44.416356 16968 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:44.443264 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.444136 10126 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:44.447003 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.428589 17071 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.440410 17173 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:44.451180 17170 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 13:19:44.446640 17096 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.446437 16969 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:44.513867 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92288.4us
W20260709 13:19:44.532866 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72904.1us
I20260709 13:19:44.517457 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:44.525923 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.452778 16968 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.449172 16837 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.512094 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.446585 17210 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.508972 17071 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:44.543496 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 284820us
W20260709 13:19:44.550225 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.512296 17173 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:44.554811 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:44.531205 16836 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:44.570915 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 92581.6us
I20260709 13:19:44.574326 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:44.575414 10198 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:44.547303 16968 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:44.574098 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.466361 17105 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 13:19:44.592294 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 180481us
W20260709 13:19:44.596583 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.528285 17186 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 13:19:44.611938 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:44.612047 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59448.5us
W20260709 13:19:44.570955 17120 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.004s
I20260709 13:19:44.552894 17094 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:44.619382 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:44.628363 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:44.629212 10200 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:44.633294 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60731.9us
W20260709 13:19:44.633862 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79191.6us
I20260709 13:19:44.634246 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.557976 17079 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:19:44.638310 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3273) took 1089 ms. Trace:
I20260709 13:19:44.604691 16840 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:44.594512 17104 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:44.643158 10205 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:19:44.620764 16852 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.664182 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:44.618077 17218 raft_consensus.cc:2804] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:44.667868 10202 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:44.615049 17202 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:44.668630 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 156758us
I20260709 13:19:44.644688 10130 rpcz_store.cc:276] 0709 13:19:43.557387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.823099 (+265712us) service_pool.cc:224] Handling call
0709 13:19:43.848594 (+ 25495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.861863 (+ 13269us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:44.296819 (+434956us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b1800376c60. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf
0709 13:19:44.297196 (+   377us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.636811 (+339615us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.637283 (+   472us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:44.637578 (+   295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.637934 (+   356us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.638195 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18900,"prepare.queue_time_us":52486,"prepare.run_cpu_time_us":791,"prepare.run_wall_time_us":3280,"raft.queue_time_us":41586,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":21111,"spinlock_wait_cycles":33039360,"thread_start_us":410805,"threads_started":3,"wal-append.queue_time_us":316412}
W20260709 13:19:44.669396 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71964.8us
I20260709 13:19:44.631214 16969 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.586490 16837 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.647226 16840 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:44.636467 17094 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:44.644538 17176 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:44.648249 17105 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:44.666419 10205 pending_rounds.cc:85] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 10
I20260709 13:19:44.728446 10205 pending_rounds.cc:107] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:19:44.729038 10205 raft_consensus.cc:2889] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:44.717087 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.717547 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3751) took 3307 ms. Trace:
I20260709 13:19:44.716949 17206 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:44.731410 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.731523 10129 rpcz_store.cc:276] 0709 13:19:41.410429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.405641 (+1995212us) service_pool.cc:224] Handling call
0709 13:19:43.502487 (+ 96846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.525679 (+ 23192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.525718 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.556102 (+ 30384us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.869698 (+313596us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:44.133680 (+263982us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.134032 (+   352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.134163 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.262281 (+128118us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:44.673219 (+410938us) raft_consensus.cc:1685] finished
0709 13:19:44.717239 (+ 44020us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.717438 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21713,"prepare.queue_time_us":99995,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1583,"raft.queue_time_us":1756,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4600,"spinlock_wait_cycles":34715392,"thread_start_us":510074,"threads_started":2,"wal-append.queue_time_us":339100}
W20260709 13:19:44.720824 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 146806us
W20260709 13:19:44.711429 17187 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.993s	user 0.000s	sys 0.006s
W20260709 13:19:44.718372 17211 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:19:44.737396 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:44.746076 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.725067 16840 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.715204 17228 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:44.753154 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 118984us
I20260709 13:19:44.722958 17218 raft_consensus.cc:697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:44.759666 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:44.772928 10198 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:44.704484 16852 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.753573 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:44.776362 17216 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:19:44.777879 17232 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:44.765332 17228 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.779662 17228 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:44.736057 17109 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:44.782089 17109 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:44.732663 17105 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:44.783424 17213 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.323s	user 0.000s	sys 0.002s
I20260709 13:19:44.783828 17105 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.785565 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3276) took 1117 ms. Trace:
I20260709 13:19:44.786105 10123 rpcz_store.cc:276] 0709 13:19:43.671461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.960282 (+288821us) service_pool.cc:224] Handling call
0709 13:19:43.960845 (+   563us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:44.008698 (+ 47853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.008780 (+    82us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:44.024988 (+ 16208us) log.cc:844] Serialized 52 byte log entry
0709 13:19:44.225421 (+200433us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:44.225475 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.225583 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.785219 (+559636us) raft_consensus.cc:1685] finished
0709 13:19:44.785248 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.785478 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":12208,"thread_start_us":143573,"threads_started":1,"wal-append.queue_time_us":292130}
I20260709 13:19:44.787693 17094 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.795539 17202 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.798137 17096 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.799437 17105 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:44.800388 16727 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.506s	user 0.015s	sys 0.000s
I20260709 13:19:44.800940 17096 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:44.796134 17237 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:44.794759 10200 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:19:44.773293 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:44.807852 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:44.809604 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:44.810492 10202 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:44.743219 17171 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:44.752812 17104 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:44.750327 17230 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:44.726315 16969 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:44.801975 17096 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:44.840077 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3269) took 1474 ms. Trace:
I20260709 13:19:44.816422 17186 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:44.843088 10198 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:44.840631 10127 rpcz_store.cc:276] 0709 13:19:43.365902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.417334 (+ 51432us) service_pool.cc:224] Handling call
0709 13:19:43.417826 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.421081 (+  3255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.421134 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.421964 (+   830us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.580715 (+158751us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:43.580835 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.580951 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.515436 (+934485us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:44.834095 (+318659us) raft_consensus.cc:1685] finished
0709 13:19:44.834125 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.839975 (+  5850us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2395,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":155,"spinlock_wait_cycles":74031360,"thread_start_us":157980,"threads_started":1,"wal-append.queue_time_us":229616}
I20260709 13:19:44.862285 17218 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.863308 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:44.866688 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 832) took 1044 ms. Trace:
I20260709 13:19:44.867288 10124 rpcz_store.cc:276] 0709 13:19:43.822256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.842101 (+ 19845us) service_pool.cc:224] Handling call
0709 13:19:44.866395 (+1024294us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:44.866590 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263411072}
I20260709 13:19:44.868247 16837 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:44.871732 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.875739 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:44.876586 10047 raft_consensus.cc:2376] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16. Re-sending same reply.
I20260709 13:19:44.883713 17119 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:44.884966 17104 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:44.885476 17104 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:44.885895 17104 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:44.843324 17230 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:44.896777 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 274095us
I20260709 13:19:44.898813 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:44.897799 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.912904 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:44.914362 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49312.7us
W20260709 13:19:44.915295 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.951s (SENT)
I20260709 13:19:44.923648 10168 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:44.924168 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:44.900846 16727 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:44.943415 16727 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.944072 16727 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:44.944303 10863 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.976s (SENT)
I20260709 13:19:44.935523 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:44.946942 10049 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:44.937719 17228 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:44.939548 17067 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.006s
I20260709 13:19:44.910132 16840 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.960012 10202 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 25.
I20260709 13:19:44.967059 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:44.968344 10053 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:44.969847 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:44.970788 10053 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:44.974591 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 238986us
I20260709 13:19:44.921116 17166 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:44.978444 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:44.984159 10056 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:44.987082 10048 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:45.004334 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68973.7us
I20260709 13:19:44.964720 17136 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:44.953490 17206 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:45.009442 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44676.1us
I20260709 13:19:44.974100 17171 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.990897 16840 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.037634 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:44.945973 17210 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.019573 17136 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:45.045069 17136 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:44.964918 17067 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:45.064155 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115605us
I20260709 13:19:45.065397 10170 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:45.071527 17228 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:45.079259 17219 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.566s	user 0.005s	sys 0.000s
I20260709 13:19:45.081139 17067 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:45.051548 17225 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:45.083706 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1084) took 1320 ms. Trace:
I20260709 13:19:45.084501 10891 rpcz_store.cc:276] 0709 13:19:43.762963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.763291 (+   328us) service_pool.cc:224] Handling call
0709 13:19:45.083330 (+1320039us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:45.083600 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438668672}
I20260709 13:19:45.089989 10056 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:19:45.094416 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.098074 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.093413 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:45.043023 17210 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.084271 17225 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:45.112290 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 43509.4us
W20260709 13:19:45.114136 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99273.2us
W20260709 13:19:45.121698 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.092522 17119 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:45.112238 17071 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.525s	user 0.015s	sys 0.000s
W20260709 13:19:45.126950 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80130us
I20260709 13:19:45.100924 17105 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:45.117278 17225 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:45.116868 17067 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:45.064018 17171 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:45.126190 17203 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.881s	user 0.010s	sys 0.007s
I20260709 13:19:45.132195 17119 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:45.147154 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 389509us
I20260709 13:19:45.158645 17243 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:19:45.158850 16854 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.139389 17071 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.161914 17119 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.149662 17203 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.143677 17176 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:19:45.103089 17228 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:19:45.039541 17230 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.175450 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:45.142246 17238 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:19:45.179909 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.119802 17074 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:19:45.187517 10128 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 13:19:45.196159 10202 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:45.187037 17244 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:19:45.173239 17250 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:19:45.119002 17220 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:19:45.173260 16969 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:45.193149 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.196256 10128 pending_rounds.cc:85] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 14
I20260709 13:19:45.231483 10128 pending_rounds.cc:107] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:19:45.231930 10128 raft_consensus.cc:2889] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:45.145459 17067 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.202605 17074 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.234521 17074 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:45.235222 17203 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.236164 17096 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.235770 17074 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.236941 17096 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.237766 17074 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.202832 16852 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:45.239707 17067 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.241055 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.241334 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.242086 10124 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:45.233732 17251 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:45.244444 17251 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.244865 17251 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.243788 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.205073 17171 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:45.246596 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:45.207929 16854 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:45.212565 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65423.9us
I20260709 13:19:45.236230 16727 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.203769 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:45.211288 17094 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.233664 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:45.263482 10127 raft_consensus.cc:2376] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 13. Re-sending same reply.
W20260709 13:19:45.257491 10863 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.867s (SENT)
W20260709 13:19:45.261439 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.291833 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 113706us
I20260709 13:19:45.293032 10202 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:45.306605 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.307662 10129 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
W20260709 13:19:45.308168 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.300954 17173 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.000s
W20260709 13:19:45.357197 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99781.9us
I20260709 13:19:45.207952 17114 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:19:45.364897 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:45.357702 10020 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:45.392894 10096 connection.cc:256] RPC call timeout handler was delayed by -2.3968361209999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2844s for a response
W20260709 13:19:45.354841 17244 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:19:45.403079 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:45.404199 10202 raft_consensus.cc:2393] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:45.411738 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:45.413851 10049 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:45.409911 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:45.422923 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 280818us
W20260709 13:19:45.427659 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.291286 17094 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.393273 17096 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.441471 17256 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.442070 16967 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:45.442292 17256 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:45.443782 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84265.5us
I20260709 13:19:45.444976 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:45.445595 17256 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.446621 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:45.448985 10897 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:45.448985 17236 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.450505 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:45.451886 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:45.453450 16727 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:45.475714 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:45.464593 10198 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:45.466042 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.464149 10053 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:45.472908 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.479465 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.442873 16967 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.482331 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.482441 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.418054 17203 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:45.449061 17202 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:45.485783 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:45.486845 10203 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:45.485311 16967 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:45.485783 17236 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.491518 17203 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:45.476925 17126 consensus_meta.cc:287] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.907s	user 0.007s	sys 0.011s
I20260709 13:19:45.492964 10197 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:45.493218 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:45.494002 10052 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:45.469161 16969 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.492964 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:45.363082 17173 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.499339 10199 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:45.499672 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2925502539999343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16180000000000003s for a response
I20260709 13:19:45.473852 17096 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:45.514173 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67676.7us
I20260709 13:19:45.528028 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:45.478724 17242 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:45.528813 10198 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.477475 17256 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:45.544873 10203 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:45.529547 17104 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.643s	user 0.015s	sys 0.000s
W20260709 13:19:45.528232 16968 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.899s	user 0.009s	sys 0.007s
I20260709 13:19:45.525763 17236 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.482175 17067 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:45.556293 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.537441 17242 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.580943 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:45.603156 10052 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:45.550719 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 157007us
I20260709 13:19:45.573797 16968 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.517840 16969 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:45.614370 10865 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.976s (SENT)
I20260709 13:19:45.615008 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:45.537019 17096 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.574860 17071 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:45.625149 10096 connection.cc:256] RPC call timeout handler was delayed by -2.3608361209999202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2884s for a response
I20260709 13:19:45.642887 10198 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:45.644631 10894 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:45.553635 17104 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.654850 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:45.678076 10129 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:45.607358 17242 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:45.545770 17203 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:45.679196 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 149625us
I20260709 13:19:45.688385 10052 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:45.692723 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66505.1us
W20260709 13:19:45.691442 17209 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.913s	user 0.018s	sys 0.024s
W20260709 13:19:45.700407 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 113660us
I20260709 13:19:45.718640 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.719419 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3275) took 2084 ms. Trace:
I20260709 13:19:45.720265 10020 leader_election.cc:400] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:19:45.720750 10020 leader_election.cc:403] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:19:45.722299 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:45.719970 10131 rpcz_store.cc:276] 0709 13:19:43.635250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.916045 (+280795us) service_pool.cc:224] Handling call
0709 13:19:43.916697 (+   652us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:43.978703 (+ 62006us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:43.999884 (+ 21181us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.027577 (+ 27693us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:44.028405 (+   828us) log.cc:844] Serialized 52 byte log entry
0709 13:19:44.040330 (+ 11925us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:44.040798 (+   468us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.053546 (+ 12748us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.056639 (+  3093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.056717 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.291260 (+234543us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800163840. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:44.746911 (+455651us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b1800163840. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:45.517017 (+770106us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:45.719167 (+202150us) raft_consensus.cc:1685] finished
0709 13:19:45.719192 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.719353 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8747,"prepare.queue_time_us":158084,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":145,"raft.queue_time_us":18334,"raft.run_cpu_time_us":614,"raft.run_wall_time_us":3954,"spinlock_wait_cycles":122850944,"thread_start_us":24503,"threads_started":1,"wal-append.queue_time_us":83795}
W20260709 13:19:45.726342 10096 connection.cc:256] RPC call timeout handler was delayed by -2.3311361210000996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2917s for a response
W20260709 13:19:45.728247 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113924us
I20260709 13:19:45.735344 10126 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:45.738905 17119 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.540s	user 0.015s	sys 0.008s
I20260709 13:19:45.750082 10050 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:45.753046 17263 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:19:45.691785 17171 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:19:45.766062 10129 raft_consensus.cc:2468] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:45.768488 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:45.770186 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:45.781106 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.784001 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:45.786862 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 376992us
W20260709 13:19:45.775198 17259 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:19:45.787420 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:45.765022 17094 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:45.780759 17262 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:19:45.769843 17233 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.796887 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:45.805505 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:45.777210 17126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33772 (request call id 1) took 7887 ms. Trace:
I20260709 13:19:45.777184 17114 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:19:45.800841 17186 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:45.791829 17173 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.772707 17230 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:19:45.808166 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:45.812832 17225 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.635s	user 0.015s	sys 0.000s
I20260709 13:19:45.778111 17119 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.809361 17094 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:45.851630 17261 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:45.808887 17233 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.839691 17166 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:45.843622 17275 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:19:45.804129 17236 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:19:45.852712 17170 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:19:45.885512 17222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.026s	sys 0.026s
W20260709 13:19:45.888680 17222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.026s	sys 0.026s
I20260709 13:19:45.844857 17186 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:45.885526 17166 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:45.890455 17231 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.095s	user 0.000s	sys 0.007s
I20260709 13:19:45.890978 17166 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.883312 16852 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:45.892503 17233 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.893332 16969 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:45.893483 16852 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.894042 16852 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:45.890847 17186 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.843794 17267 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 13:19:45.900588 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:45.895977 17287 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.901675 17287 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:45.901836 10052 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:45.904208 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:45.905189 10053 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:45.902870 17287 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.895072 16852 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:45.898043 17170 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:45.922907 17170 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:45.888749 17270 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 13:19:45.855724 17173 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:45.849764 17067 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:45.923631 17170 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.943228 16968 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.944348 16968 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:45.944944 16968 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:45.945971 16968 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.947507 17096 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:45.896916 17267 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:45.923091 16852 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.948226 17267 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:45.951968 17273 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:19:45.954046 17170 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:45.897504 17265 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 13:19:45.954875 17285 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:19:45.839138 17280 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
I20260709 13:19:45.840361 17225 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:45.897593 17286 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:45.813822 17126 rpcz_store.cc:276] 0709 13:19:37.883097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.897036 (+ 13939us) service_pool.cc:224] Handling call
0709 13:19:38.368707 (+471671us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4c001ac880. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2762d0a20 00007fa2762e1a29 00007fa2762a7644 00007fa2762cef2d 00007fa2762ceb93 00007fa277f5d1af 00007fa277f5cf3e 00007fa277fe2888 00007fa277fff75c 00007fa27800076a 00007fa277fff91b
0709 13:19:38.370872 (+  2165us) log.cc:844] Serialized 494 byte log entry
0709 13:19:45.739653 (+7368781us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8066,"raft.queue_time_us":8430,"raft.run_cpu_time_us":1558,"raft.run_wall_time_us":154329,"spinlock_wait_cycles":70177152,"thread_start_us":438387,"threads_started":1,"wal-append.queue_time_us":921789,"wal-append.run_cpu_time_us":7651,"wal-append.run_wall_time_us":1095026}
W20260709 13:19:46.009716 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 204289us
W20260709 13:19:45.964238 17074 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.005s
I20260709 13:19:45.941038 17104 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.021059 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:46.020833 16967 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.006s
I20260709 13:19:45.973176 17170 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:46.030761 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 310638us
I20260709 13:19:45.986223 17094 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:46.031952 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.037245 10128 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.097527 10053 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:19:46.099222 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.116513 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3335) took 1175 ms. Trace:
I20260709 13:19:46.116133 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.120651 10056 rpcz_store.cc:276] 0709 13:19:44.941307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.121122 (+179815us) service_pool.cc:224] Handling call
0709 13:19:45.136060 (+ 14938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.138398 (+  2338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.138446 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.144482 (+  6036us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.479613 (+335131us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:45.479667 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.479736 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.116132 (+636396us) raft_consensus.cc:1685] finished
0709 13:19:46.116169 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.116422 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1695,"raft.queue_time_us":2510,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":5249,"thread_start_us":251423,"threads_started":1,"wal-append.queue_time_us":414014}
I20260709 13:19:46.075548 17287 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:46.100605 17259 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:19:46.125653 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.074122 17171 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:46.128736 17283 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:46.134331 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.135432 10056 raft_consensus.cc:2376] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19. Re-sending same reply.
W20260709 13:19:46.114109 17218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.247s	user 0.009s	sys 0.009s
W20260709 13:19:46.202323 17218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.336s	user 0.010s	sys 0.010s
I20260709 13:19:46.130975 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.137048 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:46.035679 16967 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.206118 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:46.113920 17291 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 13:19:46.164646 17225 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:46.167385 17119 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:46.133487 17171 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.183540 17094 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:46.185649 17223 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
W20260709 13:19:46.199429 17231 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:46.215546 10056 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:19:46.200755 17251 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.955s	user 0.015s	sys 0.002s
I20260709 13:19:46.216248 17251 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.218166 16967 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.218370 17251 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:46.201576 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 422585us
I20260709 13:19:46.219084 17170 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:46.219753 17170 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:46.219508 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:46.220855 17170 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.221654 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:46.210650 16940 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.213716 17171 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.223594 17251 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:46.224442 17251 raft_consensus.cc:2764] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:46.225294 10056 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:46.226524 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:46.226678 16967 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:46.219336 10047 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:46.228642 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1090) took 2135 ms. Trace:
I20260709 13:19:46.231746 17291 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:46.236873 17291 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.237402 17291 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.237536 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:46.239732 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.240091 16967 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:46.240844 10055 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:46.240974 16967 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:46.242540 16967 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.248042 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.250110 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.251616 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.252879 10129 raft_consensus.cc:2410] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:19:46.252982 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.254496 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.254886 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.255677 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.256138 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.256592 10129 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 12.
I20260709 13:19:46.259063 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.259989 16968 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.260934 16968 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.263901 16968 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.264982 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.265916 10129 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:19:46.268728 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:46.269655 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.267479 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:46.270679 16968 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:46.270947 17239 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:46.271265 16968 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.271981 16968 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:46.273249 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:46.274140 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.274955 10129 raft_consensus.cc:2393] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:46.275558 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.277141 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.278882 10129 raft_consensus.cc:2410] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:46.281265 17239 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:46.260736 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.284408 16968 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.285327 17218 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.285912 17218 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:46.287974 17242 consensus_meta.cc:287] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.000s
I20260709 13:19:46.288442 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.289149 10891 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:46.289387 10126 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:46.296806 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.298297 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.299337 10053 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:46.300154 10129 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:46.301147 10894 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:46.303396 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.304554 10129 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.288545 17242 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.286828 16968 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.307255 17242 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.308161 17218 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:46.303943 16854 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.004s	user 0.014s	sys 0.002s
I20260709 13:19:46.309602 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.310312 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.311256 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.311391 16968 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.311751 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.312131 10130 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:46.310462 10127 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:46.311230 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.312636 16968 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.315780 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.316186 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.314427 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:46.316973 10125 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:46.319437 10864 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:46.319938 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.320664 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:46.321702 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:46.320689 17280 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:46.322067 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:46.322547 17280 raft_consensus.cc:697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:46.323341 17297 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:46.323947 17297 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.323807 17280 consensus_queue.cc:237] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.324502 17297 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:46.326968 16968 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.333247 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.334295 10122 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.337145 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.338338 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.347698 10127 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:46.353572 10205 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:46.333261 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:46.353269 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.354707 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.231848 10893 rpcz_store.cc:276] 0709 13:19:44.093556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.125513 (+ 31957us) service_pool.cc:224] Handling call
0709 13:19:44.138467 (+ 12954us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:44.157522 (+ 19055us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:44.317832 (+160310us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b18004b1e40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:19:44.459624 (+141792us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.602628 (+143004us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.081537 (+478909us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b18004b1e40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:45.889752 (+808215us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:46.029602 (+139850us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b18004b1e40. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:19:46.111724 (+ 82122us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:46.112534 (+   810us) log.cc:844] Serialized 52 byte log entry
0709 13:19:46.113812 (+  1278us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:46.113870 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.113974 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.228425 (+114451us) raft_consensus.cc:1685] finished
0709 13:19:46.228446 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.228580 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":637,"prepare.queue_time_us":69519,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":3500,"raft.queue_time_us":96095,"raft.run_cpu_time_us":656,"raft.run_wall_time_us":86027,"spinlock_wait_cycles":79061632,"thread_start_us":271365,"threads_started":3,"wal-append.queue_time_us":207167}
I20260709 13:19:46.355216 10204 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:46.361985 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.370574 10126 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:19:46.374797 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.361912 17230 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.388550 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:46.388799 17230 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.389904 17251 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:19:46.390864 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.391482 17290 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.391032 17288 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:46.392405 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.382956 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.394065 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:46.394358 10196 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:46.382381 10125 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:46.383973 10131 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:46.381853 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.384546 10126 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:46.386528 17239 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:46.401810 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.402132 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.387426 10203 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:19:46.390167 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.370772 10865 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:46.393508 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:46.407222 10127 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:19:46.353943 16968 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:46.411475 10122 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:46.417434 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.419620 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.381489 17286 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:46.426486 10050 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:46.428980 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.430469 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.392884 17237 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:46.383634 17281 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:46.399785 17071 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.432936 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:46.355878 16940 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:19:46.437016 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.449596 10056 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:19:46.447348 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.446688 10204 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:46.437021 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.451712 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.457475 10049 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:19:46.410517 17210 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:46.459682 17105 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.460379 17105 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:46.465520 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.465943 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.408701 17230 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.471223 10055 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:46.467648 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.474175 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.474735 10126 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:46.475409 10049 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:46.394447 17287 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 13:19:46.455425 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.478673 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.479487 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.482963 10895 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:46.487015 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.483892 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.487651 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.485910 10050 raft_consensus.cc:2410] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:46.480329 17286 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:46.489392 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.490101 10050 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:46.392354 17290 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.419727 17289 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:46.494864 10052 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:46.495529 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.495703 10056 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:46.491020 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.497439 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.499337 10053 raft_consensus.cc:2410] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:19:46.496886 10050 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:46.502736 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.503507 10049 raft_consensus.cc:2410] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:46.503675 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.507344 10049 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:19:46.510090 10130 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:46.512952 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.513866 10056 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:46.515740 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.516187 17105 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.518625 10052 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:46.519203 10048 raft_consensus.cc:2393] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:46.524366 17290 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:46.457536 17237 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.528462 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3331) took 1588 ms. Trace:
I20260709 13:19:46.529078 10051 rpcz_store.cc:276] 0709 13:19:44.940095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.946632 (+  6537us) service_pool.cc:224] Handling call
0709 13:19:44.973941 (+ 27309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.974185 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:45.130720 (+156535us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.985382 (+854662us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b1800370ae0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287
0709 13:19:46.008480 (+ 23098us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.527402 (+518922us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b1800370ae0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:19:46.527680 (+   278us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.527953 (+   273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.528022 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:46.528120 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.528213 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.528361 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39890,"prepare.queue_time_us":165191,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":123,"raft.queue_time_us":12276,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":11028,"spinlock_wait_cycles":12289920,"thread_start_us":586703,"threads_started":3,"wal-append.queue_time_us":480074}
I20260709 13:19:46.532800 17171 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.464461 17071 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.530300 17105 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.536942 17173 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:46.537475 17173 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:46.538192 17071 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.541522 17267 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.476509 17281 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:46.534427 10893 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:46.434111 17096 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:19:46.434621 17288 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:19:46.551505 10201 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:46.553438 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:46.551561 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 180845us
I20260709 13:19:46.542891 17267 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.555614 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.547031 17286 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:46.547418 17203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.012s	sys 0.020s
I20260709 13:19:46.558082 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.559000 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.558009 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.566553 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.574867 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.561496 10055 raft_consensus.cc:2376] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17. Re-sending same reply.
I20260709 13:19:46.576296 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.578847 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.528519 17237 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.544179 17289 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:46.582690 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:46.587764 10056 raft_consensus.cc:2376] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19. Re-sending same reply.
I20260709 13:19:46.592422 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:46.560920 17203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.013s	sys 0.020s
I20260709 13:19:46.586561 10020 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:46.538726 17173 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.562343 17288 consensus_queue.cc:1243] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:46.601519 10051 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:19:46.602231 10051 pending_rounds.cc:85] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 9
I20260709 13:19:46.582129 16967 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:19:46.597369 17210 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.594615 17291 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.612556 17292 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.650578 17292 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.651429 17281 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.587363 17230 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.595871 17170 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:46.617396 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.575436 17286 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:46.646936 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.672497 10050 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:46.616035 10051 pending_rounds.cc:107] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:19:46.634146 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.689193 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.690868 10050 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:19:46.635202 10052 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:46.639834  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:46.628506 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.693856 10051 raft_consensus.cc:2889] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:46.690546 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56498.6us
I20260709 13:19:46.695997 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.643277 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.616662 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.696746 10122 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:46.658972 17266 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.584s	user 0.003s	sys 0.000s
I20260709 13:19:46.697832 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:46.696979 10126 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:46.645802 10123 raft_consensus.cc:2955] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 5 to 10, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:46.620393 16967 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:46.653257 17292 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.696033 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:46.613159 17203 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:46.729033 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.683199 17251 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.662479  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 806f35e66c234d569cbafd8b3981518f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:46.643729 17291 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.644909 17171 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:46.710959 17242 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.714115 16840 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.715214 16967 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:46.687498 17202 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:19:46.768651 17237 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:46.785740 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56788.8us
I20260709 13:19:46.817977 10096 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.627918 17210 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.757305 10122 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:46.779423 17239 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:46.781656 17170 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:46.804352  9939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 141940us
W20260709 13:19:46.805125 17299 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:19:46.735056 17292 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.627977 17289 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.824120 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.718258 17230 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.840229 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:46.831553 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:46.801481 10126 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:46.774524 17251 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.842440 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.851511 10125 raft_consensus.cc:2393] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:46.836896 17297 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.511s	user 0.014s	sys 0.004s
I20260709 13:19:46.823264 16968 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:46.855734 10122 raft_consensus.cc:2410] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:46.855516 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:46.856645 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:46.858270 10201 raft_consensus.cc:1240] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:46.781740 17242 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.872400 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.811510 16840 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.878121 17222 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:46.883070 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.884940 16968 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.883914 17222 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.887792 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:46.889057 17222 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:46.854364 17237 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:46.890547 17237 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:46.891181 17237 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:46.891748 17237 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:46.892427 16837 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.893236 16837 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.899163 17251 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:46.861363 17239 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:46.843747 17071 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:46.861471 17292 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:46.913618 10169 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:46.915805 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.887825 10197 raft_consensus.cc:1217] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Deduplicated request from leader. Original: 17.8->[18.9-18.9]   Dedup: 18.9->[]
I20260709 13:19:46.919597 10197 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:19:46.883991 10200 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:46.900327 17279 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:46.901988 10203 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:46.902870 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:46.862033 17228 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.933595 10124 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:19:46.935292 10054 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:46.929729 10126 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:19:46.924535 16940 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.879920 16836 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.944180 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3346) took 1206 ms. Trace:
I20260709 13:19:46.945278 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:46.942094 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3298) took 1213 ms. Trace:
I20260709 13:19:46.959309 10126 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:46.958968 10124 rpcz_store.cc:276] 0709 13:19:45.729260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.912640 (+1183380us) service_pool.cc:224] Handling call
0709 13:19:46.913059 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.941669 (+ 28610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.942005 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.957940 10054 rpcz_store.cc:276] 0709 13:19:45.737384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.737561 (+   177us) service_pool.cc:224] Handling call
0709 13:19:46.934585 (+1197024us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b5800003398. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfddfc0 00007fa27cfc8929 00007fa27cfc83d2 00007fa27653fbd4
0709 13:19:46.934949 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.943720 (+  8771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.944113 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2853760}
I20260709 13:19:46.906863 17230 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:46.862155 17297 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:46.941465 17176 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.936447 17279 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:46.931727 17222 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.948482 17243 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.958041 17251 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:47.069876 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:47.075989 10048 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.957662 17228 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.087509 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:47.088028 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.973745 17288 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 13:19:47.090286 10049 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:47.091874 10170 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:47.093221 10895 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:46.962009 16940 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.097886 17171 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:47.098603 17171 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.101382 17171 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.039580 17126 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:47.107362 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:47.112202 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:47.113797 10893 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:47.023304 17176 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.094944 16837 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:47.134464 10054 raft_consensus.cc:1240] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:46.971800 16836 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.136371 16840 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:47.136772 10055 raft_consensus.cc:1240] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:47.136948 17105 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:47.138505 16940 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:47.037283 17222 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.140156 17220 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:47.108621 17279 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:47.142275 17267 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.002s
I20260709 13:19:47.047257 17251 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.143502 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:47.185585 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:47.186110 10127 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.144889 17228 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:47.201481 10893 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:47.145835 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:47.149094 17288 consensus_queue.cc:1243] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 24.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:19:47.149870 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:47.080631 17239 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:47.139707 17303 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:19:47.088475 17292 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.166584 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.209764 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:47.210559 10055 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:47.105067 17210 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:47.042971 17243 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.172911 17071 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:47.188460 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 333024us
I20260709 13:19:47.207592 10895 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:47.162050 17105 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:47.223623 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 310129us
I20260709 13:19:47.224208 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:47.186574 10050 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.171844 17220 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.259223 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:47.259965 10048 raft_consensus.cc:2468] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:47.273309 10048 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:47.259207 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:47.275177 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 119841us
I20260709 13:19:47.274962 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:47.316810 10048 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 17 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 13:19:47.276474 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49580.2us
I20260709 13:19:47.278476 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:47.326510 10048 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:47.209354 17281 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.227950 17228 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.182852 16940 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:19:47.303069 10127 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:47.214673 17239 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.297435 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3337) took 2355 ms. Trace:
I20260709 13:19:47.339143 10047 rpcz_store.cc:276] 0709 13:19:44.941855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.231569 (+289714us) service_pool.cc:224] Handling call
0709 13:19:45.232010 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.228833 (+996823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.296987 (+1068154us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54001b7954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:47.297324 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270101888}
W20260709 13:19:47.298084 17203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.020s	sys 0.016s
I20260709 13:19:47.278991 17288 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:47.330824 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:47.352226 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:47.304286 10055 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:19:47.335945 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65119us
I20260709 13:19:47.354780 10128 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:47.337106 10168 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:47.358809 17325 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:47.359871 17325 consensus_queue.cc:1243] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:19:47.360499 17325 raft_consensus.cc:2804] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:47.360714 17280 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:47.361037 17325 raft_consensus.cc:697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:47.361197 17280 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:47.362187 17325 consensus_queue.cc:237] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.364514 17330 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:47.365020 17330 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:47.365236 17272 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.475s	user 0.004s	sys 0.007s
I20260709 13:19:47.366288 17176 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:47.365967 17330 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.332903 17228 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.368121 17297 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.369391 17228 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:47.369602 17222 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:47.341724 17281 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.343576 17203 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:47.373229 17281 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.310931 17242 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.362274 17280 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.299258 17210 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:47.378768 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:47.384491 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 291153us
I20260709 13:19:47.389534 10894 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:47.380019 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:47.398442 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:47.336338 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.413919 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.402343 17220 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:47.232054 17279 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.452827 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:47.453950 10894 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:47.371953 17173 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:47.462535 10894 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:47.418421 17210 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.469092 10894 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:47.469498 17239 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.471360 17251 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.471758 17243 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:47.338166 17105 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.485059 10050 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:47.409937 17281 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.416589 17330 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:47.501219 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 242112us
I20260709 13:19:47.502545 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:47.504679 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106364us
I20260709 13:19:47.374254 17203 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:47.416797  9939 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:47.484290 17210 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.274847 17071 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.508234 17105 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:47.500326 17281 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.568858 16836 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:47.652314 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 199868us
W20260709 13:19:47.649214 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 291312us
I20260709 13:19:47.461189 17267 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:19:47.515698 17318 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:47.657791 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.659878 10201 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:47.657354 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.656984 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3788) took 1342 ms. Trace:
I20260709 13:19:47.683034 10130 rpcz_store.cc:276] 0709 13:19:46.314907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.316803 (+  1896us) service_pool.cc:224] Handling call
0709 13:19:46.317161 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.510746 (+193585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.548842 (+1038096us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54002d0d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:47.656857 (+108015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442942848}
I20260709 13:19:47.658371 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:47.690877 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3302) took 1007 ms. Trace:
I20260709 13:19:47.695993 10197 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:19:47.691835 10129 rpcz_store.cc:276] 0709 13:19:46.683513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.697969 (+ 14456us) service_pool.cc:224] Handling call
0709 13:19:46.698304 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.698559 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:47.040491 (+341932us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.282482 (+241991us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.689892 (+407410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:47.690003 (+   111us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:47.690397 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.690525 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.690755 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218102,"prepare.queue_time_us":450078,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":103,"raft.queue_time_us":257701,"spinlock_wait_cycles":8623360,"thread_start_us":696497,"threads_started":3,"wal-append.queue_time_us":437377}
I20260709 13:19:47.700063 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:47.658147  9939 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 246925us
W20260709 13:19:47.709949 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52703.9us
I20260709 13:19:47.711572 10019 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.655236 17326 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:47.715485 10130 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:47.715708 17326 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:47.716513 17326 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:47.690975 17210 consensus_queue.cc:1059] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:19:47.693094 17324 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:47.721553 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3300) took 1078 ms. Trace:
I20260709 13:19:47.722028 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.722121 10123 rpcz_store.cc:276] 0709 13:19:46.642951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.644954 (+  2003us) service_pool.cc:224] Handling call
0709 13:19:46.645378 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.645645 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:46.827494 (+181849us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.720717 (+893223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:47.721177 (+   460us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:47.721245 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.721325 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.721478 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310455,"raft.queue_time_us":244753,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":9294,"thread_start_us":691809,"threads_started":2,"wal-append.queue_time_us":691063}
W20260709 13:19:47.729787 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58645.1us
I20260709 13:19:47.731921 10130 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:47.733485 10130 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:47.681648 17318 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:47.693017 17096 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:47.766072 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44163.4us
I20260709 13:19:47.743156 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.767367 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:47.771369 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 268942us
I20260709 13:19:47.772732 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:47.784327 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.786701 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:47.694429 17311 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:19:47.740710 17126 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.797464 17210 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.798334 17109 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.727459 17315 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:47.816110 10170 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:47.759456 17335 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:47.765172 17318 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.719420 17251 consensus_queue.cc:1059] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 13:19:47.727246 17242 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:19:47.779423 17288 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.764961 17307 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.060s
W20260709 13:19:47.859647 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 116595us
I20260709 13:19:47.786132 17328 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 13:19:47.818217 17292 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:47.853865 17315 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.789825 17290 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:47.790154 17316 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:47.823048 17352 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:47.865615 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49626.3us
I20260709 13:19:47.830892 17335 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:47.817704 17243 consensus_queue.cc:1059] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:19:47.845166 17119 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:47.896998 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:47.901489 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 129069us
I20260709 13:19:47.825102 17210 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.820672 17267 consensus_queue.cc:1059] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 13:19:47.917264 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.893070 17325 raft_consensus.cc:2804] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:47.824947 17109 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:47.931095 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:47.929487 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:47.897205 17222 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:47.940029 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:47.942286 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:47.940585 17316 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:47.949990 17316 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:47.917307 17292 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:47.958220 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.660s (SENT) [suppressed 35 similar messages]
I20260709 13:19:47.906533 17346 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:19:47.973415 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:47.945218 17328 raft_consensus.cc:3055] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:47.941481 17243 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:19:47.976580 17237 consensus_meta.cc:287] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.063s	user 0.023s	sys 0.006s
I20260709 13:19:47.982795 17355 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:47.958050 17071 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:47.913304 17315 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.021812 17358 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.953745 17350 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:47.926079 17352 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:47.983103 17323 log.cc:927] Time spent T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.287s	user 0.000s	sys 0.002s
I20260709 13:19:47.983875 17328 raft_consensus.cc:740] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.101343 17326 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:48.090540 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 137205us
I20260709 13:19:47.948558 17222 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.094841 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 136647us
W20260709 13:19:48.085664 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 145694us
W20260709 13:19:48.109045 17342 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:19:48.111737 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3311) took 1189 ms. Trace:
I20260709 13:19:48.256453 10127 rpcz_store.cc:276] 0709 13:19:46.922791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.381652 (+458861us) service_pool.cc:224] Handling call
0709 13:19:47.382130 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.656398 (+274268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:47.656516 (+   118us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.678271 (+ 21755us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.949728 (+271457us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:47.949794 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.949887 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.111445 (+161558us) raft_consensus.cc:1685] finished
0709 13:19:48.111471 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.111653 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149345,"raft.queue_time_us":293186,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":10971,"thread_start_us":301197,"threads_started":2,"wal-append.queue_time_us":292225}
W20260709 13:19:48.112749 17349 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:19:48.117956 17338 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 13:19:48.124027 17337 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:19:48.128257 17316 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.304100 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3308) took 1469 ms. Trace:
I20260709 13:19:48.304586 10122 rpcz_store.cc:276] 0709 13:19:46.834359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.902245 (+ 67886us) service_pool.cc:224] Handling call
0709 13:19:46.902828 (+   583us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:46.929447 (+ 26619us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:47.109208 (+179761us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.656299 (+547091us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.689824 (+ 33525us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:47.704726 (+ 14902us) log.cc:844] Serialized 52 byte log entry
0709 13:19:47.704900 (+   174us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:47.704937 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.705040 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.303713 (+598673us) raft_consensus.cc:1685] finished
0709 13:19:48.303742 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.304001 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99015,"prepare.queue_time_us":200296,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":5752,"raft.queue_time_us":356455,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":9836,"spinlock_wait_cycles":58176768,"thread_start_us":508338,"threads_started":2,"wal-append.queue_time_us":809409}
I20260709 13:19:48.128844 17344 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:48.307492 17344 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.308145 17344 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.129629 17119 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.002393 17281 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:48.133741 17343 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:19:48.140882 17352 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:48.142168 17346 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:48.314347 17322 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.950s	user 0.006s	sys 0.025s
W20260709 13:19:48.143594 17336 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:19:48.092235 17350 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:48.143750 17328 consensus_queue.cc:260] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.086988 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113637us
I20260709 13:19:48.317806 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3310) took 1449 ms. Trace:
I20260709 13:19:48.320371 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:48.150624 17330 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:19:48.005451 17237 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.102025 17358 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.005697 17335 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.089035 17307 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:48.092495 17359 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.130370 17292 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.010985 17355 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:48.158604 17109 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:48.271008 17357 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.240046 17210 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:48.275475 17220 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.824s	user 0.005s	sys 0.013s
I20260709 13:19:48.277925 17222 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:48.280747 17315 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.283092 17364 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.292178 17242 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.316797 17350 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:48.345793 17307 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.325037 10130 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:48.326887 10199 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:48.333307 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:48.318514 10126 rpcz_store.cc:276] 0709 13:19:46.868658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.976392 (+107734us) service_pool.cc:224] Handling call
0709 13:19:47.025993 (+ 49601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.026371 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:47.026402 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.041212 (+ 14810us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.494923 (+453711us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:47.494989 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.495097 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.317419 (+822322us) raft_consensus.cc:1685] finished
0709 13:19:48.317456 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.317708 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9195,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":9887,"thread_start_us":378650,"threads_started":1,"wal-append.queue_time_us":599467}
W20260709 13:19:48.336428 10123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.007s	sys 0.008s
I20260709 13:19:48.322451 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:48.163870 17347 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:48.354920 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.348847 17355 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:48.330670 17203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.011s
I20260709 13:19:48.465577 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.466305 10891 raft_consensus.cc:1813] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:48.468580 10891 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:48.469183 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.469607 10897 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:19:48.470161 10897 pending_rounds.cc:85] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 12
I20260709 13:19:48.470510 10897 pending_rounds.cc:107] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:19:48.470861 10897 raft_consensus.cc:2889] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.472357 10205 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:48.473356 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.473807 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.474407 10203 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
W20260709 13:19:48.464269 17279 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.003s	user 0.011s	sys 0.007s
I20260709 13:19:48.359071 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 926) took 1774 ms. Trace:
I20260709 13:19:48.344043 17281 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:48.372910 10123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.009s	sys 0.009s
I20260709 13:19:48.323293 17352 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:48.442534 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.376989 17347 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.508631 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.509419 17347 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.516978 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.525583 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3318) took 1492 ms. Trace:
I20260709 13:19:48.526425 10124 rpcz_store.cc:276] 0709 13:19:47.032987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.730939 (+697952us) service_pool.cc:224] Handling call
0709 13:19:47.774750 (+ 43811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.784657 (+  9907us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:47.784705 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.785311 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.332886 (+547575us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:48.332965 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.333594 (+   629us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.525098 (+191504us) raft_consensus.cc:1685] finished
0709 13:19:48.525150 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.525435 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35324,"raft.queue_time_us":153625,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":9855,"thread_start_us":474537,"threads_started":1,"wal-append.queue_time_us":651665}
I20260709 13:19:48.529518 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.530593 10124 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:48.532876 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.418226 16967 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.378948 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 56615us
I20260709 13:19:48.475673 17230 raft_consensus.cc:3055] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:48.540508 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:48.541678 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 254617us
W20260709 13:19:48.422375 17305 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.572s	user 0.007s	sys 0.007s
I20260709 13:19:48.374248 17236 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.413107 17126 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:48.552713 17126 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.553890 17126 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.554878 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.383626 10054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3358) took 1195 ms. Trace:
I20260709 13:19:48.378279 17359 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.557991 17280 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:48.368674 17326 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:48.367398 17335 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:48.557253 10054 rpcz_store.cc:276] 0709 13:19:47.188536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.188710 (+   174us) service_pool.cc:224] Handling call
0709 13:19:47.227873 (+ 39163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.228417 (+   544us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:47.228462 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.231226 (+  2764us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.598193 (+366967us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:47.598269 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.598388 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.383284 (+784896us) raft_consensus.cc:1685] finished
0709 13:19:48.383320 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.383523 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51127,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":13788,"thread_start_us":283031,"threads_started":1,"wal-append.queue_time_us":498241}
I20260709 13:19:48.562085 17286 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.370877 17237 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:48.370008 17173 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.002s
I20260709 13:19:48.395579 17361 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:19:48.566203 17324 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.566830 17324 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.567619 17120 raft_consensus.cc:3055] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:48.568143 17120 raft_consensus.cc:740] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.412592 17358 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.569229 17120 consensus_queue.cc:260] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.570132 17366 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:48.570640 17366 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.570755 17120 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:48.597287 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.599547 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.600425 10129 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:48.601565 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.604607 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3324) took 1569 ms. Trace:
I20260709 13:19:48.605165 10130 rpcz_store.cc:276] 0709 13:19:47.034655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.780296 (+745641us) service_pool.cc:224] Handling call
0709 13:19:47.780769 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.371254 (+590485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.604275 (+233021us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b540023afd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:48.604531 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250906496}
I20260709 13:19:48.397183 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.610627 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.611357 10127 raft_consensus.cc:1813] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:48.611997 10127 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:48.612984 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.613670 10127 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:48.427561 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.616287 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.617702 16940 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:48.618288 16940 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:48.619762 17321 raft_consensus.cc:3055] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:19:48.620309 17321 raft_consensus.cc:740] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:48.617714 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.621291 17321 consensus_queue.cc:260] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.622735 17321 raft_consensus.cc:3060] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:48.359544 17220 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.625437 17220 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:48.362951 17242 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.475718 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.475973 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.626734 10896 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:48.626981 10200 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:48.628047 17242 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:48.628772 10200 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: [15.13-19.15]
I20260709 13:19:48.475674 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.476655 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:48.471853 17289 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.561s	user 0.012s	sys 0.006s
I20260709 13:19:48.631939 10197 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:48.361361 17318 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.353766 17325 raft_consensus.cc:697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:48.355755 17357 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:48.465538 17203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.003s	sys 0.014s
I20260709 13:19:48.602905 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.500970 10053 rpcz_store.cc:276] 0709 13:19:46.584737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.584910 (+   173us) service_pool.cc:224] Handling call
0709 13:19:46.585334 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.599656 (+ 14322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.599697 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:46.614143 (+ 14446us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.175440 (+561297us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:47.175511 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.175629 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.313373 (+1137744us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:48.356324 (+ 42951us) raft_consensus.cc:1685] finished
0709 13:19:48.356351 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.358968 (+  2617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15274,"raft.queue_time_us":13947,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":60377,"spinlock_wait_cycles":24550656,"thread_start_us":460159,"threads_started":1,"wal-append.queue_time_us":680209}
I20260709 13:19:48.367538 17364 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.367347 17360 consensus_queue.cc:1175] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e9a849ac7eda426abfca97cf8b15da0f" responder_term: 21 status { last_received { term: 21 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 21 index: 12 } }
I20260709 13:19:48.434996 10201 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:48.545138 10051 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:48.548837 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.548889 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.653360 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.534245 17281 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.479686 17307 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:48.393764 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.653918 10124 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:48.658699 10126 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:48.571732 17366 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.573155 16967 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:48.545722 17352 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:19:48.576020 17306 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.725s	user 0.000s	sys 0.015s
W20260709 13:19:48.577180 17304 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.680s	user 0.009s	sys 0.006s
I20260709 13:19:48.671830 10892 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:48.673950 10198 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:19:48.546247 17230 raft_consensus.cc:740] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:48.579773 17291 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:19:48.682453 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3793) took 2343 ms. Trace:
I20260709 13:19:48.619391 16940 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.698438 10126 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:48.635939 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.699810 10127 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:48.374709 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:48.459800 16840 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.634963 10020 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:48.606685 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:48.702296 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.641429 10129 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:48.637149 17289 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.727s	user 0.013s	sys 0.007s
I20260709 13:19:48.664703 10124 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:48.530634 17344 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.669072 17350 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.716450 17366 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:48.717871 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.667323 17355 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.687031 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.689399 10199 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 929) took 1030 ms. Trace:
I20260709 13:19:48.588936 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3356) took 1422 ms. Trace:
I20260709 13:19:48.735001 10127 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:48.734251 10052 rpcz_store.cc:276] 0709 13:19:47.166318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.170400 (+  4082us) service_pool.cc:224] Handling call
0709 13:19:47.209305 (+ 38905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.258342 (+ 49037us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:47.492822 (+234480us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.665121 (+172299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:47.665242 (+   121us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:47.665343 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.588595 (+923252us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b540010b8d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:48.588649 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.588857 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2066,"prepare.queue_time_us":459060,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":5171,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":27199,"spinlock_wait_cycles":112062720,"thread_start_us":363793,"threads_started":2,"wal-append.queue_time_us":405582}
I20260709 13:19:48.735766 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3329) took 1511 ms. Trace:
I20260709 13:19:48.675491 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3306) took 1859 ms. Trace:
I20260709 13:19:48.733968 10130 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:48.736167 10127 rpcz_store.cc:276] 0709 13:19:47.224886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.700664 (+1475778us) service_pool.cc:224] Handling call
0709 13:19:48.734592 (+ 33928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.735452 (+   860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.735683 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:48.733398 10199 rpcz_store.cc:276] 0709 13:19:47.658818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.658974 (+   156us) service_pool.cc:224] Handling call
0709 13:19:47.687863 (+ 28889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.372111 (+684248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.685424 (+313313us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54001ef5d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:48.689283 (+  3859us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468181760}
W20260709 13:19:48.694715 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 401727us
W20260709 13:19:48.697342 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.690071 10131 rpcz_store.cc:276] 0709 13:19:46.338565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.419411 (+ 80846us) service_pool.cc:224] Handling call
0709 13:19:46.453761 (+ 34350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.529264 (+ 75503us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:46.843750 (+314486us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.205171 (+361421us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.469011 (+263840us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b1800468be0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:47.496991 (+ 27980us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b1800468be0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:19:47.497227 (+   236us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.497899 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.498103 (+   204us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:47.508393 (+ 10290us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.655951 (+147558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.681878 (+1025927us) spinlock_profiling.cc:243] Waited 75.1 ms on lock 0x7b540023bed4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:48.681969 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.682057 (+    88us) raft_consensus.cc:1685] finished
0709 13:19:48.682076 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.682342 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":559616,"prepare.queue_time_us":587725,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":157,"raft.queue_time_us":528699,"raft.run_cpu_time_us":550,"raft.run_wall_time_us":25616,"spinlock_wait_cycles":201462016,"thread_start_us":585536,"threads_started":2,"wal-append.queue_time_us":543124}
I20260709 13:19:48.700953 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3327) took 1548 ms. Trace:
I20260709 13:19:48.740983 10126 rpcz_store.cc:276] 0709 13:19:47.152177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.699811 (+1547634us) service_pool.cc:224] Handling call
0709 13:19:48.700336 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.700603 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.700629 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:48.700674 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:48.700710 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.700769 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.700874 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:48.701396 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 326725us
I20260709 13:19:48.361956 17365 consensus_queue.cc:1059] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
W20260709 13:19:48.704838 10865 connection.cc:256] RPC call timeout handler was delayed by -1.0245241760001136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:19:48.703498 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 68622.1us
I20260709 13:19:48.748829 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:48.673372 17281 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.713069 17344 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:48.713928 10204 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:48.713285 17325 consensus_queue.cc:237] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:48.718731 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.763578 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1106) took 1107 ms. Trace:
I20260709 13:19:48.708442 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3325) took 1673 ms. Trace:
I20260709 13:19:48.764731 10894 rpcz_store.cc:276] 0709 13:19:47.655760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.655947 (+   187us) service_pool.cc:224] Handling call
0709 13:19:48.686759 (+1030812us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b58001d0798. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274edfb71 00007fa274edfa22 00007fa27ddf8c09 00007fa27cf52ee0 00007fa27d01c0ab 00007fa27d01c131 00007fa27cfca0a0 00007fa27cfc9984 00007fa2765446d4 00007fa276544662
0709 13:19:48.763478 (+ 76719us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24490880}
I20260709 13:19:48.765072 10124 rpcz_store.cc:276] 0709 13:19:47.034925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.659249 (+1624324us) service_pool.cc:224] Handling call
0709 13:19:48.659668 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.705460 (+ 45792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.708356 (+  2896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:48.765906 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.760480 10049 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 13:19:48.766695 10131 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:48.703014 17352 raft_consensus.cc:2764] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:48.772275 10129 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:48.775182 10129 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:19:48.776842 10129 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:19:48.746606 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.748080 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:48.762372 10128 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:19:48.778791 10122 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:48.778671 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 237342us
I20260709 13:19:48.762423 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:48.763569 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:48.780212 10047 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:19:48.780890 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:48.781657 10894 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:48.766856 10049 pending_rounds.cc:85] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 15
I20260709 13:19:48.781059 10893 raft_consensus.cc:1240] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:48.782665 10049 pending_rounds.cc:107] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 13:19:48.781044 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.781354 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:48.783064 10049 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.736340 10125 rpcz_store.cc:276] 0709 13:19:46.816320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.855250 (+ 38930us) service_pool.cc:224] Handling call
0709 13:19:46.960689 (+105439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.961021 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:47.279558 (+318537us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.766933 (+487375us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.674872 (+907939us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b180022e520. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287
0709 13:19:48.675009 (+   137us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:48.675062 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:48.675132 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.675228 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.675396 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100955,"prepare.queue_time_us":198760,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":101,"raft.queue_time_us":307669,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":18247,"spinlock_wait_cycles":21376384,"thread_start_us":795582,"threads_started":4,"wal-append.queue_time_us":811805}
I20260709 13:19:48.768780 10131 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:19:48.783823 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.784828 10124 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:48.783890 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:48.789315 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:48.750751 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:48.791610 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:48.792179 10205 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:48.792431 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:48.792475 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.793720 10126 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:48.739373 17202 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:48.801237 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.710985 17372 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:48.802124 10200 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:48.802488 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3317) took 1769 ms. Trace:
I20260709 13:19:48.760103 16940 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.807350 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.803090 10123 rpcz_store.cc:276] 0709 13:19:47.032686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.725122 (+692436us) service_pool.cc:224] Handling call
0709 13:19:47.725384 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.725571 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:47.759222 (+ 33651us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.778453 (+1019231us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.802064 (+ 23611us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:48.802136 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:48.802192 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.802282 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.802416 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34222,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":117,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":127,"thread_start_us":1018376,"threads_started":2,"wal-append.queue_time_us":699671}
W20260709 13:19:48.805126 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54517.4us
I20260709 13:19:48.771605 17239 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:48.708716 16840 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.826452 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:48.808519 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:48.827929 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:48.829213 10894 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
I20260709 13:19:48.822046 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.813900 10056 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:48.808324 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.783102 17288 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:48.831070 10123 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:48.832382 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.833379 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.836719 10197 raft_consensus.cc:2410] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:19:48.835026 10125 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:48.834244 10127 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.842482 10897 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:48.851732 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.852594 10131 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:48.760576 17256 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:19:48.803298 17287 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:48.856469 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.857892 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:48.858773 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:48.865626 10122 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:48.861650 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:48.766206 17237 consensus_queue.cc:1059] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:19:48.873888 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65447us
I20260709 13:19:48.813604 17326 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:48.813342 16852 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:48.795220 17281 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:19:48.874065 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.887787 10124 raft_consensus.cc:2376] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19. Re-sending same reply.
I20260709 13:19:48.888958 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.803472 17242 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:48.863523 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:48.890224 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.891003 10127 raft_consensus.cc:2376] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17. Re-sending same reply.
I20260709 13:19:48.880542 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.811268 17372 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.796602 17292 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:19:48.837493 17173 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:48.824265 17344 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:19:48.939249 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.942821 10200 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:48.886250 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.835873 17288 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:48.883622 10198 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:19:48.852427 17279 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:48.944387 10203 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 19 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:48.910642 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.889781 10124 raft_consensus.cc:2376] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20. Re-sending same reply.
I20260709 13:19:48.895210 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:48.874477 17369 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:19:48.874250 17203 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:48.865065 17287 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.931552 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:48.885329 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.952698 10199 raft_consensus.cc:2410] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:48.806169 17202 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:48.947207 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:48.881951 16852 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.907933 17367 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.853847 17308 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 13:19:48.892351 17225 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:48.947369 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 64710.4us
I20260709 13:19:48.945402 10126 raft_consensus.cc:2376] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22. Re-sending same reply.
I20260709 13:19:48.876562 17315 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:19:48.896538 17242 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:48.802708 17230 consensus_queue.cc:260] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:48.964604 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:48.984419 10127 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.966955 17279 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.969928 17365 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:48.899955 17372 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.958125 17287 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:48.949687 17203 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:48.981024 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 108006us
I20260709 13:19:48.895943 17237 raft_consensus.cc:3055] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:48.992087 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45001.9us
I20260709 13:19:48.957093 17291 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:19:48.963967 16852 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:48.995893 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74458.2us
I20260709 13:19:48.999493 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:49.001010 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:48.997853 10169 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:49.006592 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 144987us
I20260709 13:19:49.002542 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:48.972724 17225 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.930843 17243 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:19:48.886798 17374 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 13:19:48.973596 17318 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:48.992373 17346 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:48.981148 17242 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:49.019168 17279 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:49.019294 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:49.022522 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 928) took 1942 ms. Trace:
I20260709 13:19:48.981618 17357 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.029428 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:48.971949 17222 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:19:49.006353 17203 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.002190 17220 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.040755 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.007154 17237 raft_consensus.cc:740] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:49.071736 10051 rpcz_store.cc:276] 0709 13:19:47.080270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.080503 (+   233us) service_pool.cc:224] Handling call
0709 13:19:49.022270 (+1941767us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400235ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:49.022446 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3299385088}
I20260709 13:19:48.971179 17367 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.076120 10128 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
W20260709 13:19:49.076517 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 68915.8us
I20260709 13:19:49.096894 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:49.038932 17324 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.083040 10127 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:49.032752 17230 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:49.031304 17225 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:49.077760 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:49.069034 17279 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:49.123045 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 125292us
I20260709 13:19:49.042989 17346 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:49.069609 17355 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:49.118643 17120 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.000s
I20260709 13:19:49.119478 16840 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:49.119736 17381 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:19:49.092443 17220 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:49.121037 17379 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:19:49.129693 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 100332us
I20260709 13:19:49.074648 17229 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:19:49.174527 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:49.157675 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 139943us
I20260709 13:19:49.159618 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:49.125519 17318 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.125002 17324 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:49.164846 10200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:49.126397 17279 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:49.131067 17346 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:49.206939 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:49.185266 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.096247 17386 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:19:49.237632 10894 raft_consensus.cc:2393] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 25: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:49.137111 17347 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:19:49.215754 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.155362 17374 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:49.204790 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.219254 10200 raft_consensus.cc:606] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:49.263022 10200 raft_consensus.cc:614] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:49.221205 17324 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:49.235337 17280 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.006s
I20260709 13:19:49.236104 17394 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:19:49.214531 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117766us
W20260709 13:19:49.235455 17385 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:19:49.267938 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63034.7us
I20260709 13:19:49.276332 10895 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:19:49.277631 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.272387 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 67702.6us
I20260709 13:19:49.279536 10019 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.283236 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:49.235342 17376 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:49.286257 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:49.266079 17366 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.549s	user 0.004s	sys 0.009s
I20260709 13:19:49.291896 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3331) took 2065 ms. Trace:
I20260709 13:19:49.281277 17281 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.271032 17394 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:49.271785 17297 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:49.248070 17236 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.216573 17308 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 13:19:49.288036 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.265198 17324 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:49.315016 10130 rpcz_store.cc:276] 0709 13:19:47.226348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.708865 (+1482517us) service_pool.cc:224] Handling call
0709 13:19:48.709453 (+   588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.742432 (+ 32979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.291566 (+549134us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54001261d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:49.291820 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24079232}
W20260709 13:19:49.347052 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59132.4us
I20260709 13:19:49.282747 17374 raft_consensus.cc:697] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:49.307626 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 765063us
I20260709 13:19:49.266527 17202 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:49.345492 16967 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
I20260709 13:19:49.313721 17386 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:49.294502 17352 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.312553 17355 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.328084 17347 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:49.286645 17397 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:49.358868 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.286772 17376 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.308540 17220 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:49.310041 17367 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:49.347271 17321 consensus_meta.cc:287] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.005s
I20260709 13:19:49.347083 17236 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.368880 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.386307 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:49.341604 17281 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.305081 17398 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:49.351687 17173 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:49.368566 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:49.294301 17237 consensus_queue.cc:260] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.367010 17347 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.367306 17397 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.360745 17315 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:49.371943 16967 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:19:49.394137 17398 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.401621 17347 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:49.431823 17372 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.390039 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:49.385002 17394 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.439721 17366 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:49.440690 17366 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:49.403610 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:49.367506 17402 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:19:49.391682 10197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:49.433065 17364 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:49.384507 17365 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:19:49.393031 17384 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.407369 17363 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:19:49.372470 17352 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.381737 17376 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:49.438064 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69569.8us
I20260709 13:19:49.420296 17397 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.419441 17239 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:49.372689 17355 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.445919 10197 raft_consensus.cc:606] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:49.390319 17386 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:49.425750 17126 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:49.408201 17396 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:19:49.434083 17346 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:49.452842 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62898.5us
I20260709 13:19:49.425314 17398 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.444075 17237 raft_consensus.cc:3060] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:49.461858 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 47452.3us
I20260709 13:19:49.476579 17372 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.448700 17173 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:49.455701 17318 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:49.454205 17384 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:49.492044 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:49.495558 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:49.498860 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:49.516476 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.500102 17404 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.478600 17225 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:49.511514 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 152713us
I20260709 13:19:49.479887 17386 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.474362 16852 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.489428 17315 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.524436 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:49.523874 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:49.542834 10131 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:49.525516 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.419066 17374 consensus_queue.cc:237] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.511945 17318 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.518710 17384 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.548195 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:49.527788 16940 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.713s	user 0.015s	sys 0.000s
I20260709 13:19:49.553207 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.534503 17386 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.542189 17321 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:49.545835 17372 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:49.569392 10197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:49.571522 17173 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.547351 17291 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.575347 17365 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:49.576433 10895 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1101) took 2198 ms (client timeout 1574 ms). Trace:
I20260709 13:19:49.578151 10196 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:49.579321 10197 raft_consensus.cc:606] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:49.585867 17411 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:49.527472 17404 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.586555 17236 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.561628 17239 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:49.554014 17318 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:49.521340 17407 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:19:49.587631 17346 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:49.508875 17366 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:49.592854 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.546468 17202 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:49.582759 17281 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:49.607254 10895 rpcz_store.cc:269] 0709 13:19:47.377796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.379643 (+  1847us) service_pool.cc:224] Handling call
0709 13:19:49.576188 (+2196545us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001e7154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:49.576337 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4335747328}
W20260709 13:19:49.613168 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60069us
I20260709 13:19:49.652047 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:49.613946 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.590903 17405 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:49.546428 17315 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.608873 17365 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.609474 17321 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.646076 17419 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:49.663511 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:49.668946 17407 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.669564 17425 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.669745 17407 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.670169 17425 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.673075 17407 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.611886 17291 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.683243 17412 raft_consensus.cc:2764] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:19:49.691166 17291 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.695398 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.659332 17405 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:49.676131 17365 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:49.725642 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:49.728746 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 204381us
I20260709 13:19:49.662416 17202 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.681138 17419 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:49.728746 17355 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.688714 17407 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.746681 17202 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.738620 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.728801 17236 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:49.779654 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.677980 17321 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.743191 17096 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:49.788708 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:49.789798 10130 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:49.754382 17230 consensus_meta.cc:287] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.002s
I20260709 13:19:49.789428 10204 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.754805 17372 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:49.755014 17425 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.782425 17202 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.737072 17279 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.811385 17415 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:49.812114 17321 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.778474 17225 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.814975 17239 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:49.827332 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.821822 10053 raft_consensus.cc:1275] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:19:49.847544 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47175.5us
I20260709 13:19:49.790768 17423 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:49.878584 17417 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:19:49.810817 17427 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:49.849067 17203 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.001s
W20260709 13:19:49.880488 17409 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 13:19:49.880095 17203 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:49.845928 17242 consensus_meta.cc:287] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.002s
I20260709 13:19:49.875595 10096 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:49.876463 17315 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.878312 17423 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.879992 17427 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.822209 17347 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.755829 17407 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.790529 17355 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:19:49.800560 16967 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.816376 17352 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.787541 17398 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.848778 17364 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.881448 17242 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:49.916574 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:49.911017 16967 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.912946 10894 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: []
I20260709 13:19:49.822962 17202 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.922482 10896 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:49.858911 17236 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:49.894094 17404 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:49.935262 17352 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:19:49.935509 17429 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:19:49.942013 17407 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:49.945583 16967 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:49.954355 17395 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.735s	user 0.004s	sys 0.000s
I20260709 13:19:49.961370 17427 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.962699 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:49.968123 17347 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:49.971014 17398 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.971766 17414 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:49.892045 17419 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:49.974679 17423 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:49.935600 17329 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:49.792881 17405 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:50.000454 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83971.6us
I20260709 13:19:49.917836 17364 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.036656 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:50.011881 17324 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.010s
I20260709 13:19:50.039350 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:50.027242 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.052812 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.053921 10050 raft_consensus.cc:2410] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:19:50.055852 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.054570 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.056856 10049 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:50.057142 10048 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:50.073215 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.080199 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.081327 10049 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:50.081717 17426 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:19:50.083951 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.084677 10131 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.073446 17352 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.087807 17428 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:50.088748 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.089531 10050 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:50.090590 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.092103 17242 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:50.094502 17397 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.000s
I20260709 13:19:50.038671 17347 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.083818 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.080327 17375 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:50.094990 10049 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:50.101127 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.102015 10049 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:50.101785 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.096508 10050 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:50.103230 10125 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:50.107810 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.044772 17329 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:50.115440 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:50.117215 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 191288us
I20260709 13:19:50.063742 17355 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:50.118736 10050 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:50.121681 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.123584 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.124579 10052 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:50.125698 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.126451 10052 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:50.128381 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.064621 17203 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:50.114459 17395 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:19:50.142771 10049 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:19:50.144712 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.029727 17372 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.042809 17096 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:50.150231 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.151731 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.154824 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 799) took 1648 ms. Trace:
I20260709 13:19:50.065107 17324 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.155380 10897 rpcz_store.cc:276] 0709 13:19:48.506642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.541681 (+ 35039us) service_pool.cc:224] Handling call
0709 13:19:48.637706 (+ 96025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.843252 (+205546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.154511 (+1311259us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400662e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.154743 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470120832}
I20260709 13:19:50.096375 17397 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.087839 17352 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:50.147392 17287 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.140s	user 0.018s	sys 0.007s
I20260709 13:19:50.148983 10049 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:19:50.107331 17347 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.156697 10125 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 16: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:50.162024 17434 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:50.184471 17202 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.186982 17324 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.187990 10047 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 24: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:50.189137 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.192579 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.165263 10049 pending_rounds.cc:85] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:50.207580 10049 pending_rounds.cc:107] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:19:50.207607 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.208024 10049 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.207631 10128 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:50.150344 17279 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.195233 10131 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:50.176466 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 931) took 1426 ms. Trace:
I20260709 13:19:50.211710 10053 rpcz_store.cc:276] 0709 13:19:48.749856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.750813 (+   957us) service_pool.cc:224] Handling call
0709 13:19:48.751188 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.858839 (+1107651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.176150 (+317311us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400199454. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.176385 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2430966528}
I20260709 13:19:50.199062 10130 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:50.213969 10053 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:50.217126 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.144034 10048 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
W20260709 13:19:50.201296 17406 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.782s	user 0.003s	sys 0.000s
I20260709 13:19:50.202663 17347 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.179078 17423 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:19:50.189978 17329 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.177392 17373 raft_consensus.cc:3055] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:50.175612 17394 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:50.242390 17352 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.236032 17423 consensus_queue.cc:1243] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:19:50.231293 17236 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:50.243956 17094 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:50.244455 17094 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:50.245379 17094 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.246759 17094 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.202687 17364 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.149660 17355 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.219926 10053 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:50.219102 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.228797 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:50.232016 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.232754 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.296367 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67689.2us
I20260709 13:19:50.297379 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:50.207707 17237 consensus_meta.cc:287] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.696s	user 0.019s	sys 0.001s
I20260709 13:19:50.129720 17375 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.234117 10048 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:50.234455 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.293677 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.236325 17373 raft_consensus.cc:740] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:50.228770 17414 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:50.248540 17350 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.248754 16852 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.204865 17365 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.249339 17096 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.251619 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3344) took 1422 ms. Trace:
I20260709 13:19:50.197765 17287 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.312155 10053 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:50.254453 17288 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:50.256233 17318 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.613s	user 0.020s	sys 0.001s
I20260709 13:19:50.263670 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.345939 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.346972 10122 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:50.348532 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.349395 10122 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:50.310250 17326 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:50.266397 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50726 (request call id 935) took 1475 ms. Trace:
I20260709 13:19:50.271095 10205 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:50.273272 10128 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:50.351483 10196 rpcz_store.cc:276] 0709 13:19:48.791473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.807029 (+ 15556us) service_pool.cc:224] Handling call
0709 13:19:50.265974 (+1458945us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54001efad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:50.266279 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734964992}
I20260709 13:19:50.353529 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.354424 10128 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:19:50.274756 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3337) took 1547 ms. Trace:
I20260709 13:19:50.277030 17397 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:50.300933 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.355293 10129 rpcz_store.cc:276] 0709 13:19:48.727720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.777768 (+ 50048us) service_pool.cc:224] Handling call
0709 13:19:48.778121 (+   353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:48.864000 (+ 85879us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:49.101454 (+237454us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.363616 (+262162us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:49.364407 (+   791us) log.cc:844] Serialized 52 byte log entry
0709 13:19:49.364592 (+   185us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:49.366523 (+  1931us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.398679 (+ 32156us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.417112 (+ 18433us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:49.512351 (+ 95239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.273317 (+760966us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b1800243a00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:50.274358 (+  1041us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.274419 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.274473 (+    54us) raft_consensus.cc:1685] finished
0709 13:19:50.274485 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.274677 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32250,"prepare.queue_time_us":133974,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":132254,"raft.run_cpu_time_us":653,"raft.run_wall_time_us":15538,"spinlock_wait_cycles":246948736,"thread_start_us":500533,"threads_started":4,"wal-append.queue_time_us":229505}
W20260709 13:19:50.358134 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.301421 10095 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:50.299360 17236 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:50.238634 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.312290 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.325614 17414 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:50.317036 10048 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:50.333765 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.332427 10126 rpcz_store.cc:276] 0709 13:19:48.828797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.032020 (+203223us) service_pool.cc:224] Handling call
0709 13:19:49.078840 (+ 46820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.079225 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.079274 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.090952 (+ 11678us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.274346 (+183394us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:49.274407 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.274521 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.188387 (+913866us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:50.251234 (+ 62847us) raft_consensus.cc:1685] finished
0709 13:19:50.251257 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.251549 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64540928,"thread_start_us":144818,"threads_started":1,"wal-append.queue_time_us":253713}
W20260709 13:19:50.203933 17376 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.726s	user 0.008s	sys 0.014s
W20260709 13:19:50.362994 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61683.4us
I20260709 13:19:50.342149 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.350633 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.204924 17434 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.364193 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.366614 10196 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 25.
I20260709 13:19:50.355836 10893 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:19:50.359787 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.359012 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.369513 10130 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:50.360311 10131 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:19:50.362649 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 124089us
I20260709 13:19:50.370746 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.371433 10131 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:50.371965 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.360031 17288 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.374907 17375 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.375535 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.308393 17355 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.376598 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.378544 17355 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.369716 10129 raft_consensus.cc:2393] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:50.362106 10053 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:50.383595 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.375669 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.315775 17352 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:19:50.364234 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.387143 10126 raft_consensus.cc:2393] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:50.380574 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.385524 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.386549 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.390273 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.391026 10129 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:50.392005 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.393512 10053 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:19:50.394894 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.369841 17390 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.398532 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.367285 17326 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.366989 17350 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:50.399930 10048 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:50.400521 17355 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:50.414984 17279 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.363757 17414 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:50.415680 17279 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.418180 17279 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.420970 17434 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.421855 17434 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:50.423254 17256 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:50.424067 17256 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:50.425122 17256 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.385318 17236 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:50.365969 17237 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:50.412611 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.413551 10048 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:50.353659 17318 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.413233 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.411854 17390 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.431348 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:50.431704 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.434993 10131 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:50.438964 10126 raft_consensus.cc:2376] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19. Re-sending same reply.
W20260709 13:19:50.414119 17384 consensus_meta.cc:287] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.849s	user 0.018s	sys 0.006s
I20260709 13:19:50.442916 17287 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.444203 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.445436 10050 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:50.446022 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 938) took 1646 ms. Trace:
W20260709 13:19:50.455555 10019 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.687s (SENT)
I20260709 13:19:50.460302 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.460791 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.460114 10202 rpcz_store.cc:276] 0709 13:19:48.800048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.800189 (+   141us) service_pool.cc:224] Handling call
0709 13:19:48.835551 (+ 35362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.835932 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.835962 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.169298 (+1333336us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:19:50.169884 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.290565 (+120681us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:50.290609 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.290708 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.445597 (+154889us) raft_consensus.cc:1685] finished
0709 13:19:50.445630 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.445926 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45000,"raft.queue_time_us":36398,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":5575,"spinlock_wait_cycles":9046912,"thread_start_us":120101,"threads_started":1,"wal-append.queue_time_us":273749}
I20260709 13:19:50.368826 17376 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.457101 10204 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
W20260709 13:19:50.484337 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.499363 17372 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:50.500097 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.498910 10202 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:19:50.501596 17444 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:19:50.501839 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.504734 10126 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:50.503101 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.507452 10054 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:50.486747 10131 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:50.514739 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:50.515743 10130 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:50.511991 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109878us
W20260709 13:19:50.485348 17386 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.924s	user 0.014s	sys 0.010s
I20260709 13:19:50.458765 17279 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:50.410220 17373 consensus_queue.cc:260] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.503397 17365 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.523917 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:50.455250 17237 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.537493 10130 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:50.540935 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:50.543308 17445 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:50.543716 17373 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:50.558599 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.566236 17376 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.523166 17318 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.569164 17237 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.549005 17326 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:50.572395 17202 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.573175 17202 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.573931 17414 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.576090 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.578333 10054 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:50.576267 10019 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.875s (SENT)
I20260709 13:19:50.579929 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.483393 17384 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.577476 10051 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:19:50.582129 10051 pending_rounds.cc:85] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 13
I20260709 13:19:50.582582 10051 pending_rounds.cc:107] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:19:50.582963 10051 raft_consensus.cc:2889] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.583491 10051 pending_rounds.cc:107] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:19:50.583827 10051 raft_consensus.cc:2889] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.529759 17386 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.613631 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.621094 10055 raft_consensus.cc:1240] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:50.655777 10049 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:50.611140 17288 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:50.657401 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67640.7us
I20260709 13:19:50.662406 17329 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:50.613505 17236 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.613356 17326 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:50.700145 10896 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60866 (request call id 1109) took 1575 ms (client timeout 1729 ms). Trace:
W20260709 13:19:50.700932 10896 rpcz_store.cc:269] 0709 13:19:49.124679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.124934 (+   255us) service_pool.cc:224] Handling call
0709 13:19:50.699839 (+1574905us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54004aef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:50.700026 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806556032}
W20260709 13:19:50.706245 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.711974 10893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 539) took 1899 ms. Trace:
I20260709 13:19:50.712718 10893 rpcz_store.cc:276] 0709 13:19:48.812620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.812787 (+   167us) service_pool.cc:224] Handling call
0709 13:19:48.853523 (+ 40736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.370020 (+1516497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.711465 (+341445us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400604854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.711862 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3453961984}
I20260709 13:19:50.706353 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.720054 17357 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.720808 17357 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.721345 17357 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.748955 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 942) took 1270 ms. Trace:
I20260709 13:19:50.747855  9948 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:50.749503 10055 rpcz_store.cc:276] 0709 13:19:49.478462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.478659 (+   197us) service_pool.cc:224] Handling call
0709 13:19:49.489767 (+ 11108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.621719 (+1131952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.748618 (+126899us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540037b754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.748868 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601204096}
I20260709 13:19:50.754345 17328 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:50.756068 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2529) took 1871 ms. Trace:
I20260709 13:19:50.757395 10123 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:50.759902 10049 raft_consensus.cc:1240] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:50.761988 10049 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:50.756811 10205 rpcz_store.cc:276] 0709 13:19:48.884479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.884646 (+   167us) service_pool.cc:224] Handling call
0709 13:19:50.270867 (+1386221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.354995 (+ 84128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.755713 (+400718us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001efad0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:50.755961 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3183557376}
I20260709 13:19:50.773708 17415 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:50.775271 17415 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:50.775719 17415 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:50.776919 17415 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.766945 17363 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.768301 10123 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:19:50.768783 17228 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:50.769758 10053 raft_consensus.cc:1240] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:50.766083 17402 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:50.790318 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:50.791165 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:50.793321 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.831732 10897 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:50.793787 17390 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.789525 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:50.799687 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.835258 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.802004 10204 raft_consensus.cc:1240] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:19:50.803738 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:50.809692 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3350) took 1314 ms. Trace:
I20260709 13:19:50.836320 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:50.836673 10892 raft_consensus.cc:2393] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:50.836966 10900 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:50.810156 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 938) took 1638 ms. Trace:
I20260709 13:19:50.806061 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:50.837714 10051 rpcz_store.cc:276] 0709 13:19:49.172156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.172326 (+   170us) service_pool.cc:224] Handling call
0709 13:19:49.172777 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.584531 (+1411754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.809770 (+225239us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002e8954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.810056 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3229330688}
I20260709 13:19:50.836344 10127 rpcz_store.cc:276] 0709 13:19:49.495286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.497172 (+  1886us) service_pool.cc:224] Handling call
0709 13:19:49.497518 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.497778 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:49.514099 (+ 16321us) log.cc:844] Serialized 14 byte log entry
0709 13:19:50.808972 (+1294873us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b18003cbda0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:19:50.809239 (+   267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:50.809304 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:50.809363 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.809448 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.809610 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2567,"prepare.queue_time_us":17037,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3964,"raft.queue_time_us":260,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":53,"spinlock_wait_cycles":9813760,"thread_start_us":31823,"threads_started":1,"wal-append.queue_time_us":138761}
I20260709 13:19:50.839169 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.803642 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:50.819407 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.835973 10198 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:50.750756  9948 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 17 to 22, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 22 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:50.832613 10894 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:50.804828 17267 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.682077 17236 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.833351 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.835718 10893 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.814181 17412 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:50.854666 10169 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 1.729s (SENT)
I20260709 13:19:50.856665 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.856827 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.857577 10197 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:50.857930 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:50.841876 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:50.815557 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.863025 10198 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:19:50.863574 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.864316 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.864445 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.865381 10170 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.840953 10205 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:50.865970 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.866611 17325 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:50.839395 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:50.867081 17325 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:50.867770 17318 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.868597 10051 raft_consensus.cc:1275] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:19:50.868198 17325 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.869012 10050 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:50.869506 17325 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:50.870170 10051 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:19:50.873998 17344 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:50.874729 10201 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 937) took 2075 ms. Trace:
I20260709 13:19:50.874671 17344 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.875487 17361 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:50.875378 10201 rpcz_store.cc:276] 0709 13:19:48.799731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.799893 (+   162us) service_pool.cc:224] Handling call
0709 13:19:48.800261 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.953129 (+1152868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.953173 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:49.960333 (+  7160us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:49.960413 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.874420 (+914007us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400354654. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:50.874482 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.874650 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":2649721728}
I20260709 13:19:50.877338 17344 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.877643 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3354) took 1381 ms. Trace:
W20260709 13:19:50.880416 17447 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:19:50.882375  9948 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:50.884816  9948 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 16, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 16 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:19:50.888126 17452 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:50.847901 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.813385 17279 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:50.828539 17228 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:50.864712 17414 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.865825 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.848012 10123 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:19:50.903841 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.878901 10170 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.897146 10204 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:50.878180 10129 rpcz_store.cc:276] 0709 13:19:49.496607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.402764 (+906157us) service_pool.cc:224] Handling call
0709 13:19:50.403176 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:50.514149 (+110973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.514189 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:50.523516 (+  9327us) log.cc:844] Serialized 52 byte log entry
0709 13:19:50.686407 (+162891us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:50.808631 (+122224us) log.cc:844] Serialized 14 byte log entry
0709 13:19:50.808902 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.809001 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.877357 (+ 68356us) raft_consensus.cc:1685] finished
0709 13:19:50.877385 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.877571 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121787,"prepare.queue_time_us":122044,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":20247,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":229,"thread_start_us":208610,"threads_started":2,"wal-append.queue_time_us":319415}
I20260709 13:19:50.860376 17415 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:50.905324 10205 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:50.907399 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3829) took 1154 ms. Trace:
I20260709 13:19:50.908799 17291 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:50.908051 10131 rpcz_store.cc:276] 0709 13:19:49.753012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.524665 (+771653us) service_pool.cc:224] Handling call
0709 13:19:50.525071 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:50.527156 (+  2085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.527194 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:50.532915 (+  5721us) log.cc:844] Serialized 52 byte log entry
0709 13:19:50.687970 (+155055us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:50.688017 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.688101 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.907105 (+219004us) raft_consensus.cc:1685] finished
0709 13:19:50.907154 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.907322 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2759,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2505,"thread_start_us":154443,"threads_started":1,"wal-append.queue_time_us":360791}
I20260709 13:19:50.909580 17291 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:50.830559 17363 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.912417 17363 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:50.891992 17386 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.912593 17291 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:50.915872 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.916867 10198 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:50.917081 17363 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:50.833673 17326 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.919261 17126 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:50.919557 10170 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.920878 17126 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:50.921268 17126 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.921643 17126 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:50.921828 17363 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:50.923775 17344 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:50.928428 17202 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:50.929234 17094 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.635s	user 0.007s	sys 0.007s
I20260709 13:19:50.931407 17363 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:19:50.934427 17453 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:19:50.851935 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:50.936300 10052 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:50.935771 17363 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:50.863576 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:50.937752 17446 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:19:50.938498 10122 raft_consensus.cc:2410] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:19:50.938148 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86812.6us
I20260709 13:19:50.939785 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.831336 17402 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:50.941440 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.941216 17402 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.942468 17279 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:50.907299 17366 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:50.846261 16840 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:50.943209 17291 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:50.945324 10197 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: [15.13-19.15]
I20260709 13:19:50.942785 17402 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.942839 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.948597 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.937752 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:50.880496 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:50.936568 17454 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 13:19:50.950623 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:50.951643 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.954521 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.958096 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.859051 17292 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:50.969208 10199 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:50.971783 10196 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:19:50.974068 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:50.974637 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:50.978021 10893 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:50.945809 17407 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:50.982283 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.980881 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:50.868284 17318 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.008826 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:50.986536 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:51.002210 10048 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
I20260709 13:19:50.943311 17279 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:51.012177 10196 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:19:50.903491 17328 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:51.014081 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.948132 17366 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.012595 17453 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.078s	user 0.003s	sys 0.000s
I20260709 13:19:51.017511 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.007786 17291 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:51.019550 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:50.982268 17442 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:19:51.022560 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:51.015249 17335 consensus_queue.cc:1059] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:19:50.851487 17267 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.015650 17318 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.057770 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3355) took 1560 ms. Trace:
I20260709 13:19:50.953433 16840 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:50.982636 17202 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:51.076411 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.021276 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.983541 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:50.978847 17363 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:51.050429 17358 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:51.008062 17407 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:51.040337 17366 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:51.052350 17454 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:51.123101 10200 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:19:50.956177 17344 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:19:51.033963 17292 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.097743 17126 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.128657 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1113) took 1683 ms. Trace:
I20260709 13:19:51.130053 17126 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:51.132097 17386 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:51.133566 17407 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.136060 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3832) took 1376 ms. Trace:
I20260709 13:19:51.136593 10130 rpcz_store.cc:276] 0709 13:19:49.759595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.540336 (+780741us) service_pool.cc:224] Handling call
0709 13:19:50.540692 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.540929 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.540953 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.541414 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.816424 (+275010us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:50.816483 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.816593 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.135708 (+319115us) raft_consensus.cc:1685] finished
0709 13:19:51.135739 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.135975 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":204329,"threads_started":1,"wal-append.queue_time_us":520264}
I20260709 13:19:51.123214 10202 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:19:51.067548 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.067950 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.142804 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:51.143258 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 75490us
I20260709 13:19:51.144227 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.069463 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:51.071583 10900 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 24.
I20260709 13:19:51.078281 10204 raft_consensus.cc:2410] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 13:19:51.080916 17363 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.081467 17373 consensus_meta.cc:287] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
I20260709 13:19:51.082105 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.087546 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:51.147039 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 77711.3us
I20260709 13:19:51.088260 17288 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:51.095139 17372 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:51.096469 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.096378 17236 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:50.935922 17228 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:50.983073 17390 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:51.051445 17094 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.052079 17384 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:51.120474 16840 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.048660 17291 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.040910 17279 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:51.129240 10891 rpcz_store.cc:276] 0709 13:19:49.444913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.445149 (+   236us) service_pool.cc:224] Handling call
0709 13:19:49.456371 (+ 11222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.456737 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:49.653739 (+197002us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.784267 (+130528us) log.cc:844] Serialized 14 byte log entry
0709 13:19:50.721560 (+937293us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b18001751e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:51.127980 (+406420us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b18001751e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:19:51.128172 (+   192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.128237 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:51.128304 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.128406 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.128570 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8846,"prepare.queue_time_us":178369,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":2770,"raft.queue_time_us":9133,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10375,"spinlock_wait_cycles":17372928,"thread_start_us":325608,"threads_started":2,"wal-append.queue_time_us":162397,"wal-append.run_cpu_time_us":2204,"wal-append.run_wall_time_us":1124264}
I20260709 13:19:51.154729 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:51.160636 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 1.10478e+06us
I20260709 13:19:51.058293 10128 rpcz_store.cc:276] 0709 13:19:49.498360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.404248 (+905888us) service_pool.cc:224] Handling call
0709 13:19:50.440496 (+ 36248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.440772 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:50.485806 (+ 45034us) log.cc:844] Serialized 14 byte log entry
0709 13:19:50.664608 (+178802us) log.cc:844] Serialized 14 byte log entry
0709 13:19:50.933748 (+269140us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:51.057268 (+123520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.057351 (+    83us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:51.057410 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.057514 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.057683 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170253,"prepare.queue_time_us":49491,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":136,"raft.queue_time_us":82971,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":5965,"spinlock_wait_cycles":28911488,"thread_start_us":521054,"threads_started":2,"wal-append.queue_time_us":207215}
I20260709 13:19:51.146629 17286 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.187844 17286 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.189143 17384 raft_consensus.cc:2764] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:51.189668 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:51.163501 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 372076us
I20260709 13:19:51.096977 17267 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:51.191036 10168 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:51.176527 17291 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.177956 10899 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:19:51.177582 17279 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.209299 17357 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:51.210484 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.194972 10199 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:51.211808 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.212431 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.213251 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.213683 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.214633 10204 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:51.214558 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.215057 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.216010 17325 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:51.216441 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:51.179783 17236 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.217181 17359 consensus_queue.cc:1059] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:51.217566 10196 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 24.
I20260709 13:19:51.217355 17236 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.219677 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:51.221469 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.222924 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:51.223408 10202 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:51.178517 17094 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.195032 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:51.235167 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.197398 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.197924 10894 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.235971 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:51.198179 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:51.238030 10892 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:51.198875 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:51.200178 17307 consensus_queue.cc:1048] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:51.200990 17289 consensus_queue.cc:1059] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:51.201789 16940 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:51.202157 17373 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:51.202571 10898 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:51.171152 17344 consensus_queue.cc:1243] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer 01bfaf1ad56a4deea886bbc64dec466c log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:51.164502 17451 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:51.203935 17267 raft_consensus.cc:993] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: : Instructing follower e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:51.147069 17407 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:51.216594 17450 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.146246 17363 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:51.192867 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.225270 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:51.275337 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.220654 17292 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:51.218675 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.287997 10196 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:19:51.227295 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:51.166193 17355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.014s	sys 0.014s
I20260709 13:19:51.228919 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:51.230506 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:51.231990 10049 raft_consensus.cc:1240] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:51.295266 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.216481 17325 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:51.246585 10056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.016s
I20260709 13:19:51.247035 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:51.309501 10056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.011s	sys 0.016s
I20260709 13:19:51.248237 17256 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.202399 17318 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.252750 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.242195 17203 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.325949 10122 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.255642 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.257380 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.254890 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.263075 10204 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.327558 10131 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:51.264889 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 868) took 1048 ms. Trace:
I20260709 13:19:51.329334 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.329214 10124 rpcz_store.cc:276] 0709 13:19:50.215998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.791415 (+575417us) service_pool.cc:224] Handling call
0709 13:19:50.791810 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.792245 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:19:50.831990 (+ 39745us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.264451 (+432461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.264521 (+    70us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:51.264573 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.264647 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.264814 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31477,"prepare.queue_time_us":69730,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":11897,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":17333,"thread_start_us":420293,"threads_started":1}
W20260709 13:19:51.280565 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.190536 17286 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.332159 10131 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.285909 10894 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:51.334015 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:51.334210 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:51.327512 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3828) took 1575 ms. Trace:
W20260709 13:19:51.286182 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93427.6us
I20260709 13:19:51.292112 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:51.335726 10130 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:51.292851 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:51.262969 17228 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.294616 10053 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:51.348827 10891 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:51.233764 17397 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.247051 17402 raft_consensus.cc:2804] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:51.363312 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.365202 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.365891 10129 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.291931 17289 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:51.337236 10893 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
I20260709 13:19:51.293566 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.321007 10200 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:51.340155 10123 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:51.380868 10128 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:51.335813 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:51.335286 10125 rpcz_store.cc:276] 0709 13:19:49.752028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.510258 (+758230us) service_pool.cc:224] Handling call
0709 13:19:50.519480 (+  9222us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:50.519975 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.520015 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:50.520717 (+   702us) log.cc:844] Serialized 52 byte log entry
0709 13:19:50.678549 (+157832us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:51.267879 (+589330us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.326903 (+ 59024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.327052 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.327147 (+    95us) raft_consensus.cc:1685] finished
0709 13:19:51.327170 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.327414 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58597,"prepare.queue_time_us":129607,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":1006,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":11352,"thread_start_us":587573,"threads_started":2,"wal-append.queue_time_us":354455}
W20260709 13:19:51.345464 10047 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 943) took 1682 ms (client timeout 2086 ms). Trace:
I20260709 13:19:51.344390 10130 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:51.381762 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49499.4us
I20260709 13:19:51.269804 17281 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:51.269968 17359 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.240712 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.352916 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:51.264209 17373 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:19:51.303507 17355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.015s	sys 0.014s
I20260709 13:19:51.291893 17346 raft_consensus.cc:1081] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Signalling peer e9a849ac7eda426abfca97cf8b15da0f to start an election
I20260709 13:19:51.269521 17451 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.306602 17288 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:51.294586 17126 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:19:51.377199 10122 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:51.335027 10051 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
W20260709 13:19:51.381983 10047 rpcz_store.cc:269] 0709 13:19:49.663021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.663212 (+   191us) service_pool.cc:224] Handling call
0709 13:19:51.345139 (+1681927us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540019a854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:51.345374 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200182144}
I20260709 13:19:51.249559 17352 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.326450 17419 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:51.383492 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.265866 17096 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 13:19:51.391733 10053 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:51.400511 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.387392 17281 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:51.400739 17373 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.416389 17281 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.417570 17281 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:51.421310 17459 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.327s	user 0.004s	sys 0.001s
I20260709 13:19:51.422964 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3827) took 1679 ms. Trace:
I20260709 13:19:51.386684 17415 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:19:51.404454 17451 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.423601 10126 rpcz_store.cc:276] 0709 13:19:49.743516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.506516 (+763000us) service_pool.cc:224] Handling call
0709 13:19:50.521410 (+ 14894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.529252 (+  7842us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:50.529288 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.546918 (+ 17630us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.672676 (+125758us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:50.672764 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.672913 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.420573 (+747660us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:51.422466 (+  1893us) raft_consensus.cc:1685] finished
0709 13:19:51.422503 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.422869 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30102,"raft.queue_time_us":10970,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":3531,"spinlock_wait_cycles":62636416,"thread_start_us":97168,"threads_started":1,"wal-append.queue_time_us":519127}
I20260709 13:19:51.427357 17236 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.428309 17236 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:51.428879 17236 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:51.403999 17359 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.429980 17236 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.343863 17256 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:51.388376 17457 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:19:51.436152 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.436523 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.432726 10130 raft_consensus.cc:1275] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:19:51.437191 10130 pending_rounds.cc:85] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 10
I20260709 13:19:51.437525 10130 pending_rounds.cc:107] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:19:51.437837 10130 raft_consensus.cc:2889] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.400003 10202 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:51.434981 10899 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:51.352053 17203 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.440482 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:51.441926 17329 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.766s	user 0.013s	sys 0.000s
I20260709 13:19:51.442698 10198 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:19:51.443426 17328 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.444206 10126 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:51.443981 17328 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.451012 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.452667 17318 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:51.453600 17307 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:51.453891 17329 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:51.454149 17307 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:51.454687 17307 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:51.454610 17329 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.457444 17329 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:51.446578 17366 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.459364 10201 raft_consensus.cc:2368] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24.
W20260709 13:19:51.460041 10203 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.014s	sys 0.019s
I20260709 13:19:51.305805 17450 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:51.460523 10203 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.014s	sys 0.019s
I20260709 13:19:51.442770 17203 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:51.389245 17402 raft_consensus.cc:697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:51.401813 17355 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:51.368989 17397 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.430317 17094 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.422794 17291 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.436827 17359 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.524161 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.525252 10126 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:51.415750 17352 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.547624 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.548750 10128 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:19:51.551213 17397 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.554330 17450 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.556119 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:51.557071 10202 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:51.559190 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.566381 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.567435 10122 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.576679 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:51.583959 10895 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.042s	sys 0.034s
W20260709 13:19:51.588649 10895 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.042s	sys 0.034s
W20260709 13:19:51.590363 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.591307 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.592506 10128 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
W20260709 13:19:51.593593 10019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.001s	sys 0.024s
W20260709 13:19:51.594142 10019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.001s	sys 0.024s
W20260709 13:19:51.594501 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 517210us
W20260709 13:19:51.563555 17237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.005s	sys 0.025s
W20260709 13:19:51.608779 10050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.015s	sys 0.012s
I20260709 13:19:51.581229 17450 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:51.627765 10168 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:51.610675 17237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.005s	sys 0.025s
I20260709 13:19:51.639555 17390 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:51.640451 17390 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:51.641682 17390 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:51.643738 17443 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:51.647001 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.630996 17402 consensus_queue.cc:237] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.648646 10128 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.656714 17291 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.663095 17463 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.664070 17463 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.643241 10122 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:51.669327 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.670872 10122 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 23: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:51.591058 17346 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:51.624045 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.705397 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:51.620313 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
W20260709 13:19:51.612169 10050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.016s	sys 0.012s
W20260709 13:19:51.681041 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.634822 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:51.709486 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74776.6us
I20260709 13:19:51.710506 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:51.713311 10048 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.711076 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.722537 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.669955 17450 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.708961 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:51.752511 10049 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:51.606731 17222 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:19:51.633737 17357 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.758163 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.759307 10129 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:51.712566 10020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.015s	sys 0.021s
I20260709 13:19:51.627905 17202 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:51.702699 17346 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.704444 17326 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:51.764396 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.751502 10128 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:51.750051 17463 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.735929 17443 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:51.755285 17450 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:51.754714 17393 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:51.767498 17346 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:51.667752 17355 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:51.813536 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 91144.5us
W20260709 13:19:51.761389 10020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.016s	sys 0.021s
W20260709 13:19:51.805342 17466 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:19:51.796144 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:51.828670 10128 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.795588 17443 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.839641 10055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46788 (request call id 2703) took 1056 ms. Trace:
I20260709 13:19:51.841917 10055 rpcz_store.cc:276] 0709 13:19:50.783582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.783798 (+   216us) service_pool.cc:224] Handling call
0709 13:19:50.784268 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.784746 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.784784 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.785376 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.291300 (+505924us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:51.291364 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.291455 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.839315 (+547860us) raft_consensus.cc:1685] finished
0709 13:19:51.839347 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.839543 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2976,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":481676,"threads_started":1,"wal-append.queue_time_us":1051053}
I20260709 13:19:51.817224 10168 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:51.805555 17393 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:51.818434 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 674283us
I20260709 13:19:51.799770 17366 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:51.850843 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.795838 17326 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:51.832441 17222 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:51.839763 17372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.028s	sys 0.010s
I20260709 13:19:51.843297 17443 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:51.863525 10170 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.002s	sys 0.018s
W20260709 13:19:51.864063 10170 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.002s	sys 0.019s
W20260709 13:19:51.864521 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 820582us
I20260709 13:19:51.866168 10129 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:51.843076 17256 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:51.846854 17393 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:51.875587 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.878286 10129 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 15.
I20260709 13:19:51.895522 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 941) took 2411 ms. Trace:
W20260709 13:19:51.896799 10897 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.005s	sys 0.036s
W20260709 13:19:51.897305 10897 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.005s	sys 0.036s
I20260709 13:19:51.896234 10056 rpcz_store.cc:276] 0709 13:19:49.463338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.463715 (+   377us) service_pool.cc:224] Handling call
0709 13:19:49.489483 (+ 25768us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.489993 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.490044 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.493042 (+  2998us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.726211 (+1233169us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b18000b8780. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:51.310063 (+583852us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:19:51.310124 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.310237 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.865174 (+554937us) raft_consensus.cc:1685] finished
0709 13:19:51.865207 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.874713 (+  9506us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2377,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":2898,"spinlock_wait_cycles":14729088,"thread_start_us":583608,"threads_started":1,"wal-append.queue_time_us":2176717}
W20260709 13:19:51.868014 17372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.028s	sys 0.011s
I20260709 13:19:51.855314 17330 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:51.876493 10900 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.029s	sys 0.010s
W20260709 13:19:51.915942 10900 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.029s	sys 0.010s
I20260709 13:19:51.917342 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:51.844833 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2533) took 1052 ms. Trace:
W20260709 13:19:51.906672 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55998.6us
I20260709 13:19:51.911675 10201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:36318
I20260709 13:19:51.919867 10201 raft_consensus.cc:606] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Received request to transfer leadership to TS e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.920469 10201 raft_consensus.cc:614] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:51.922463 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:51.911855 10128 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 14.
I20260709 13:19:51.929919 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:51.930703 10049 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:51.933360 10048 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:51.934602 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:51.936390 10048 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:51.874574 17222 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:51.939471 10053 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:51.950616 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:51.951241 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.951469 10056 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:51.951895 10129 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:51.876956 17352 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:51.954322 10864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.014s	sys 0.007s
W20260709 13:19:51.954828 10864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.015s	sys 0.008s
I20260709 13:19:51.919332 10203 rpcz_store.cc:276] 0709 13:19:50.792193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.792511 (+   318us) service_pool.cc:224] Handling call
0709 13:19:50.792887 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.793150 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:51.481323 (+688173us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.844052 (+362729us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.844328 (+   276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.844393 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:51.844473 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.844580 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.844762 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4747,"prepare.queue_time_us":814156,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":30961,"raft.queue_time_us":1020,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":9597,"spinlock_wait_cycles":48251136,"thread_start_us":1029306,"threads_started":2}
W20260709 13:19:51.960183 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.955307 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 648288us
W20260709 13:19:51.965675 10125 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.016s	sys 0.040s
W20260709 13:19:51.967085 10125 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.016s	sys 0.040s
I20260709 13:19:51.929648 17372 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:51.953022 17279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.027s	sys 0.011s
I20260709 13:19:51.965091 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:51.957492 17464 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.033s	sys 0.021s
I20260709 13:19:51.939457 17330 raft_consensus.cc:697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:51.980661 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58301.2us
W20260709 13:19:51.961541 17225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.004s	sys 0.018s
W20260709 13:19:51.968760 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.009s	sys 0.038s
I20260709 13:19:51.860441 17202 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:51.986547 17358 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.012s	sys 0.033s
W20260709 13:19:51.973428 17279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.028s	sys 0.011s
W20260709 13:19:51.993070 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.009s	sys 0.038s
W20260709 13:19:51.977456 17464 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.033s	sys 0.022s
I20260709 13:19:51.990075 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:51.976284 10049 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
W20260709 13:19:52.017988 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 682203us
I20260709 13:19:51.980087 17281 raft_consensus.cc:2764] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:19:51.995862 10048 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:52.007321 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:52.013842 10204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.018s	sys 0.016s
I20260709 13:19:52.019304 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:51.987381 17472 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 13:19:52.022462 10204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.019s	sys 0.016s
W20260709 13:19:52.012909 17225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.004s	sys 0.019s
W20260709 13:19:51.982694 16840 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.013s	sys 0.023s
I20260709 13:19:52.024699 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:52.019431 17415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.007s	sys 0.010s
W20260709 13:19:52.034044 10866 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.018s	sys 0.014s
I20260709 13:19:52.034437 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:52.034529 10866 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.018s	sys 0.014s
I20260709 13:19:52.035408 10893 raft_consensus.cc:3060] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:52.035418 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:52.036281 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.030292 10891 raft_consensus.cc:1240] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 13:19:52.013608 17358 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.013s	sys 0.033s
I20260709 13:19:51.987174 17480 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:51.982959 17280 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:19:52.021152 17289 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.011s	sys 0.007s
W20260709 13:19:52.043442 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 365020us
W20260709 13:19:52.045639 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:52.033515 16840 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.014s	sys 0.023s
W20260709 13:19:52.038636 17415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.007s	sys 0.011s
W20260709 13:19:52.012315 17267 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.012s	sys 0.021s
I20260709 13:19:52.034118 17279 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:52.042260 17479 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:52.039657 17358 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:52.043141 17289 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.012s	sys 0.008s
I20260709 13:19:52.071627 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.034396 17464 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:52.090018 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.102267 10893 raft_consensus.cc:2468] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
W20260709 13:19:52.076126 17267 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.012s	sys 0.022s
I20260709 13:19:52.054507 17335 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:52.143796 17237 raft_consensus.cc:3055] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:52.097895 16840 raft_consensus.cc:2764] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:19:52.098595 17326 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.023434 17222 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.058477 17330 consensus_queue.cc:237] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.140125 17479 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:52.084007 17481 consensus_queue.cc:1059] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:19:52.142163 17279 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:19:52.156780 17237 raft_consensus.cc:740] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:52.177311 17475 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.256s	user 0.000s	sys 0.002s
I20260709 13:19:52.130630 17464 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:52.204658 17358 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.223240 17375 raft_consensus.cc:3055] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:52.223790 17375 raft_consensus.cc:740] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:52.204284 17307 consensus_meta.cc:287] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.749s	user 0.007s	sys 0.007s
I20260709 13:19:52.225728 17307 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:52.227092 17325 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.228698 17325 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:52.229171 17407 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.936s	user 0.013s	sys 0.000s
I20260709 13:19:52.232601 17307 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.233706 17307 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:52.234200 17307 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:52.234622 17307 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:52.236887 17451 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.812s	user 0.013s	sys 0.000s
I20260709 13:19:52.237493 17451 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.238865 10127 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:52.240520 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2536) took 1447 ms. Trace:
W20260709 13:19:52.241315 17478 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:19:52.224705 17375 consensus_queue.cc:260] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:52.240649 17497 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:19:52.244149 17375 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:52.245119 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3400) took 1455 ms. Trace:
I20260709 13:19:52.245575 10050 rpcz_store.cc:276] 0709 13:19:50.789760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.869866 (+ 80106us) service_pool.cc:224] Handling call
0709 13:19:50.870228 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.016203 (+145975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.016253 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.034569 (+ 18316us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.707202 (+672633us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:51.707253 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.242162 (+534909us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400391054. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:52.242215 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.244721 (+  2506us) raft_consensus.cc:1685] finished
0709 13:19:52.244746 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.245036 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4564,"raft.queue_time_us":3740,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":12189,"spinlock_wait_cycles":326447872,"thread_start_us":615663,"threads_started":1,"wal-append.queue_time_us":876684}
I20260709 13:19:52.241015 10205 rpcz_store.cc:276] 0709 13:19:50.793283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.866739 (+ 73456us) service_pool.cc:224] Handling call
0709 13:19:50.901590 (+ 34851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.905998 (+  4408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.240263 (+1334265us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b54001d0450. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:52.240445 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79238400}
W20260709 13:19:52.196866 17228 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.007s
I20260709 13:19:52.260735 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:52.262821 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 281028us
W20260709 13:19:52.271575 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 280716us
W20260709 13:19:52.277469 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 258296us
I20260709 13:19:52.278777 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.280157 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.281678 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:52.287796 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3409) took 1049 ms. Trace:
W20260709 13:19:52.330377 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.348354 17256 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.349150 17256 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:52.350271 17393 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.302933 17477 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:52.360364 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 270436us
I20260709 13:19:52.357738 10169 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:52.374396 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.376741 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:52.377851 10897 raft_consensus.cc:2376] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20. Re-sending same reply.
I20260709 13:19:52.352948 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.354594 17228 consensus_queue.cc:1059] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:19:52.332412 17390 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.308202 17493 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:52.336149 17415 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.353011 17326 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.386557 17482 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 13:19:52.387789 17499 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:19:52.388829 17489 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:19:52.428893 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1121) took 1257 ms. Trace:
I20260709 13:19:52.403730 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.355571 17492 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.350329 17373 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.934s	user 0.013s	sys 0.001s
W20260709 13:19:52.414664 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66900.1us
I20260709 13:19:52.381649 17402 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:52.457239 17119 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.466233 17119 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.466439 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:52.337886 17237 consensus_queue.cc:260] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:52.393651 17346 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.001s
I20260709 13:19:52.438750 10900 rpcz_store.cc:276] 0709 13:19:51.171461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.172161 (+   700us) service_pool.cc:224] Handling call
0709 13:19:51.172541 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.172762 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:51.916713 (+743951us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.428469 (+511756us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.428536 (+    67us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:52.428584 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.428662 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.428819 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":726699,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":90348,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":1164907,"threads_started":3,"wal-append.queue_time_us":489151}
I20260709 13:19:52.415999 10895 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 813) took 1582 ms. Trace:
W20260709 13:19:52.472539 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 71666us
I20260709 13:19:52.479827 17480 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:52.480700 17279 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.428082 17393 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.487789 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3408) took 1250 ms. Trace:
I20260709 13:19:52.488461 16940 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:52.490345 17443 consensus_meta.cc:287] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 13:19:52.490798 10131 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:52.417351 17326 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.498302 10051 rpcz_store.cc:276] 0709 13:19:51.237796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.409164 (+171368us) service_pool.cc:224] Handling call
0709 13:19:51.409740 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.410230 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.410274 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.410968 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.023135 (+612167us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b1800450040. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:52.284530 (+261395us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:52.284574 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.284646 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.487240 (+202594us) raft_consensus.cc:1685] finished
0709 13:19:52.487280 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.487663 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":119,"spinlock_wait_cycles":7673088,"thread_start_us":261133,"threads_started":1,"wal-append.queue_time_us":1058694}
I20260709 13:19:52.508994 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3371) took 1247 ms. Trace:
I20260709 13:19:52.516892 10131 rpcz_store.cc:276] 0709 13:19:51.261275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.333149 (+ 71874us) service_pool.cc:224] Handling call
0709 13:19:51.336098 (+  2949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.490503 (+1154405us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001700a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa2780061c0 00007fa278007231 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c
0709 13:19:52.508649 (+ 18146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.508926 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253579008}
I20260709 13:19:52.407567 17126 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:19:52.410825 17390 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.414484 17415 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:52.494681 17363 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.169s	user 0.017s	sys 0.000s
I20260709 13:19:52.476548 17373 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:52.491022 17237 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:52.497220 17450 consensus_meta.cc:287] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.003s
I20260709 13:19:52.414105 17493 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.499782 17402 raft_consensus.cc:2804] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:52.519955 17289 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:52.491451 17346 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.523802 17256 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:52.507436 17279 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.493312 10895 rpcz_store.cc:276] 0709 13:19:50.834069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.834691 (+   622us) service_pool.cc:224] Handling call
0709 13:19:50.835005 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.835273 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:51.604722 (+769449us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.103533 (+498811us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.412862 (+309329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.415509 (+  2647us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:52.415573 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.415695 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.415906 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28520,"prepare.queue_time_us":1024986,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":102,"raft.queue_time_us":135155,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"thread_start_us":1500537,"threads_started":3,"wal-append.queue_time_us":571625}
I20260709 13:19:52.530716 17405 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:52.539619 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73645.9us
W20260709 13:19:52.547042 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.524499 17393 raft_consensus.cc:3055] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:52.498140 17492 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.529577 17451 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.520462 17443 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.535136 17511 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.539029 17488 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:52.554314 17289 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:52.559839 17256 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:52.528362 17516 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:19:52.540465 17450 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.572945 17119 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.607705 17450 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.554487 17402 raft_consensus.cc:697] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:52.609870 17402 consensus_queue.cc:237] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:52.532668 17363 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.612624 17346 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.614298 17363 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.557513 17393 raft_consensus.cc:740] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:52.577251 17517 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:52.604947 17510 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:19:52.605253 17502 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:19:52.616056 17393 consensus_queue.cc:260] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.618041 17256 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.620472 17415 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.621201 17256 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.622699 17493 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.623936 17443 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.652434 17279 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.705267 17326 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.574790 17096 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:52.632257 17390 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.579757 17511 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.812269 17511 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:52.815160 17492 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:52.815588 17373 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:52.803184 17289 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:52.831266 17307 consensus_meta.cc:287] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.596s	user 0.017s	sys 0.000s
W20260709 13:19:52.874178 17375 consensus_meta.cc:287] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.629s	user 0.018s	sys 0.002s
I20260709 13:19:52.889429 17384 raft_consensus.cc:3055] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:52.890316 17384 raft_consensus.cc:740] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:52.935338 17384 consensus_queue.cc:260] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.937492 17384 raft_consensus.cc:3060] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:53.077116 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 584118us
W20260709 13:19:53.078896 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.082003 10094 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.004s	sys 0.018s
W20260709 13:19:53.082944 10094 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.004s	sys 0.018s
W20260709 13:19:53.085421 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.091557 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.005s	sys 0.004s
W20260709 13:19:53.092401 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.005s	sys 0.004s
I20260709 13:19:53.097214 10096 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:52.557780 10126 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:53.104348 10124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6"
dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
 from {username='slave'} at 127.0.0.1:45986
I20260709 13:19:53.105702 10125 raft_consensus.cc:1240] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:53.109061 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.110711 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:53.120538 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:53.123497 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3853) took 1491 ms. Trace:
I20260709 13:19:53.124308 10131 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:19:53.124179 10128 rpcz_store.cc:276] 0709 13:19:51.631725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.929523 (+297798us) service_pool.cc:224] Handling call
0709 13:19:51.929939 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:51.947036 (+ 17097us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:51.947078 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:51.962526 (+ 15448us) log.cc:844] Serialized 52 byte log entry
0709 13:19:52.309946 (+347420us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:52.310044 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.310174 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.122682 (+812508us) raft_consensus.cc:1685] finished
0709 13:19:53.122734 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.123225 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9267,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":10621,"thread_start_us":290619,"threads_started":1,"wal-append.queue_time_us":528240}
I20260709 13:19:53.127480 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:53.138341 10020 leader_election.cc:400] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24.
W20260709 13:19:52.534873 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 226480us
W20260709 13:19:53.143746 10169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.032s	sys 0.017s
W20260709 13:19:53.147905 10169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.032s	sys 0.017s
I20260709 13:19:53.149093 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:53.144405 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.146381 10196 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:19:53.146561 10197 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:19:53.154338 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.161021 10170 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:53.147394 10020 leader_election.cc:403] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 13:19:53.172492 10203 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.821s	user 0.001s	sys 0.018s
W20260709 13:19:53.173319 10203 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.001s	sys 0.018s
I20260709 13:19:53.256038 17326 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:53.257114 17361 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.257877 17126 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.257644 17326 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.258213 17361 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:53.260033 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 99337.5us
W20260709 13:19:53.264531 16940 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.001s	sys 0.005s
I20260709 13:19:53.264251 17222 consensus_queue.cc:1048] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:53.265190 17318 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:53.267820 17325 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:53.269761 17355 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.265208 16940 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.001s	sys 0.005s
I20260709 13:19:53.289947 17292 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:53.290422 17256 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:53.291278 10170 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:53.291447 17415 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:53.292043 17346 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:53.293018 17493 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:53.294214 17492 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:53.296664 17326 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:53.297305 10863 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.016s	sys 0.021s
I20260709 13:19:53.297925 17450 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:53.258850 17126 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:52.709301 17096 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:53.351763 10863 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.017s	sys 0.021s
I20260709 13:19:53.339661 17119 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:53.340988 17358 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:53.341765 17292 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:53.342236 17390 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:53.291491 17363 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.343446 17256 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.347563 17415 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.371423 17096 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.374619 17373 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.266122 17307 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.375627 17373 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.376308 17096 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:53.378273 17307 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:53.365789 17358 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.379659 17358 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:53.385492 10128 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:53.379761 17280 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.388453 17415 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.389130 17450 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.390848 17292 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.327453 17515 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 23. Current term is 24.
W20260709 13:19:53.395062 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 257440us
I20260709 13:19:53.386662 17280 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.396121 17363 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:53.413322 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 122427us
I20260709 13:19:53.344394 17202 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:53.435845 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 946) took 1175 ms. Trace:
I20260709 13:19:53.436671 10203 rpcz_store.cc:276] 0709 13:19:52.259909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.260261 (+   352us) service_pool.cc:224] Handling call
0709 13:19:52.284422 (+ 24161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.284663 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:53.174243 (+889580us) log.cc:844] Serialized 14 byte log entry
0709 13:19:53.434969 (+260726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.435092 (+   123us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:53.435271 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.435439 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.435712 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":164685,"prepare.queue_time_us":70268,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":150118,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":318,"thread_start_us":922613,"threads_started":2}
W20260709 13:19:53.456147 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.461563 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.464848 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:53.464960 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:53.464351 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:53.476266 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:53.477567 10205 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
W20260709 13:19:53.479651 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.481622 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:53.491987 17222 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:53.492811 17222 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:53.493399 17222 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:53.499614 17346 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:53.500592 17346 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.501171 17346 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:53.505517 17126 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:53.413458 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 858435us
I20260709 13:19:53.508365 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:53.509872 17287 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:53.514027 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.516678 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.517694 10128 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:53.527258 17126 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:53.529723 17363 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.471004 10895 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:53.543478 17358 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.546674 17358 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:53.534251 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82999.3us
I20260709 13:19:53.574169 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3767) took 2553 ms. Trace:
I20260709 13:19:53.550595 17287 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:53.533545 10900 raft_consensus.cc:1813] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:53.575774 10900 raft_consensus.cc:2410] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:19:53.553741 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3382) took 2024 ms. Trace:
I20260709 13:19:53.577400 10126 rpcz_store.cc:276] 0709 13:19:51.528836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.529170 (+   334us) service_pool.cc:224] Handling call
0709 13:19:51.576366 (+ 47196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.104316 (+1527950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.553106 (+448790us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54001261d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:53.553508 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2256447744}
I20260709 13:19:53.579993 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.581949 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.583061 10126 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:53.584659 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3378) took 2184 ms. Trace:
I20260709 13:19:53.585287 10130 rpcz_store.cc:276] 0709 13:19:51.403088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.428009 (+ 24921us) service_pool.cc:224] Handling call
0709 13:19:51.428426 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.438435 (+ 10009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.583949 (+2145514us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b54001c7d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:53.584422 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8771968}
I20260709 13:19:53.589020 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3375) took 2252 ms. Trace:
I20260709 13:19:53.589643 10123 rpcz_store.cc:276] 0709 13:19:51.336573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.381503 (+ 44930us) service_pool.cc:224] Handling call
0709 13:19:51.386803 (+  5300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.584243 (+197440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.584286 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.584340 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:51.584373 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.588355 (+2003982us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:53.588467 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.588791 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453469952}
I20260709 13:19:53.289009 17355 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.591954 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.592916 10123 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:53.555565 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2548) took 2317 ms. Trace:
I20260709 13:19:53.595256 10196 rpcz_store.cc:276] 0709 13:19:51.238233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.317456 (+ 79223us) service_pool.cc:224] Handling call
0709 13:19:51.317899 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.153353 (+1835454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.554959 (+401606us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540035acd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:53.555446 (+   487us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4202528640}
I20260709 13:19:53.596076 17329 raft_consensus.cc:3055] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:53.596904 17329 raft_consensus.cc:740] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:53.576453 17288 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:53.598654 17288 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:19:53.600685 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.378913 17373 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:53.574877 10197 rpcz_store.cc:276] 0709 13:19:51.020410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.025597 (+  5187us) service_pool.cc:224] Handling call
0709 13:19:51.043911 (+ 18314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.154238 (+2110327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.573676 (+419438us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54002189d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:53.574067 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4832666368}
I20260709 13:19:53.601737 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.614758 17280 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:53.638844 10130 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:19:53.639034 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.640432 10129 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:53.640944 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.641809 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.642712 10129 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:53.643292 17361 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:53.643371 17225 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:53.648571 17450 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.626138 17287 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.688150 10198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2543) took 2474 ms (client timeout 1957 ms). Trace:
I20260709 13:19:53.688239  9948 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:53.689018 10198 rpcz_store.cc:269] 0709 13:19:51.213124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.214261 (+  1137us) service_pool.cc:224] Handling call
0709 13:19:53.687524 (+2473263us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400217fd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:53.687830 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522387840}
I20260709 13:19:53.691020  9948 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 19 to 22, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 22 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:53.635696 17256 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.689047 17222 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.732924 10126 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:53.704726 17355 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.745337 10123 raft_consensus.cc:2468] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:53.759294 10127 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3367) took 2499 ms (client timeout 1671 ms). Trace:
W20260709 13:19:53.759963 10127 rpcz_store.cc:269] 0709 13:19:51.259770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.259924 (+   154us) service_pool.cc:224] Handling call
0709 13:19:53.758734 (+2498810us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b54000a00d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:53.759000 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168362752}
W20260709 13:19:53.761053 17237 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.221s	user 0.018s	sys 0.004s
I20260709 13:19:53.764235 10129 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:53.765493 17539 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.722293 17287 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.774420 17539 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:53.784709 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49089.3us
I20260709 13:19:53.793872 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:53.794708 10126 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:53.795245 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.800266 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:53.801672 10130 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:53.803978 10130 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: [18.14-19.15]
I20260709 13:19:53.781203 17237 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.809458 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:53.807636 17483 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.819365 17292 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.834872 17286 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.822667 17464 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.823603 17415 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.825590 17287 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.825883 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:53.848004 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.826377 17538 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.535s	user 0.003s	sys 0.000s
I20260709 13:19:53.830976 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:53.831629 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.832314 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.832814 17483 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.833583 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:53.834537 17292 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.818775 10126 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:53.832165 10054 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:53.790503 17355 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.842531 17545 log.cc:927] Time spent T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:19:53.843025 17352 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.791497 17256 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.831664 17222 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.837574 17286 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:53.865939 17384 consensus_meta.cc:287] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.901s	user 0.015s	sys 0.011s
I20260709 13:19:53.849534 17307 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.879381 10863 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:53.879734 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 86945us
I20260709 13:19:53.878809 10130 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:53.885159 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:53.900450 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.900795 10048 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:53.902305 17096 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.004s
I20260709 13:19:53.905867 17307 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.907346 10049 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:53.908573 10126 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:53.885257 17355 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.840457 17237 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.911367 17256 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.912130 17355 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.914513 17256 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.915020 17515 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:53.916071 17515 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.916843 17372 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:53.917321 17280 raft_consensus.cc:2804] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:53.918087 17280 raft_consensus.cc:697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:53.918613 17515 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.918893 10051 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:53.919692 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:53.920914 17203 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:53.921345 17203 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:53.922685 17203 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.924127 17203 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:53.924582 17237 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.925180 17483 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.878535 17287 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:19:53.877202 17286 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.928006 17376 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:53.928880 17376 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.929561 17222 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:53.930487 17222 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.879247 17318 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:53.894986 17358 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.931646 17373 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:53.932534 17358 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.933094 17222 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:53.883028 10050 raft_consensus.cc:2376] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25. Re-sending same reply.
I20260709 13:19:53.933322 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.931741 17318 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.908739 17307 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.932698 17373 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.917714 17372 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.936841 10124 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:53.914572 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:53.935058 17539 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:53.932752 10170 connection.cc:256] RPC call timeout handler was delayed by -1.2576228890000039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 13:19:53.948426 10170 connection.cc:256] RPC call timeout handler was delayed by -1.1280228890000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 13:19:53.948902 10170 connection.cc:256] RPC call timeout handler was delayed by -1.1190228890000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:19:53.945555 10124 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:53.949874 17344 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.950557 17344 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.950951 17292 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.952868 10052 raft_consensus.cc:1275] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:19:53.953457 17373 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.954416 17344 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:53.934890 17358 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:53.959345 10128 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:53.959131 10048 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:53.907418 17477 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.956995 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.963863 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.915227 17415 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:53.972486 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:53.934805 17280 consensus_queue.cc:237] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:53.881999 17352 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:53.947194 17492 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.004s	sys 0.023s
I20260709 13:19:53.979172 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:53.964156 17344 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.999082 10051 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:54.003440 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:53.970520 17288 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.005540 17376 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.005177 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.006116 17366 consensus_queue.cc:1048] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:19:54.006280 17375 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:54.013296 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 134007us
I20260709 13:19:53.975185 17477 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.020004 10054 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:54.026363 10127 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
W20260709 13:19:54.031486 10899 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 550) took 2725 ms (client timeout 1957 ms). Trace:
I20260709 13:19:54.038816 10055 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
W20260709 13:19:54.033354 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.039430 17288 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.039808 17372 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.040202 10056 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:54.040995 17352 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:52.289052 10047 rpcz_store.cc:276] 0709 13:19:51.238064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.411818 (+173754us) service_pool.cc:224] Handling call
0709 13:19:51.412256 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.412699 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.412740 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.462165 (+ 49425us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.989101 (+526936us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:52.286683 (+297582us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.287245 (+   562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.287371 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.287446 (+    75us) raft_consensus.cc:1685] finished
0709 13:19:52.287467 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.287686 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3842,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"thread_start_us":725273,"threads_started":2,"wal-append.queue_time_us":676312}
I20260709 13:19:54.042745 10168 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.044010 10168 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.045162 10168 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:54.046280 17554 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:54.046507 17325 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.048076 17126 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:54.048342 17407 raft_consensus.cc:2804] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:54.048727 17373 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:54.048974 17126 raft_consensus.cc:697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.049223 17407 raft_consensus.cc:697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.049484 17373 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.048352 17325 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.052265 17325 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:54.056308 17415 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.023862 17318 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:54.017166 10170 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.559s (SENT) [suppressed 19 similar messages]
W20260709 13:19:54.062981 10020 connection.cc:256] RPC call timeout handler was delayed by -1.2462754110001697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:19:54.038515 10899 rpcz_store.cc:269] 0709 13:19:51.306266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.333854 (+ 27588us) service_pool.cc:224] Handling call
0709 13:19:54.028195 (+2694341us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b54001e5854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:54.031206 (+  3011us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164646272}
W20260709 13:19:54.063465 10020 connection.cc:256] RPC call timeout handler was delayed by -1.076175411000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
I20260709 13:19:54.064153 10124 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.037604 17375 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.070111 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:54.038015 17492 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.005s	sys 0.024s
W20260709 13:19:54.073311 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.074527 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:54.088140 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.089502 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.056686 17286 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.082914 17375 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:54.050743 17407 consensus_queue.cc:237] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.104192 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.050632 17126 consensus_queue.cc:237] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.120736 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.109164 17287 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:54.118847 10051 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:54.050797 17373 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.112093 17318 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.132122 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.134162 10126 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:54.086615 17344 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.146831 10127 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:54.148283 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:54.153230 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49138.3us
I20260709 13:19:54.154618 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.129659 17287 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.151155 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.143246 17372 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.164803 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.165745 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.143445 17288 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.166416 17488 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 13:19:54.166834 17288 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.170331 17288 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:54.166076 17397 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.119s	user 0.010s	sys 0.086s
I20260709 13:19:54.167438 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.166095 10126 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:54.140882 17366 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:54.138751 17203 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:54.176818 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:54.158120 17287 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:54.180750 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 155968us
W20260709 13:19:54.173043 17397 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.233s	user 0.011s	sys 0.087s
I20260709 13:19:54.170871 17539 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:54.196385 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 180263us
I20260709 13:19:54.198932 10051 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 14.
I20260709 13:19:54.196933 17539 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:54.224081 17366 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:54.228979 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.228829 17419 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:54.229751 17419 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:19:54.230410 10019 connection.cc:256] RPC call timeout handler was delayed by -1.0329746649999834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 13:19:54.102488 17222 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 8 to 13, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:19:54.229751 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.175601 17318 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.234629 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:54.235345 10122 raft_consensus.cc:1240] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:19:54.224395 17280 consensus_queue.cc:1059] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:19:54.238301 17286 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.239197 17344 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.234634 17477 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:54.239926 17346 consensus_meta.cc:287] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.738s	user 0.013s	sys 0.007s
I20260709 13:19:54.285501 17346 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.287807 17487 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.288566 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.288539 17487 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.289430 17517 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.290335 17517 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.292820 17487 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.293180 17517 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.287237 17375 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.288610 17346 leader_election.cc:290] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:54.261602 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50995.7us
I20260709 13:19:54.259013 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.296556 10127 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:54.267596 10051 raft_consensus.cc:1275] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:54.270035 10126 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:19:54.273509 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 96999.7us
I20260709 13:19:54.274112 10050 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 13:19:54.276731 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45420.5us
I20260709 13:19:54.301290 10050 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:54.283833 17539 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.243553 17366 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:54.300889 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.277518 17203 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:54.207043 17372 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.302716 17539 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.311043 10050 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:54.317682 17375 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.318496 17419 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.321487 17419 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:54.322170 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.323025 10131 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:54.323282 10048 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:54.324189 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.324884 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:19:54.325906 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.326560 10131 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.328075 17407 consensus_queue.cc:1059] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:54.328428 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.329496 17407 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:54.309409 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.359261 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.360069 10891 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:19:54.358738 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:54.358801 10899 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:19:54.361995 10893 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:19:54.342505 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:54.365449 10048 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:19:54.330184 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.318748 17372 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:54.339711 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.372251 10050 raft_consensus.cc:1240] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 20. Current term is 21. Ops: [18.14-20.15]
W20260709 13:19:54.372632 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.373131 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.375423 10126 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:54.376583 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.376652 10050 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:54.377394 10126 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
W20260709 13:19:54.377638 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.290814 17325 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.332094 17228 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:54.331657 17376 raft_consensus.cc:3055] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:54.384634 17419 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:54.384940 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.385782 10892 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:54.395999 10196 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:19:54.333479 17318 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:54.406991 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 106298us
W20260709 13:19:54.384004 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53897.6us
I20260709 13:19:54.378572 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.419759 10126 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:19:54.384873 17344 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:54.402047 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:54.421205 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.423974 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.424840 10205 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
W20260709 13:19:54.425499 10169 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Timed out: RequestConsensusVote RPC to 127.8.183.68:44775 timed out after 2.046s (SENT)
I20260709 13:19:54.330646 10198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292"
dest_uuid: "d51ad94710c348889d417f7f00032948"
mode: GRACEFUL
new_leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
 from {username='slave'} at 127.0.0.1:36318
W20260709 13:19:54.429715 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49550.4us
I20260709 13:19:54.340206 17330 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:19:54.427321 17344 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.432767 17419 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:54.436297 17344 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.437207 17419 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.438535 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.440606 17477 consensus_queue.cc:1059] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:19:54.443550 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.445401 10894 raft_consensus.cc:1275] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:19:54.447389 17488 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:54.448020 17488 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:54.400089 17517 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:54.449244 17488 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.450188 17517 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:54.453373 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51446.3us
I20260709 13:19:54.456442 10131 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:54.383567 17228 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.450239  9948 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:54.458405 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.461367 10122 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.453600 17373 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:19:54.450871 17488 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.391588 17376 raft_consensus.cc:740] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.478621 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.479547 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.484546 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:54.460134 17517 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.485316 10205 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:54.492156 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.499552 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.502557 10131 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
W20260709 13:19:54.509037 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 83643.4us
I20260709 13:19:54.509940 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:54.510901 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:54.487392 17375 consensus_queue.cc:1059] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:19:54.511806 17407 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.512353 17407 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:54.512799 17407 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.513376 17346 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:19:54.513909 17346 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.514328 17346 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:54.516320 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.516616 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.517194 10131 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:19:54.518471 17517 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.519443 17517 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:54.520116 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.520118 17517 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.521520 17289 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.522488 17289 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.523047 17289 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:54.525617 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.527807 17376 consensus_queue.cc:260] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.529870 17376 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:54.518631 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.531756 10131 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:54.533460 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.534579 17537 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.535167 17537 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.535590 17537 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:54.536804 17344 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.539786 17361 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:54.541620 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:54.543483 17126 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:54.543960 17126 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.544368 17126 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:54.543987 10205 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:54.554378 10198 raft_consensus.cc:606] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Received request to transfer leadership to TS 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:54.555164 10198 raft_consensus.cc:614] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:54.463649 17228 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:54.559710 17344 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.563802 17477 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:54.556794 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78192.7us
I20260709 13:19:54.570596 10126 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:54.573536 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.571481 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 72092us
I20260709 13:19:54.572816 10127 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
I20260709 13:19:54.576076 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.580336 10126 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 8 to 13, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:54.576824 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:54.585464 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 110071us
I20260709 13:19:54.497593 17373 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:19:54.546334 17407 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.576951 17344 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:54.597841 17225 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:54.630105 17225 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:54.630648 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60387.8us
W20260709 13:19:54.600019 10095 connection.cc:256] RPC call timeout handler was delayed by -1.320428118000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:19:54.631635 10095 connection.cc:256] RPC call timeout handler was delayed by -1.2511281179999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 13:19:54.631971 10095 connection.cc:256] RPC call timeout handler was delayed by -1.1989281180001399s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 13:19:54.600207 17256 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.601559 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3393) took 1498 ms. Trace:
I20260709 13:19:54.576184 17366 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:19:54.607767 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.611409 10200 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:54.611794 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.613559 10202 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:54.617211 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:54.618448 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.619668 17329 consensus_queue.cc:260] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.597878 17375 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:19:54.600243 10893 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:19:54.651602 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.598889 10168 leader_election.cc:304] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:54.653508 10199 raft_consensus.cc:2368] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20.
I20260709 13:19:54.576377 17318 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:54.633292 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.589591 17477 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.643554 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:54.630661 17225 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:54.628321 17344 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.661527 17393 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:54.664300 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.666433 17256 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.667740 17267 raft_consensus.cc:2804] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:54.670047 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:54.689776 10199 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:19:54.692575 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63906us
I20260709 13:19:54.652695 10122 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:54.707105 10863 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:54.707912 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75239.5us
I20260709 13:19:54.711341 10127 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:19:54.713006 17372 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:54.713563 17372 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.713977 17372 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:54.710372 17267 raft_consensus.cc:697] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.720721 17267 consensus_queue.cc:237] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.689209 17344 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:54.732136 17407 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.736266 17516 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:54.737013 17516 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 15 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:54.737413 17256 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:54.738131 17516 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.741597 17490 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:19:54.743029 17490 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:54.743515 17490 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:54.672263 10900 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:54.672750 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:54.656579 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57779.5us
I20260709 13:19:54.628961 17419 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.679468 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.680210 17477 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.680613 17329 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:54.680864 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:54.681203 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48566.8us
I20260709 13:19:54.681720 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.682017 17412 raft_consensus.cc:2764] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:19:54.682432 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.637658 10125 rpcz_store.cc:276] 0709 13:19:53.103436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.114337 (+ 10901us) service_pool.cc:224] Handling call
0709 13:19:53.114869 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.320357 (+205488us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:53.717237 (+396880us) log.cc:844] Serialized 14 byte log entry
0709 13:19:54.600936 (+883699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:54.601002 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:54.601056 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.601295 (+   239us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.601482 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567807,"prepare.queue_time_us":132602,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":568361,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":6836608,"thread_start_us":655341,"threads_started":2,"wal-append.queue_time_us":390971}
I20260709 13:19:54.673430 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:54.683751 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.756978 10168 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:54.758277 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.746459 17490 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.777885 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.802470 10199 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:54.757390 17289 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.803706 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.804508 10199 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:54.805799 17289 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:54.779049 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70075.9us
I20260709 13:19:54.777114  9948 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 21, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:19:54.786069 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.807992 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:54.786787 17407 raft_consensus.cc:2804] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:54.810174 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.787640 10122 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:54.790418 17291 raft_consensus.cc:3055] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:54.795367 10019 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:54.797017 17225 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.798237 10205 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:54.812960 10895 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:54.805611 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.815713 10124 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:54.820199 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 878) took 1035 ms. Trace:
I20260709 13:19:54.820839 10123 rpcz_store.cc:276] 0709 13:19:53.784361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.683426 (+899065us) service_pool.cc:224] Handling call
0709 13:19:54.820099 (+136673us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:54.688252 17287 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.838663 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:54.732671 17203 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:19:54.841123 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.841342 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:54.842993 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.845511 10127 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:19:54.843014 10126 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:54.810788 17330 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:54.848914 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:45986 (request call id 3389) took 1726 ms. Trace:
I20260709 13:19:54.849189 17412 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:54.850370 10124 rpcz_store.cc:276] 0709 13:19:53.089853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.103966 (+ 14113us) service_pool.cc:224] Handling call
0709 13:19:54.789706 (+1685740us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b540036b354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277ffba3d 00007fa27cfcbcd0 00007fa276550254 00007fa2765501e2 00007fa276550111 00007fa27655008c 00007fa27654f342 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:54.816247 (+ 26541us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911020288}
I20260709 13:19:54.848898 17419 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.856945 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.816720 17291 raft_consensus.cc:740] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:54.862227 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:54.860029 10050 raft_consensus.cc:1240] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:54.850780 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:54.863027 10899 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:54.743701 17126 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:54.863755 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129124us
I20260709 13:19:54.862325 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.777114 17346 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.848256 17373 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:54.895674 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.895749 10202 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:54.850003 17358 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.902284 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.866437 17477 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.907048 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:54.905548 10198 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:54.910570 10196 raft_consensus.cc:2393] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 25: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:54.910852 10129 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:54.866271 17397 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:54.873263 17289 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.909204 10205 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:54.911571 10203 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:19:54.915876 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 876) took 1132 ms. Trace:
I20260709 13:19:54.916641 10129 rpcz_store.cc:276] 0709 13:19:53.783380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.891567 (+1108187us) service_pool.cc:224] Handling call
0709 13:19:54.915744 (+ 24177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:54.928210 10127 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:54.900907 17492 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:54.899194 17373 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.935150 10200 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:19:54.938148 10199 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:54.937951 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:54.939406 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130186us
I20260709 13:19:54.892745 17537 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.941280 10205 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:54.942343 10203 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:54.918071 17203 raft_consensus.cc:2804] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:54.947474 10094 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.947331 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:54.952157 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78114.4us
I20260709 13:19:54.937194 17373 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:54.954825 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 98050.1us
I20260709 13:19:54.934665 17358 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.930308 17477 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:54.968012 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.951687 17558 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:54.974771 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.975535 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 323521us
W20260709 13:19:54.977229 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 2.086s (SENT)
W20260709 13:19:54.977483 17552 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 1.004s	user 0.002s	sys 0.002s
I20260709 13:19:54.949131 17291 consensus_queue.cc:260] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.979614 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:54.982765 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.943753 17289 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:54.962515 17203 raft_consensus.cc:697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:54.987648 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:54.979025 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.938745 17397 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.003903 10198 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:55.004787 10203 raft_consensus.cc:1240] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:55.008520 10053 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:55.010612 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:54.978931 17286 raft_consensus.cc:3055] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:55.007499 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:54.999294 17407 raft_consensus.cc:697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:54.972398 17225 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:54.988981 17477 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.015488 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:54.997186 17361 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:55.019958 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.013607 10052 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.020946 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:54.997560 17318 consensus_queue.cc:1059] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:19:55.035842 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.047052 10048 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:55.016932 17286 raft_consensus.cc:740] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:55.046355 10047 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:55.032727 17330 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:55.022646 17361 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:55.053054 17361 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:55.036865 17289 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:55.029480 17291 raft_consensus.cc:3060] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:55.034019 17287 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.034229 17492 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:55.056061 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:55.035784 17318 raft_consensus.cc:3055] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:55.041579 17516 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:19:55.053499 17407 consensus_queue.cc:237] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.060133 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.071390 17318 raft_consensus.cc:740] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:55.076807 17318 consensus_queue.cc:260] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.078317 17318 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:55.079340 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.080488 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.080511 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:55.082880 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:55.084174 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:55.085482 17228 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.002s
I20260709 13:19:55.024206 17419 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.028826 17126 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:55.086102 17584 log.cc:927] Time spent T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:55.067389 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:55.048899 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.060038 17537 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:55.070025 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 259915us
W20260709 13:19:55.071398 10863 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.087741 10051 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 945) took 1016 ms. Trace:
I20260709 13:19:55.088167 10051 rpcz_store.cc:276] 0709 13:19:54.071513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.340738 (+269225us) service_pool.cc:224] Handling call
0709 13:19:54.341121 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.341616 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.341650 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.371592 (+ 29942us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.665597 (+294005us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b18000b8780. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:19:54.897113 (+231516us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:54.897176 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.897287 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.087382 (+190095us) raft_consensus.cc:1685] finished
0709 13:19:55.087409 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.087662 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39141,"raft.queue_time_us":28709,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":11475,"spinlock_wait_cycles":21223296,"thread_start_us":230789,"threads_started":1,"wal-append.queue_time_us":606367}
I20260709 13:19:55.093575 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.092689 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.095716 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.096608 10052 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.097988 17358 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.098129 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.099165 10052 raft_consensus.cc:3060] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:55.103921 17579 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:19:55.040719 17559 consensus_queue.cc:1059] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
W20260709 13:19:55.105401 17376 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.011s
I20260709 13:19:55.106191  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:55.085784 10197 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:19:55.109166 17376 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.109992 17376 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:55.112408 17517 consensus_meta.cc:287] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.590s	user 0.013s	sys 0.000s
I20260709 13:19:55.113090 17517 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.116081 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.117072 10048 raft_consensus.cc:2393] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 13: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.117213 17376 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.118251 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:55.123741 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 126784us
I20260709 13:19:55.124974 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:55.129015 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3428) took 1206 ms. Trace:
I20260709 13:19:55.074601 17346 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.135811 17397 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.137024 17464 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:55.137549 17464 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:55.137931 17464 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:55.140026 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.140215 10197 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: []
I20260709 13:19:55.141274 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.096693 10056 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.142130 10047 raft_consensus.cc:2376] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24. Re-sending same reply.
I20260709 13:19:55.142925 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.103238 17203 consensus_queue.cc:237] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.129616 10049 rpcz_store.cc:276] 0709 13:19:53.923012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.923212 (+   200us) service_pool.cc:224] Handling call
0709 13:19:53.924018 (+   806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.129172 (+205154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.129207 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.176259 (+ 47052us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.250205 (+ 73946us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:54.266924 (+ 16719us) log.cc:844] Serialized 14 byte log entry
0709 13:19:54.267265 (+   341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.128437 (+861172us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54002e81d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:55.128501 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.128566 (+    65us) raft_consensus.cc:1685] finished
0709 13:19:55.128691 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.128890 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6243,"prepare.queue_time_us":16577,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":8802,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":21884,"spinlock_wait_cycles":388162816,"thread_start_us":32025,"threads_started":1,"wal-append.queue_time_us":221448}
I20260709 13:19:55.119062 10048 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:55.143093 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.164126 10047 raft_consensus.cc:2468] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
W20260709 13:19:55.095901 17580 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:19:55.110642 17372 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.090461 17344 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.164582 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.147657 17286 consensus_queue.cc:260] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.125000 17328 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.148041 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:55.176476 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49468.9us
I20260709 13:19:55.156633 17492 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.219921 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.220052 17492 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.157203 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.221276 10049 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.222280 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.222628 17492 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.159359 17361 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.224270 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.155465 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.225587 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.226413 10055 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.226483 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.227483 10051 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:19:55.228061 10205 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 26: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:55.228466 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.143693 10056 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:55.195323  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:55.195533  9948 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 17, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 17 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.195834 17330 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.170601 10052 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:55.215408 10199 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:19:55.219441 17286 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:55.157824 17508 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.178263 17589 log.cc:927] Time spent T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 13:19:55.229604 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.234958 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.237887 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.235153 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.173205 17517 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:55.241110 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.247988 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.225200 17361 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.249917 10049 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.251500 10048 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:19:55.248502 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.253897 10055 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:55.232501 17519 raft_consensus.cc:3055] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:55.256747 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.258397 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.260416 10056 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:55.173887 17228 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.268860 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.263684 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.274999 10056 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:55.253911 10052 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:55.263478 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.264797 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.248994 17508 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.255771 17222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33772 (request call id 9) took 5454 ms. Trace:
I20260709 13:19:55.283042 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.283964 10049 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:55.284502 10047 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:19:55.284946 10055 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.286386 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
W20260709 13:19:55.288337 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 346052us
I20260709 13:19:55.258360 17519 raft_consensus.cc:740] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:55.288574 10202 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:55.290666 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.293370 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.298013 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.300120 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.300540 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:55.303012 10056 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:19:55.304507 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.301679 10201 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 13.
I20260709 13:19:55.317005 10052 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22. Ops: []
I20260709 13:19:55.322227 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3417) took 1173 ms. Trace:
I20260709 13:19:55.323369 10128 rpcz_store.cc:276] 0709 13:19:54.149229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.165404 (+ 16175us) service_pool.cc:224] Handling call
0709 13:19:54.177084 (+ 11680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.603903 (+426819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.603944 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.604517 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.907330 (+302813us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:54.907392 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.907527 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.321765 (+414238us) raft_consensus.cc:1685] finished
0709 13:19:55.321812 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.322101 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":406739,"raft.queue_time_us":408738,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":154,"thread_start_us":302209,"threads_started":1,"wal-append.queue_time_us":328768}
I20260709 13:19:55.321631 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.327585 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.327528 10049 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 26: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:55.328393 10047 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:55.279716 17559 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:19:55.337778 10052 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:55.345161 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:55.331264 17517 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:55.360113 17555 log.cc:927] Time spent T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.228s	user 0.010s	sys 0.000s
I20260709 13:19:55.361900 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48538 (request call id 952) took 1258 ms. Trace:
I20260709 13:19:55.362432 10050 rpcz_store.cc:276] 0709 13:19:54.103382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.377775 (+274393us) service_pool.cc:224] Handling call
0709 13:19:54.378125 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.875584 (+497459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.361529 (+485945us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54002e95d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:55.361819 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105996160}
I20260709 13:19:55.366168 17228 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:55.369935 17344 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:55.371500 17373 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.282680 17222 rpcz_store.cc:276] 0709 13:19:49.779360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.787407 (+  8047us) service_pool.cc:224] Handling call
0709 13:19:49.814440 (+ 27033us) log.cc:844] Serialized 494 byte log entry
0709 13:19:50.172174 (+357734us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b54003d34a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7d142 00007fa277f5deaa 00007fa277f64d53 00007fa277f64cd9 00007fa277f64c69 00007fa277f64c31 00007fa277f63dfd 000000000038f354 000000000038f199 00007fa274f17475
0709 13:19:50.746415 (+574241us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b54003d34a0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7d142 00007fa277f5deaa 00007fa277f64d53 00007fa277f64cd9 00007fa277f64c69 00007fa277f64c31 00007fa277f63dfd 000000000038f354 000000000038f199 00007fa274f17475
0709 13:19:55.233841 (+4487426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5297,"raft.queue_time_us":8536,"raft.run_cpu_time_us":7115,"raft.run_wall_time_us":1274243,"spinlock_wait_cycles":175128704}
I20260709 13:19:55.373342 17519 consensus_queue.cc:260] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.376180 10052 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:55.380141 10055 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:55.378098 10047 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:19:55.383087 10054 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:55.385663 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 139821us
I20260709 13:19:55.389123 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.387750 10051 raft_consensus.cc:2468] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
W20260709 13:19:55.402036 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 165708us
W20260709 13:19:55.411494 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.412634 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.392092 17517 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:55.411424 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.423843 17490 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:55.438815 17464 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.441602 17464 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:55.453486  9951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 199) took 82356 ms (client timeout 15000 ms). Trace:
W20260709 13:19:55.455924 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 137847us
I20260709 13:19:55.414937 17519 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:55.462019 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:55.425320 17372 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:55.475463 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:55.487673 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 210548us
I20260709 13:19:55.472424 17490 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:55.494076 17492 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:19:55.510537 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.515202  9939 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cebbf225b866442ab895577b0496de1b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 13:19:55.454129  9951 rpcz_store.cc:269] 0709 13:18:33.097166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.097403 (+   237us) service_pool.cc:224] Handling call
0709 13:19:55.453136 (+82355733us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:43.569099 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:43.778017 (+208918us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:43.778086 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:19:43.778136 (+    50us) tablet.cc:662] Decoding operations
0709 13:19:43.779079 (+   943us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:43.779278 (+   199us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:43.779319 (+    41us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:19:43.819736 (+ 40417us) tablet.cc:701] Row locks acquired
0709 13:19:43.819805 (+    69us) write_op.cc:260] PREPARE: finished
0709 13:19:43.820041 (+   236us) write_op.cc:270] Start()
0709 13:19:43.820266 (+   225us) write_op.cc:276] Timestamp: P: 1783603183819956 usec, L: 0
0709 13:19:43.820319 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 13:19:43.821377 (+  1058us) log.cc:844] Serialized 2079 byte log entry
0709 13:19:45.215722 (+1394345us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b1800282a00. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:45.858298 (+642576us) op_driver.cc:464] REPLICATION: finished
0709 13:19:46.462942 (+604644us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa27800132a 00007fa277f86f14 00007fa277f86ec3 00007fa277f86e38 00007fa277f86df3 00007fa277f860dc 00007fa277fafe16 00007fa277f97cbb 00007fa277f8239d 00007fa277f85d75
0709 13:19:46.584215 (+121273us) write_op.cc:301] APPLY: starting
0709 13:19:46.597445 (+ 13230us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:46.657763 (+ 60318us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:46.657808 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:47.134862 (+477054us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:47.208830 (+ 73968us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:19:47.258258 (+ 49428us) write_op.cc:312] APPLY: finished
0709 13:19:47.328479 (+ 70221us) log.cc:844] Serialized 57 byte log entry
0709 13:19:47.681111 (+352632us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:47.704010 (+ 22899us) write_op.cc:454] Released schema lock
0709 13:19:47.713794 (+  9784us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:47.714471 (+   677us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:47.714510 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3865494,"reported_tablets":52,"spinlock_wait_cycles":1623296,"child_traces":[["op",{"apply.queue_time_us":552402,"apply.run_cpu_time_us":42401,"apply.run_wall_time_us":1157768,"cfile_cache_hit":22,"cfile_cache_hit_bytes":24987,"num_ops":5,"prepare.queue_time_us":578694,"prepare.run_cpu_time_us":29688,"prepare.run_wall_time_us":610855,"raft.queue_time_us":570140,"raft.run_cpu_time_us":55187,"raft.run_wall_time_us":670821,"replication_time_us":1999089,"spinlock_wait_cycles":41608960,"thread_start_us":1528313,"threads_started":5,"wal-append.queue_time_us":956256,"wal-append.run_cpu_time_us":55819,"wal-append.run_wall_time_us":2644608}]]}
I20260709 13:19:55.520022 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.496981 17490 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:55.521274 10053 raft_consensus.cc:3060] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:55.529242 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.535310 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:19:55.538364 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 759930us
I20260709 13:19:55.552446 17325 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:55.552582 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:55.553916 10894 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:55.555720 17490 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.512692 17492 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:55.558037 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.557107 17537 raft_consensus.cc:3055] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:55.575243 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:55.582069 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:55.582866 10893 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:19:55.582298 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:19:55.585155 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 295380us
I20260709 13:19:55.573503 17492 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:55.573083 17517 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.577266 17537 raft_consensus.cc:740] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:55.591367 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:55.597133 17203 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:55.597708 17203 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.598181 17203 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:55.598440 17477 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:55.573242 17372 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 13:19:55.605762 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 109604us
I20260709 13:19:55.588812 17558 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:55.616082 17558 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:19:55.617154 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.610704 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67435.4us
I20260709 13:19:55.629285 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:55.633994 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.638381 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:55.638870 10202 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:55.643652 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:55.645191 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:55.648597 10892 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:55.650372 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 61753.4us
I20260709 13:19:55.652395 10200 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:55.655866 10197 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:55.624362 17558 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:55.661132 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 256132us
I20260709 13:19:55.600409 17517 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.662879 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:55.661231 10053 raft_consensus.cc:2468] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:55.682410 17615 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:19:55.692474 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 82370.6us
I20260709 13:19:55.688401 17464 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:55.658806 17373 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:55.705863 10893 raft_consensus.cc:2468] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
I20260709 13:19:55.721302 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:55.752513 10131 raft_consensus.cc:1240] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:55.754683 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:55.755722 10125 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:55.768512 17614 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:19:55.769483 17614 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:19:55.769610 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 64598.4us
I20260709 13:19:55.770718 17614 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.772300 17614 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:55.773487 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:55.774504 10122 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:55.775933 17330 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:55.778633 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.779882 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 142347us
I20260709 13:19:55.778563 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:55.783423 10124 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:55.793778 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 130939us
I20260709 13:19:55.741752 17615 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:19:55.810071 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.822350 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.860635 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78584.3us
I20260709 13:19:55.861012 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:55.862212 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 232849us
W20260709 13:19:55.862417 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 156776us
W20260709 13:19:55.867545 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.868537 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:55.776499 17330 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:55.787096 17373 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.868294 10122 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:19:55.875308 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 248883us
I20260709 13:19:55.874768 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:55.855082 17407 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:55.893344 10893 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:55.884591 10169 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:55.853596 17615 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:55.905292 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.876655 17286 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:55.910481 10894 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:19:55.872045 17559 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:55.857729 17325 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:55.896937 17407 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:19:55.902112 17615 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:55.911657 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:55.923280 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.934324 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 49930.7us
I20260709 13:19:55.946427 10124 raft_consensus.cc:2468] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 15.
W20260709 13:19:55.955554 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58130.2us
I20260709 13:19:55.895134 17373 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:55.941159 17126 raft_consensus.cc:2804] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:19:55.906204 17477 consensus_queue.cc:1059] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 13:19:55.948925 17595 consensus_queue.cc:1048] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:19:55.922883 17393 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:19:55.974423 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 68688.4us
I20260709 13:19:56.026091 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.974666 10056 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:55.975054 17626 consensus_queue.cc:1059] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:56.026827 10051 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
W20260709 13:19:55.976524 17609 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.339s	user 0.002s	sys 0.001s
I20260709 13:19:55.981683 17596 consensus_queue.cc:1048] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:55.937196 17559 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:55.983824 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:55.941262 17407 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:56.003945 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:56.005357 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.007899 10048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:55.911211 17328 consensus_queue.cc:1059] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:19:56.013936 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 100660us
I20260709 13:19:56.051630 10203 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:19:56.018949 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:56.053000 10047 raft_consensus.cc:2410] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 13:19:56.019925 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:56.020792 10050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:56.022341 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:56.024359 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 161077us
I20260709 13:19:56.033674 10892 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.025897 10049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:56.040318 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:55.946491 17597 consensus_queue.cc:1048] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:19:56.057560 10894 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:56.051189 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:56.048424 10048 raft_consensus.cc:2376] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17. Re-sending same reply.
W20260709 13:19:56.054773 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 111851us
I20260709 13:19:56.057869 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.068924 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:56.068483 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.070648 10204 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:56.057404 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.072551 10170 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:56.057507 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.074692 10095 leader_election.cc:400] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20.
I20260709 13:19:56.075150 10095 leader_election.cc:403] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:19:56.079497 10122 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.057363 10054 raft_consensus.cc:1275] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:19:56.081036 10048 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:56.084753 10200 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:56.086650 10201 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
W20260709 13:19:56.088526 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 213850us
I20260709 13:19:55.960707 17627 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:19:56.090482 10051 raft_consensus.cc:3055] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:56.095633 10051 raft_consensus.cc:740] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:56.097922 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 70468.9us
I20260709 13:19:56.046741 17126 raft_consensus.cc:697] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.097071 10051 consensus_queue.cc:260] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.115223 10051 raft_consensus.cc:3060] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:56.078626 17594 raft_consensus.cc:3055] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:19:56.123420 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 67742us
I20260709 13:19:56.041520 17286 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.080s
I20260709 13:19:56.125530 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.129004 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:55.960132 17330 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.028898 17622 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 13:19:56.147372 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48476.8us
I20260709 13:19:56.148254 17328 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:56.155642 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83214us
I20260709 13:19:56.166049 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:56.156677 17620 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.157408 17626 raft_consensus.cc:3055] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:56.167524 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:56.169293 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.176656 10122 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:56.053000 17621 consensus_queue.cc:1059] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 13:19:56.090314 17619 consensus_queue.cc:1059] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 13:19:56.124240 17594 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.205775 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:56.211906 10122 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:56.137789 17627 consensus_queue.cc:1243] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:19:56.213338 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 130942us
I20260709 13:19:56.159468 17517 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.216150 10168 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.164113 17508 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:56.225824 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 100211us
I20260709 13:19:56.226773 10892 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 13:19:56.228327 10892 pending_rounds.cc:85] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 17
I20260709 13:19:56.184937 17626 raft_consensus.cc:740] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:56.234189 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 188529us
I20260709 13:19:56.210512 17375 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.236965 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:56.228646 10892 pending_rounds.cc:107] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 24.18
W20260709 13:19:56.249502 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 120573us
I20260709 13:19:56.251838 10892 raft_consensus.cc:2889] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:56.241142 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73675us
I20260709 13:19:56.239677 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.254805 10122 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:19:56.130191 17638 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.113s
I20260709 13:19:56.260160 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.200009 17222 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:19:56.234315 17517 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.240540 17477 raft_consensus.cc:3055] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:56.235517 17613 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20.
W20260709 13:19:56.299189 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.259254 17624 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:56.270514 17632 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.302500 17373 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:56.302346 17626 consensus_queue.cc:260] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.279517 17622 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:56.210731 17620 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.315929  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:56.263401 17361 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:56.359045 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 54301.3us
W20260709 13:19:56.343096 10125 consensus_meta.cc:287] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.587s	user 0.015s	sys 0.001s
I20260709 13:19:56.299598 17601 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.263s
I20260709 13:19:56.365399 10125 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:56.279824 17477 raft_consensus.cc:740] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:56.378847 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.379539 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.380614 10125 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 12.
I20260709 13:19:56.382951 10122 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:19:56.384354 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 147503us
I20260709 13:19:56.386874 10122 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:19:56.393417 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 141707us
I20260709 13:19:56.395349 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:56.404997 10051 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:19:56.406239 10051 pending_rounds.cc:85] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 10
I20260709 13:19:56.406786 10051 pending_rounds.cc:107] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:19:56.407436 10051 raft_consensus.cc:2889] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.408100 10125 raft_consensus.cc:1275] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:19:56.399766 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:56.385082 17617 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.413s	user 0.000s	sys 0.002s
I20260709 13:19:56.300639 17517 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:56.421022 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 167103us
I20260709 13:19:56.430012 10122 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:19:56.432246 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.390976 17601 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:19:56.270428 17126 consensus_queue.cc:237] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.440616 10125 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:56.408787 17626 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.414903 17613 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:56.441672 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:56.443545 17638 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:56.443938 17633 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:56.444428 17622 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:19:56.445576 17203 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.847s	user 0.019s	sys 0.000s
W20260709 13:19:56.445930 17639 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 13:19:56.446908 17649 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:19:56.447556 17656 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:56.448375 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53174.7us
I20260709 13:19:56.469583 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.476267 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.476522 10201 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:56.472321 10125 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:56.488613 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.482208 17203 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:56.494482 17650 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:19:56.459474 17613 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.496337 17203 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.496520 17613 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.316035 17594 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.514060 17203 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.521164 17203 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.479513 17559 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.478637 17622 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.367915 17632 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:56.511847 17558 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.003s
W20260709 13:19:56.533373 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 284394us
I20260709 13:19:56.498000 17645 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.624176 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:56.522971 17519 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 1.042s	user 0.014s	sys 0.004s
W20260709 13:19:56.549460 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 107859us
I20260709 13:19:56.633337 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:56.552500 17634 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:19:56.553198 17203 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:56.555589 17611 log.cc:927] Time spent T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.950s	user 0.009s	sys 0.000s
W20260709 13:19:56.557157 17614 consensus_meta.cc:287] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.004s
I20260709 13:19:56.489784 17477 consensus_queue.cc:260] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.559556 17393 raft_consensus.cc:3055] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:19:56.527066 17630 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:19:56.561154 17653 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.563099 10128 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:19:56.563793 17517 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:56.569346 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.570338 17632 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:56.570660 17615 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.007s
I20260709 13:19:56.499547 17656 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.579782 17641 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:56.583601 10127 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:19:56.589674 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.591830 17330 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:56.605082 10865 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:56.628337 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 228877us
I20260709 13:19:56.527707 17638 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.544706 17622 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.543779 17619 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:19:56.635449 17492 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.035s	user 0.011s	sys 0.010s
I20260709 13:19:56.488123 17633 raft_consensus.cc:697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.670076 10128 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:19:56.670739 17637 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:19:56.653762 17393 raft_consensus.cc:740] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.637925 17660 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:56.667197 17203 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.666131 17653 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.691325 17477 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:56.692093 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.692266 17644 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:56.653939 17346 raft_consensus.cc:2764] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 13:19:56.710989 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77778us
I20260709 13:19:56.746299 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:56.716645 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92560.4us
I20260709 13:19:56.747912 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:56.750001 10096 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.624245 17558 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.717764 10095 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.699504 17291 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.696437 17641 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.632200 17645 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:56.730300 17490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.121s	user 0.032s	sys 0.040s
I20260709 13:19:56.804616 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.737905 17203 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:56.756423 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.756592 17656 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.726991 17393 consensus_queue.cc:260] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.697988 17615 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.752066 17644 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.764487 17638 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:56.765501 17407 consensus_meta.cc:287] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.004s
W20260709 13:19:56.804167 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54304.5us
I20260709 13:19:56.812434 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:56.777231 17628 raft_consensus.cc:3055] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:56.788641 17624 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:56.762873 17626 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.720777 17492 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.789291 17596 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:56.790270 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:19:56.836345 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46249.2us
I20260709 13:19:56.798738 10168 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:56.726842 17633 consensus_queue.cc:237] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:56.806784 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.841224 10131 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:19:56.809346 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105702us
I20260709 13:19:56.734545 17620 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:56.775250 17627 raft_consensus.cc:3055] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:19:56.738369 17653 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.727907 17619 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:56.818960 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 72752us
W20260709 13:19:56.803885 17490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.195s	user 0.032s	sys 0.040s
I20260709 13:19:56.832588 10202 raft_consensus.cc:1240] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:56.842350 10168 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:56.816015 17614 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:56.839531 17626 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.844877 17652 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:56.845759 17647 raft_consensus.cc:2804] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:56.803407 17291 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.846287 17517 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.889459 17372 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:56.889946 17372 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:56.890008 17517 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.891072 17372 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.892271 17372 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:56.892064 17653 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.832330 17393 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:56.848233 17615 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:56.848510 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:56.894361 17653 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:56.894899 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46503.5us
I20260709 13:19:56.849184 17397 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.850610 17632 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:56.850836 10130 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:56.840803 17655 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:56.851465 17492 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.852325 17668 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.846897 17596 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.798555 17641 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:56.724576 17658 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 13:19:56.808419 17651 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:56.937438 17651 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.937968 17651 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:56.856920 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:56.939404 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 97292.9us
I20260709 13:19:56.941847 10124 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:56.864374 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:56.942461 10122 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:56.903812 17632 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.943773 10125 raft_consensus.cc:1240] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:56.956498 17632 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:56.945211 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3436) took 1225 ms. Trace:
I20260709 13:19:56.872138 17645 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.932147 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1141) took 1388 ms. Trace:
I20260709 13:19:56.873785 17614 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.957811 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.824007 17407 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.950655 17492 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:56.904438 17517 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:56.953155 17596 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:56.870184 17626 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:56.881088 17203 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.881029 17558 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.858047 17627 raft_consensus.cc:740] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:19:56.961349 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.882028 17663 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:56.959776 17492 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:56.962349 17663 raft_consensus.cc:697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.962580 17492 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:56.963177 17407 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:56.963559 17663 consensus_queue.cc:237] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.964466 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:56.965962 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.871224 17644 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:56.884442 17619 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:56.951054 17653 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.885102 17126 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:56.967698 17653 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.823870 17464 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:56.823751 17628 raft_consensus.cc:740] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:56.970180 17653 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:56.914728 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48560.5us
I20260709 13:19:56.916185 10131 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
W20260709 13:19:56.904599 17344 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.012s	sys 0.000s
I20260709 13:19:56.933132 17673 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:19:56.931489 17668 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.920605 17647 raft_consensus.cc:697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:56.986720 10055 raft_consensus.cc:1275] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:19:56.935336 17375 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.681s	user 0.009s	sys 0.007s
I20260709 13:19:56.936208 17655 raft_consensus.cc:697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:56.936549 17672 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.874449 17537 consensus_queue.cc:260] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:56.994395 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.997103 10124 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:56.943024 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78801us
I20260709 13:19:56.871745 17621 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:56.872375 17618 log.cc:927] Time spent T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.913s	user 0.006s	sys 0.000s
I20260709 13:19:56.958189 17614 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:56.873826 17656 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.879513 17631 raft_consensus.cc:3055] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:56.961170 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.945578 17583 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.883530 17328 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:56.969956 17628 consensus_queue.cc:260] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.896669 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:57.014400 17655 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:57.015815 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 119668us
I20260709 13:19:57.016744 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:57.018046 17203 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:19:56.889298 17291 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.018937 17203 consensus_queue.cc:1059] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:57.020205 17203 raft_consensus.cc:3055] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:19:57.020534 17596 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.020632 17203 raft_consensus.cc:740] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:57.021198 17596 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:56.913095 17397 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.021623 17660 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.022598 17660 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.023316 17222 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.023918 17222 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.025832 17660 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:57.026139 17222 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:57.026665 17660 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.051293 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.972543 10122 raft_consensus.cc:1240] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:56.973161 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 56787.1us
W20260709 13:19:56.974247 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.904070 17620 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.957564 10130 rpcz_store.cc:276] 0709 13:19:55.719774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.720837 (+  1063us) service_pool.cc:224] Handling call
0709 13:19:56.944931 (+1224094us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540015a254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:56.945111 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563166080}
I20260709 13:19:56.932354 17633 consensus_queue.cc:1059] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:19:56.981772 10051 raft_consensus.cc:1275] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:19:57.064564 10051 pending_rounds.cc:85] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 12
I20260709 13:19:57.064982 10051 pending_rounds.cc:107] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:19:57.065317 10051 raft_consensus.cc:2889] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.987583 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.066210 10053 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:56.958586 10899 rpcz_store.cc:276] 0709 13:19:55.530768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.543280 (+ 12512us) service_pool.cc:224] Handling call
0709 13:19:55.543877 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.544249 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:19:55.680478 (+136229us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.918009 (+1237531us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b18004ad460. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277bac4bc 00007fa274efa48b 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a
0709 13:19:56.918330 (+   321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:56.918417 (+    87us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:56.918499 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.918632 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.918843 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":214339,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":17006,"raft.queue_time_us":143,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":3836,"spinlock_wait_cycles":106730368,"thread_start_us":269495,"threads_started":2,"wal-append.queue_time_us":386232}
I20260709 13:19:56.991678 10047 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:56.991987 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.068636 10052 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:19:56.993505 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:56.974154 17344 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.012s	sys 0.000s
I20260709 13:19:57.000033 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:57.001330 10055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.004736 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:57.011600 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 174911us
I20260709 13:19:57.072992 10055 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:19:57.014751 17678 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:19:56.971803 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.018769 17619 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:57.034001 17668 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:57.100454 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.035023 17647 consensus_queue.cc:237] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.005942 17621 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.037259 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3447) took 1234 ms. Trace:
I20260709 13:19:57.106320 10131 rpcz_store.cc:276] 0709 13:19:55.803823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.860587 (+ 56764us) service_pool.cc:224] Handling call
0709 13:19:57.037014 (+1176427us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54001266d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:57.037172 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2251571584}
I20260709 13:19:57.108125 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.108866 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.039346 17490 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.040001 17559 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.991637 17375 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:56.980207 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.057107 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.071013 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.069882 10054 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 13.
I20260709 13:19:57.130926 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.133721 10130 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:57.134869 10055 raft_consensus.cc:2468] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:19:57.071978 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:56.989131 17613 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.072002 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.076736 10123 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:57.073623 10096 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:57.077463 10200 raft_consensus.cc:1240] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:19:57.141642 10123 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:57.077639 10196 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:19:57.079010 10199 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:19:57.079883 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:57.143185 10131 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:57.143453 10197 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:57.081144 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.003144 17631 raft_consensus.cc:740] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:57.144286 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.083794 10205 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:57.145740 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.084291 10900 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:57.088280 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.026835 17222 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:19:57.099453 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:57.099819 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99927.9us
W20260709 13:19:57.151500 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77964us
I20260709 13:19:57.101583 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:56.999969 17519 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 13:19:57.071851 17660 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:57.111882 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.163661 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:57.118510 10053 raft_consensus.cc:2468] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:57.070612 17628 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:57.136149 10122 raft_consensus.cc:2393] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:57.138566 10056 raft_consensus.cc:2393] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:57.139976 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:57.151654 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100707us
I20260709 13:19:57.135305 17673 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.146960 10131 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:57.152776 10047 raft_consensus.cc:2393] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:57.154655 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.179840 10124 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:57.154805 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.155426 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:57.156350 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:57.180996 10200 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:57.157191 17621 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:57.021744 17203 consensus_queue.cc:260] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.184721 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.084728 17622 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.158356 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.160013 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.098671 17397 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.170670 17596 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:57.052269 17517 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:57.165419 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.166213 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.215510 10203 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:57.101713 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.174533 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.217113 10128 raft_consensus.cc:2376] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22. Re-sending same reply.
I20260709 13:19:57.176991 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.176863 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.180269 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.187438 17393 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:19:57.185961 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.220122 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.220975 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.221108 10054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c"
I20260709 13:19:57.222044 10198 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:19:57.195688 10892 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:57.105643 17595 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:57.173250 17655 raft_consensus.cc:2764] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:19:57.235227 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.236336 10047 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:57.130873 17490 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.177606 17638 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:19:57.206372 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.210143 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.210768 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:57.218351 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.219159 10056 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:19:57.219339 10125 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.258260 10128 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:57.182997 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.258807 10056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.156994 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:57.259297 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.258634 10047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.170974 17619 consensus_queue.cc:1059] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:19:57.265102 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.151010 17559 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.266309 10053 raft_consensus.cc:2468] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 15.
I20260709 13:19:57.228928 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:57.230312 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74972.1us
W20260709 13:19:57.267640 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 110760us
I20260709 13:19:57.268042 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:57.233340 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59230.5us
I20260709 13:19:57.238293 10051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
W20260709 13:19:57.240548 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.244895 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.224364 10129 raft_consensus.cc:1240] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:57.259835 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.259981 10128 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 25.
I20260709 13:19:57.270721 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.271109 10122 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:57.271551 10128 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:19:57.268389 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.260484 10047 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
I20260709 13:19:57.276223 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.277266 10053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.277828 10892 raft_consensus.cc:1240] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20. Ops: []
I20260709 13:19:57.000836 17672 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.280480 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.280853 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.285337 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.286356 10198 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:57.286893 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.281843 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.246469 17407 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:57.294334 10131 raft_consensus.cc:2468] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:57.281901 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.295320 10053 raft_consensus.cc:1240] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:57.294967 10122 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:57.299394 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.300570 10122 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:57.303386 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92750.8us
I20260709 13:19:57.236626 17596 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:19:57.267298 17477 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.314908 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.305804 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.185114 17673 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.308194 10056 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:19:57.312141 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.328730 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.330013 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.330830 10128 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:19:57.331923 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.233536 17517 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.333758 10170 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.316883 10131 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:57.346014 10170 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:57.344422 10125 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:57.145459 17631 consensus_queue.cc:260] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.342672 10048 raft_consensus.cc:1275] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 13:19:57.294739 17644 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:57.280198 17668 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:57.363250 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:57.324566 17393 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:57.312512 17291 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:57.281013 17638 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:57.355801 10863 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:57.360917 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.318643 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.348255 10125 raft_consensus.cc:1240] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:57.379616 17222 raft_consensus.cc:2804] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:57.379381 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:57.380164 17679 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:19:57.294180 17620 consensus_queue.cc:1059] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:19:57.360575 17375 raft_consensus.cc:2804] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:19:57.368973 17644 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.349577 17407 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:57.325584 17633 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:57.349640 17656 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.425153 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.427865 10130 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:19:57.429616 10125 raft_consensus.cc:1240] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
W20260709 13:19:57.432389 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.372309 17393 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.324631 17646 consensus_queue.cc:1048] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.232s
I20260709 13:19:57.344375 17477 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.419097 17291 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.464011 10122 raft_consensus.cc:1240] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:57.464260 10130 raft_consensus.cc:1240] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:57.363360 17626 consensus_queue.cc:1059] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:19:57.372967 17583 consensus_queue.cc:1048] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:19:57.374583 17592 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:19:57.379420 17636 consensus_queue.cc:1059] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.275s
I20260709 13:19:57.460806 17222 raft_consensus.cc:697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:57.415863 17228 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:57.382534 17663 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:19:57.441377 17633 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.437065 17644 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:57.434993 10126 raft_consensus.cc:1240] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
W20260709 13:19:57.446599 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67339.5us
I20260709 13:19:57.492416 17656 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.368687 17595 consensus_queue.cc:1048] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:19:57.369015 17632 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.506283 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.370349 17688 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:19:57.508210 10125 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:19:57.509479 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 199388us
I20260709 13:19:57.567400 17228 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.468827 17393 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:57.467746 17620 raft_consensus.cc:3055] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:19:57.509913 17679 log.cc:927] Time spent T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:19:57.484278 17291 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.477802 17613 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.433038 17325 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:19:57.510828 17328 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:19:57.592818 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:57.672840 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.673566 10126 raft_consensus.cc:2468] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:57.606446 10122 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:57.477137 10130 raft_consensus.cc:1240] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:57.608090 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.581538 17633 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:57.582576 17622 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.582368 17660 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:19:57.648712 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.648761 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 293049us
I20260709 13:19:57.678990 17700 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:57.674932 10126 raft_consensus.cc:1275] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:57.622159 17228 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:57.681699 17375 raft_consensus.cc:697] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:57.643234 17620 raft_consensus.cc:740] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:57.594666 17407 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:57.696069 17683 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.404s	user 0.002s	sys 0.000s
W20260709 13:19:57.715602 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.708611 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.724646 10122 raft_consensus.cc:2393] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:19:57.728406 10200 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.007s
I20260709 13:19:57.746266 10200 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:19:57.729501 17375 consensus_queue.cc:237] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:57.732736 10197 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.004s
W20260709 13:19:57.734887 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 352898us
W20260709 13:19:57.714836 17682 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.389s	user 0.001s	sys 0.001s
I20260709 13:19:57.679847 17700 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.796110 10197 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:57.693811 17328 consensus_queue.cc:1048] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:19:57.715276 17517 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.279820 17627 consensus_queue.cc:260] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.806134 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.787902 17673 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.571375 17638 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:57.785725 17714 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:19:57.793771 17702 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.796033 17688 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:57.794854 17672 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.703612 17222 consensus_queue.cc:237] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.857213 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.837494 17627 raft_consensus.cc:3060] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:19:57.853430 17687 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.384s	user 0.002s	sys 0.000s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
W20260709 13:19:57.863035 17641 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.927s	user 0.014s	sys 0.003s
I20260709 13:19:57.850759 17559 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:57.877942 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.878517 17537 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:57.879240 17656 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:57.864110 17710 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:19:57.880187 17632 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:57.883234 17651 consensus_meta.cc:287] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.945s	user 0.012s	sys 0.016s
I20260709 13:19:57.853161 17688 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.884061 17594 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:57.884984 17126 consensus_queue.cc:1048] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:57.892441 17707 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:19:57.895519 17690 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.325s	user 0.001s	sys 0.001s
I20260709 13:19:57.903321 17626 consensus_queue.cc:1243] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 18.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:19:57.903623 17692 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
W20260709 13:19:57.905175 17695 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:19:57.905622 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3458) took 1549 ms. Trace:
I20260709 13:19:57.961818 10129 rpcz_store.cc:276] 0709 13:19:56.356098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.551335 (+195237us) service_pool.cc:224] Handling call
0709 13:19:56.551761 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.269594 (+717833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.905344 (+635750us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540014bc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:57.905546 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546099968}
I20260709 13:19:57.907783 17700 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:57.917062 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 309041us
I20260709 13:19:57.965159 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.966281 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.923539 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:57.924305 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.937863 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.967206 10126 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:57.967337 10197 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:57.941851 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.944809 10052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "01bfaf1ad56a4deea886bbc64dec466c" is_pre_election: true
I20260709 13:19:57.943867 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.909719 17477 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:57.973381 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:57.948798 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.949288 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:57.976122 10203 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:57.948393 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.835486 17620 consensus_queue.cc:260] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.953563 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.978360 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.979228 10891 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:57.953783 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.954638 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.968907 10052 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:19:57.969483 10126 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:57.969506 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:57.943852 17492 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.980s	user 0.015s	sys 0.000s
I20260709 13:19:57.916210 17490 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:57.971772 10198 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:57.918807 17641 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:57.932654 17656 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.979359 10893 raft_consensus.cc:2410] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:19:57.851876 17702 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.994344 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:57.983036 10895 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:57.984840 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:57.998304 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:57.950022 17697 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:58.003512 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.004395 10199 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:19:57.999719 10900 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 22: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:57.963557 17646 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:19:57.973384 17696 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:19:57.990347 17614 consensus_meta.cc:287] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.950s	user 0.007s	sys 0.008s
I20260709 13:19:57.998015 17656 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:57.990859 17477 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:58.006676 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:58.033937 10127 raft_consensus.cc:1240] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:58.035802 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51104us
I20260709 13:19:58.048113 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.048544 10894 raft_consensus.cc:1275] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:19:58.037750 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:58.009030 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.049865 10893 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:58.038686 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:58.018353 17344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.013s	sys 0.022s
I20260709 13:19:58.056676 10895 raft_consensus.cc:2368] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 23. Current term is 24.
I20260709 13:19:58.057214 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:58.034145 17692 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:19:58.033510  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
I20260709 13:19:57.947510 17651 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.009474 17620 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:58.010627 17697 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.062269 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.064167 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:58.044677 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.063750 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 90445.5us
I20260709 13:19:58.010785 17696 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.068423 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.069296 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:58.076380 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.094707 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3477) took 1106 ms. Trace:
I20260709 13:19:58.037103 17464 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.038645 17330 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.038873 17655 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.991541 17492 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.078054 10196 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:57.837792 17645 consensus_queue.cc:1175] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d51ad94710c348889d417f7f00032948" responder_term: 25 status { last_received { term: 25 index: 16 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: [24.16-24.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:58.047179 17614 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.091075 17344 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:58.081791 17697 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:58.115491 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.098431 10865 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:58.109289 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:58.109330 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.118916 10048 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3480) took 1128 ms. Trace:
I20260709 13:19:58.121718 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:58.108083 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 50953.9us
I20260709 13:19:58.145949 10128 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:58.119498 10048 rpcz_store.cc:276] 0709 13:19:56.990238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.373216 (+382978us) service_pool.cc:224] Handling call
0709 13:19:57.373685 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.375914 (+  2229us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:57.431166 (+ 55252us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.895619 (+464453us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b180004e2e0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa27bed858f 00007fa274f17571 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd
0709 13:19:57.906633 (+ 11014us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.906822 (+   189us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.907399 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.907553 (+   154us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:57.907775 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.907931 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.933087 (+ 25156us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.118603 (+185516us) raft_consensus.cc:1685] finished
0709 13:19:58.118637 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.118826 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":334200,"prepare.queue_time_us":24107,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":5798,"raft.queue_time_us":424516,"raft.run_cpu_time_us":551,"raft.run_wall_time_us":33722,"spinlock_wait_cycles":81815424,"thread_start_us":105025,"threads_started":1,"wal-append.queue_time_us":219139}
I20260709 13:19:58.148964 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.124528 10123 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 13.
I20260709 13:19:58.095263 10053 rpcz_store.cc:276] 0709 13:19:56.989408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.297712 (+308304us) service_pool.cc:224] Handling call
0709 13:19:57.298146 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.409385 (+111239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.409426 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.409987 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.729438 (+319451us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:57.729798 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.094330 (+364532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.094427 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.094466 (+    39us) raft_consensus.cc:1685] finished
0709 13:19:58.094478 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.094629 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205539,"prepare.queue_time_us":379,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":290520,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":29098,"spinlock_wait_cycles":24584064,"thread_start_us":529514,"threads_started":3,"wal-append.queue_time_us":289954}
I20260709 13:19:58.172120 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.174264 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.097712 17696 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:58.170972 17632 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:58.110926 17477 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:58.178349 17366 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:58.178996 10050 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3472) took 1195 ms. Trace:
I20260709 13:19:58.100585 17633 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.179237 17720 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:19:58.147953 17708 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:19:58.107385 17464 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.117583 17724 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:58.154726 17393 consensus_meta.cc:287] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.005s
I20260709 13:19:58.150964 17344 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:58.148844 17700 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:58.124689 17490 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:58.109854 17655 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.188486 10050 rpcz_store.cc:276] 0709 13:19:56.983970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.984113 (+   143us) service_pool.cc:224] Handling call
0709 13:19:56.984607 (+   494us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:56.999399 (+ 14792us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:57.168993 (+169594us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.316547 (+147554us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:57.362991 (+ 46444us) log.cc:844] Serialized 52 byte log entry
0709 13:19:57.376820 (+ 13829us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:57.376892 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.377018 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.178697 (+801679us) raft_consensus.cc:1685] finished
0709 13:19:58.178721 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.178892 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":289895,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":6884,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":17843,"thread_start_us":225637,"threads_started":2,"wal-append.queue_time_us":250352}
I20260709 13:19:58.221546 17490 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.274591 17490 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:58.276379 17633 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.282043 17477 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.283519 17710 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.284066 17710 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.268330 17344 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.148783 17613 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:58.185166 17688 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.188130 17632 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.187772 17366 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:58.197999 17700 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.191540 17726 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:19:58.313347 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.313547 17727 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.296676 17710 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.312165 17632 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.225421 17393 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.311893 17614 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.317302 17700 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.321237 17656 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.321965 17641 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:58.350461 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 177230us
I20260709 13:19:58.352715 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.355459 17724 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:58.333907 17726 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.331590 17651 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.333153 17702 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.372045 17721 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.372929 17721 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.344659 17477 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.397992 17664 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.406333 10203 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
W20260709 13:19:58.398427 17291 consensus_meta.cc:287] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.005s
I20260709 13:19:58.409267 17291 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.409929 17735 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.410665 17735 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.401368 17627 consensus_meta.cc:287] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.007s
I20260709 13:19:58.411697 17291 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.411865 17733 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.412565 17733 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.413144 17735 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.394886 17492 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.416340 17558 raft_consensus.cc:3055] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:58.417021 17558 raft_consensus.cc:740] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:58.417986 17558 consensus_queue.cc:260] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.405109 10202 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 966) took 1427 ms. Trace:
I20260709 13:19:58.398459 17477 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.419616 10202 rpcz_store.cc:276] 0709 13:19:56.977717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.981873 (+  4156us) service_pool.cc:224] Handling call
0709 13:19:56.991386 (+  9513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.039429 (+ 48043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.039473 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.040128 (+   655us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.419202 (+379074us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:57.419246 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.401407 (+982161us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b54001ff254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:58.401470 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.401522 (+    52us) raft_consensus.cc:1685] finished
0709 13:19:58.401536 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.405022 (+  3486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109706624,"thread_start_us":378602,"threads_started":1,"wal-append.queue_time_us":558951}
I20260709 13:19:58.420745 17631 raft_consensus.cc:3060] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.421458 17477 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.400748 17664 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:58.403851 17228 consensus_meta.cc:287] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.003s
I20260709 13:19:58.422555 17228 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.396449 17651 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:58.425292 17228 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.426095 17228 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.329741 17727 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.427408 17558 raft_consensus.cc:3060] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:58.428203 17664 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.421933 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.408362 10056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3488) took 1302 ms. Trace:
I20260709 13:19:58.416491 17733 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.433153 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.370337 17724 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.435046 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 82450us
W20260709 13:19:58.430922 17714 log.cc:927] Time spent T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.531s	user 0.004s	sys 0.007s
I20260709 13:19:58.439543 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.434168 10056 rpcz_store.cc:276] 0709 13:19:57.106299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.999299 (+893000us) service_pool.cc:224] Handling call
0709 13:19:58.037183 (+ 37884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.088484 (+ 51301us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:58.110734 (+ 22250us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.341322 (+230588us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.362258 (+ 20936us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.362460 (+   202us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:58.362497 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.362574 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.408097 (+ 45523us) raft_consensus.cc:1685] finished
0709 13:19:58.408120 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.408294 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21664,"prepare.queue_time_us":13382,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":328869,"spinlock_wait_cycles":14203008,"thread_start_us":163962,"threads_started":3,"wal-append.queue_time_us":67702}
I20260709 13:19:58.442561 10020 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:58.446345 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.426662 17228 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.442063 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:58.436126 17627 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.443967 17730 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.247s	user 0.000s	sys 0.002s
I20260709 13:19:58.398028 17490 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.449291 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.462285 10202 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.450546 10201 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:58.458730 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.443641 17733 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:58.463650 10053 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3491) took 1131 ms. Trace:
I20260709 13:19:58.455629 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.453312 17291 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:58.469626 10198 raft_consensus.cc:1240] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 22. Current term is 23. Ops: []
I20260709 13:19:58.469877 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.466671 10053 rpcz_store.cc:276] 0709 13:19:57.332282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.171505 (+839223us) service_pool.cc:224] Handling call
0709 13:19:58.171894 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.348233 (+176339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.348265 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:58.348850 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.400106 (+ 51256us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:58.400180 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.403452 (+  3272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.463411 (+ 59959us) raft_consensus.cc:1685] finished
0709 13:19:58.463430 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.463582 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17590,"raft.queue_time_us":196758,"thread_start_us":202628,"threads_started":2,"wal-append.queue_time_us":20350}
W20260709 13:19:58.472659 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.450795 17735 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:58.459797 17651 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:58.470525 17644 consensus_meta.cc:287] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.007s
W20260709 13:19:58.480456 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.440349 17477 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:19:58.488520 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.482304 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.493669 17739 consensus_queue.cc:1059] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:58.495273 17739 raft_consensus.cc:3055] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:58.495764 17739 raft_consensus.cc:740] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:58.462841 17627 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.496337 10201 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:19:58.496703 17739 consensus_queue.cc:260] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.497984 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.498293 10201 raft_consensus.cc:1240] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:19:58.499310 17644 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.501096 17490 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.503640 10203 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:19:58.503486 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48026.8us
I20260709 13:19:58.504930 10201 raft_consensus.cc:1240] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:19:58.464609 17721 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.504531 10202 raft_consensus.cc:2468] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:19:58.501803 17644 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.498822 17627 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.510960 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.513104 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.482116 17651 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.502822 17477 consensus_queue.cc:1243] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:19:58.476541 17735 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.519556 17492 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:58.522321 17291 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.523008 17291 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.524361 17632 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.525609 17291 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.527814 17632 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.531700 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.532644 10900 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:58.534039 17739 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:58.539688 17697 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.543321 17697 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.547346 17731 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:58.552030 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.553227 17632 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:58.553608 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
W20260709 13:19:58.555155 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.555725 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.556071 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.558259 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:58.558980 10900 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:58.504290 17726 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.504292 17655 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.569818 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.570075 17646 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.571061 17646 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:19:58.573144 17727 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:58.573966 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.574436 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.555190 17492 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:58.582551 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.575680 17735 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.586697 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.587320 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.587637 10897 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:19:58.588915 17537 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:19:58.589499 17680 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.357s	user 0.004s	sys 0.000s
I20260709 13:19:58.589845 17537 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:19:58.590543 17203 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:58.591064 17632 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:58.591653 17632 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.592061 17632 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:58.593699 17628 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.417s	user 0.018s	sys 0.000s
I20260709 13:19:58.594383 17537 consensus_queue.cc:1048] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:58.595887 17700 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.598542 17700 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.518780 17651 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.601953 10900 raft_consensus.cc:2468] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:19:58.600843 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:58.601362 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 118846us
I20260709 13:19:58.608809 10126 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:58.610944 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.612615 10129 raft_consensus.cc:2410] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:19:58.594507 17628 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.613771 17613 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:58.611503 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:58.618400 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.613884 17628 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.619093 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.642264 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.645578 10019 connection.cc:256] RPC call timeout handler was delayed by -1.1538674509999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 13:19:58.646535 10019 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.648501 10865 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.624994 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:58.627573 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.640229 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.652187 10129 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:19:58.652350 10130 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:58.653554 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.629443 10051 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:19:58.653401 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.654057 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.654615 10129 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:19:58.655568 10122 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:58.655793 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:58.656682 10129 raft_consensus.cc:2376] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16. Re-sending same reply.
W20260709 13:19:58.651261 10865 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.664052 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:58.664976 10131 raft_consensus.cc:3060] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:58.649405 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:58.658699 10865 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.670001 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.614925 17228 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.672152 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.675321 17735 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.632963 17628 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.668660 17655 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.634017 17366 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.683300 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.621485 17594 consensus_meta.cc:287] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.673s	user 0.013s	sys 0.000s
I20260709 13:19:58.706889 17735 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.707533 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.701361 17621 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.709569 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:58.709589 17621 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.710655 10897 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:58.648839 17613 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.671092 10122 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:58.685102 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.714604 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.715431 10129 raft_consensus.cc:2468] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:19:58.714613 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:58.713151 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:58.702608 10130 raft_consensus.cc:2468] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:58.717360 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:58.717769 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66637.9us
I20260709 13:19:58.718116 10130 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:58.718595 10865 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.668643 17492 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.725067 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.720005 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.726255 10129 raft_consensus.cc:2393] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:58.671466 17344 raft_consensus.cc:2764] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:58.723292 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.728538 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.729481 10122 raft_consensus.cc:2393] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:58.730499 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:58.731022 10122 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:58.713184 17638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.018s	sys 0.032s
I20260709 13:19:58.734427 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.623099 17696 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.737017 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.720196 17735 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:58.742064 17621 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.722337 17537 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:58.733757 17619 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:58.774123 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:19:58.783632 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46078.4us
W20260709 13:19:58.784129 17691 log.cc:927] Time spent T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.894s	user 0.002s	sys 0.001s
W20260709 13:19:58.784868 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:58.720363 17613 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.786445 10049 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3474) took 1801 ms. Trace:
I20260709 13:19:58.786804 10047 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3478) took 1797 ms. Trace:
I20260709 13:19:58.787312 10047 rpcz_store.cc:276] 0709 13:19:56.989744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.301177 (+311433us) service_pool.cc:224] Handling call
0709 13:19:57.301674 (+   497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.422900 (+121226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.422956 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.423024 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:57.423075 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.786538 (+1363463us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540010bdd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:58.786592 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.786732 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278016768}
I20260709 13:19:58.790149 17696 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.793929 10095 leader_election.cc:400] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 23. Current term is 24.
I20260709 13:19:58.794464 10095 leader_election.cc:403] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:19:58.735824 17492 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.799767 17622 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 23. Current term is 24.
I20260709 13:19:58.796315 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:58.800925 10052 raft_consensus.cc:1275] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:19:58.801261 10095 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:58.788184 17613 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.803057 17731 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:58.803563 17613 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:58.803594 17731 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.801406 10052 pending_rounds.cc:85] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting all ops after (but not including) 8
I20260709 13:19:58.804349 17731 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.804644 10052 pending_rounds.cc:107] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:19:58.805079 10052 raft_consensus.cc:2889] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:58.805203 17464 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.809321 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.810930 10199 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:19:58.786931 10049 rpcz_store.cc:276] 0709 13:19:56.984669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.984797 (+   128us) service_pool.cc:224] Handling call
0709 13:19:56.995794 (+ 10997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.002504 (+  6710us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:19:57.178143 (+175639us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.243360 (+ 65217us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.243909 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.244072 (+   163us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:19:57.244950 (+   878us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.245199 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.245278 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.494117 (+248839us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b1800404440. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:57.889390 (+395273us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b1800404440. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:58.415872 (+526482us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b1800404440. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:19:58.783345 (+367473us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:58.786104 (+  2759us) raft_consensus.cc:1685] finished
0709 13:19:58.786142 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.786360 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":672894,"prepare.queue_time_us":425002,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":10078,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":228217,"spinlock_wait_cycles":146477312,"thread_start_us":160149,"threads_started":2,"wal-append.queue_time_us":166422}
W20260709 13:19:58.785744 10865 leader_election.cc:336] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.731699 17628 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:58.735392 17366 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:58.734809 17638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.019s	sys 0.033s
I20260709 13:19:58.814980 10124 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:58.818017 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:58.718380 17655 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.822849 10054 raft_consensus.cc:2410] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:19:58.815448 10865 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.824321 10131 raft_consensus.cc:2468] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:19:58.816246 17492 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.827164 17672 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.806352 17613 raft_consensus.cc:697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:58.774760 17700 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:58.829368 10122 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:58.798836 17651 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:58.831835 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46140.9us
I20260709 13:19:58.816256 17738 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.812165 17222 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.832566 17537 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:19:58.835952 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 99015.6us
I20260709 13:19:58.857283 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.859297 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:58.816020 17328 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.842478 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.863997 10865 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.864758 10865 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:19:58.865379 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.846556 10130 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:19:58.843062 10096 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.545s (SENT)
I20260709 13:19:58.849085 17613 consensus_queue.cc:237] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:19:58.852720 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.853130 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.850718 17632 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.830883 17628 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:58.871469 17328 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:58.873049 17651 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:19:58.865958 10865 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:58.880645 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:58.874320 17222 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.886624 10894 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60866 (request call id 1158) took 1033 ms. Trace:
I20260709 13:19:58.886909 10892 raft_consensus.cc:1240] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:19:58.877959 17735 raft_consensus.cc:2804] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:19:58.891072 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48069.1us
W20260709 13:19:58.902112 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.581s (SENT)
I20260709 13:19:58.902305 10900 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:19:58.902654 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:58.878298 17672 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:58.897727 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:58.904891 10096 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.581s (SENT)
I20260709 13:19:58.897665 10894 rpcz_store.cc:276] 0709 13:19:57.852852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.853074 (+   222us) service_pool.cc:224] Handling call
0709 13:19:57.869326 (+ 16252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.049201 (+179875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.886267 (+837066us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001e7154. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:58.886536 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411137920}
I20260709 13:19:58.905514 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:58.912603 17328 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:58.914609 17655 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:58.856787 17738 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.920006 17492 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:58.930583 17393 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.931824 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3466) took 2274 ms. Trace:
I20260709 13:19:58.939282 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.965955 10130 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:19:58.903952 17735 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:58.967181 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.898487 17203 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.917840 17631 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:58.944698 17713 log.cc:927] Time spent T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.080s	user 0.001s	sys 0.002s
W20260709 13:19:58.946811 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 290648us
I20260709 13:19:58.947420 17628 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:58.949276 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.950310 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.976119 10196 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:58.977365 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.978668 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.951004 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.951643 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.979624 17632 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:58.980201 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.980901 10197 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
W20260709 13:19:58.953878 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49028.4us
I20260709 13:19:58.982187 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.982648 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 896) took 1790 ms. Trace:
I20260709 13:19:58.962522 17647 consensus_queue.cc:1059] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:19:58.983055 10125 rpcz_store.cc:276] 0709 13:19:57.192547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.608027 (+415480us) service_pool.cc:224] Handling call
0709 13:19:57.608599 (+   572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.621194 (+ 12595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.621260 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.644969 (+ 23709us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.739068 (+ 94099us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:19:57.739180 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.739296 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.895389 (+1156093us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:19:58.982279 (+ 86890us) raft_consensus.cc:1685] finished
0709 13:19:58.982304 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.982584 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3176,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":28026,"spinlock_wait_cycles":126318336,"thread_start_us":36766,"threads_started":1,"wal-append.queue_time_us":135165}
I20260709 13:19:58.974349 17672 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:19:58.959189 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.957518 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:58.964365 10051 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46802 (request call id 3484) took 1913 ms (client timeout 2028 ms). Trace:
I20260709 13:19:58.987843 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:19:58.987795 10051 rpcz_store.cc:269] 0709 13:19:57.043461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.206142 (+162681us) service_pool.cc:224] Handling call
0709 13:19:58.954931 (+1748789us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400135e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:58.956386 (+  1455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3127778048}
I20260709 13:19:58.964386 10126 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:58.909482 17651 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.939353 17653 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.965692 10127 rpcz_store.cc:276] 0709 13:19:56.657441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.714420 (+ 56979us) service_pool.cc:224] Handling call
0709 13:19:56.714897 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.046803 (+1331906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.931464 (+884661us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540023b4d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:58.931734 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3032910208}
I20260709 13:19:58.969890 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:58.995026 10131 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 24: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:58.995754 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:58.970894 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.971781 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.980814 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.998899 10198 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.952453 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:58.984365 17647 raft_consensus.cc:3055] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:58.985548 10196 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:19:58.986182 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.999941 10203 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.000285 10197 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:19:58.963325 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:58.968182 10130 raft_consensus.cc:2393] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:59.001483 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:19:59.002789 10130 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:58.975100 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:59.005335 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:58.982949 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.625s (SENT)
I20260709 13:19:59.006170 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:59.004969 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:59.009141 10131 raft_consensus.cc:1275] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:59.010519 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.019578 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:58.970016 17735 raft_consensus.cc:3060] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.021422 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.024097 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:59.025427 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.026572 10200 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:59.028355 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.029176 10863 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:59.029769 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:19:59.031661 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.032593 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.034435 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.036146 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.022455 17203 raft_consensus.cc:3060] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:59.006871 17647 raft_consensus.cc:740] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:59.024614 17632 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.032653 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.040593 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.041267 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.015995 17620 raft_consensus.cc:2804] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:19:59.008177 17631 raft_consensus.cc:2804] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.037555 17701 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:19:59.050225 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 168939us
I20260709 13:19:59.029516 17697 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:59.053920 10203 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:59.033466 17668 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:59.056135 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.045346 17464 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:59.058132 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.048844 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:59.112051 10130 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.037765 17490 raft_consensus.cc:2804] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:59.007818 17653 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.055187 17620 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:59.066022 17631 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.075276 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.130414 10200 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 26.
I20260709 13:19:59.076097 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:59.080029 10865 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.887s (SENT)
I20260709 13:19:59.131820 10865 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.080489 17731 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.082957 17397 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.042972 17696 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:19:59.045974 17619 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:59.086491 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:59.057397 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74496.5us
W20260709 13:19:59.071599 10170 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 60 similar messages]
I20260709 13:19:59.148772 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.150697 10131 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:19:59.113163 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:59.116273 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 58288.6us
I20260709 13:19:59.116807 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.157912 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:59.159968 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:59.160207 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73872.3us
I20260709 13:19:59.117641 10095 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:59.162247 10019 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
W20260709 13:19:59.115427 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 98698.2us
I20260709 13:19:59.110430 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.121775 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.053015 17632 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:59.129631 17620 raft_consensus.cc:3060] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:59.067545 17722 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:59.066857 17621 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:19:59.067072 17726 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:19:59.082031 17646 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
I20260709 13:19:59.072294 17697 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.131853 10196 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:19:59.110251 17641 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:19:59.051472 17655 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:19:59.165865 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:59.166386 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:59.167928 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 100517us
I20260709 13:19:59.171995 17632 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.135841 17631 raft_consensus.cc:3060] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.227396 10204 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:59.124832 17737 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:19:59.227860 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:19:59.229038 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.229332 10130 raft_consensus.cc:2468] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:19:59.169941 17732 raft_consensus.cc:2804] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:19:59.153191 17696 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.123147 17613 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:19:59.125309 17492 consensus_queue.cc:1059] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:19:59.159330 17464 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.183538 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:59.159044 17668 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:59.263491 17668 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.169023 17627 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:59.266927 10123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45970 (request call id 3871) took 2304 ms (client timeout 2137 ms). Trace:
W20260709 13:19:59.267488 10123 rpcz_store.cc:269] 0709 13:19:56.962889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.158706 (+195817us) service_pool.cc:224] Handling call
0709 13:19:59.266691 (+2107985us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54001c7354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:59.266831 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082890112}
I20260709 13:19:59.268972 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:59.124565 17622 consensus_queue.cc:1059] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:19:59.172094 17619 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:59.172726 17656 raft_consensus.cc:2764] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:19:59.125599 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.273836 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.274785 17464 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.276958 10198 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:59.277429 17464 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.278237 17464 raft_consensus.cc:3055] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:59.278633 17464 raft_consensus.cc:740] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.283826 17738 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.235754 17696 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.285034 17632 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.187429 17697 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:19:59.286511 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44850.9us
I20260709 13:19:59.148367 17397 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.242493 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:59.242954 17748 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:19:59.250355 10196 raft_consensus.cc:2468] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:19:59.293354 17397 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.294169 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.295243 10129 raft_consensus.cc:2393] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 19: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:19:59.296330 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:59.296977 10129 raft_consensus.cc:3060] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:59.186530 17646 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.299060 17646 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.300884 10052 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 3487) took 2229 ms. Trace:
I20260709 13:19:59.132220 17647 consensus_queue.cc:260] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.302386 10123 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:19:59.254783 10054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48538 (request call id 972) took 2100 ms (client timeout 1887 ms). Trace:
I20260709 13:19:59.256551 17653 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.258143 17731 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.230743 17722 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:59.258948 17641 raft_consensus.cc:2804] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:59.236207 17732 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.260738 17492 consensus_queue.cc:1243] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:59.261291 17726 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:59.319438 17641 raft_consensus.cc:697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.320749 17641 consensus_queue.cc:237] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.323202 17726 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.323915 17752 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:19:59.324307 17726 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.325431 17722 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:59.325831 17726 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:59.283665 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.326640 17722 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.327551 10203 raft_consensus.cc:3060] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:19:59.111524 17672 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:59.275969 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:59.235520 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.252224 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:19:59.302973 10054 rpcz_store.cc:269] 0709 13:19:57.154560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.194878 (+ 40318us) service_pool.cc:224] Handling call
0709 13:19:59.254462 (+2059584us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540024ac54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:59.254682 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3680327808}
I20260709 13:19:59.301355 10052 rpcz_store.cc:276] 0709 13:19:57.071087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.982244 (+911157us) service_pool.cc:224] Handling call
0709 13:19:58.000765 (+ 18521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.805677 (+804912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.300602 (+494925us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400086954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:59.300815 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839116672}
I20260709 13:19:59.343978 10203 raft_consensus.cc:2468] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
I20260709 13:19:59.278492 10198 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:19:59.324852 17697 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.270972 10892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 835) took 1176 ms. Trace:
I20260709 13:19:59.334928 10201 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
W20260709 13:19:59.346083 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 70235.3us
I20260709 13:19:59.373679 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:59.307734 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124281us
I20260709 13:19:59.351843 17651 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.352705 10204 raft_consensus.cc:2368] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
W20260709 13:19:59.353472 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 198144us
I20260709 13:19:59.334124 17668 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:19:59.360631 10170 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.366226 10123 raft_consensus.cc:2468] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:19:59.314886 17737 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.390822 17737 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.391403 17737 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.381692 17464 consensus_queue.cc:260] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.391820 17632 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.392666 17464 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:59.399222 17267 consensus_queue.cc:1059] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:59.400710 17267 raft_consensus.cc:3055] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:59.401208 17267 raft_consensus.cc:740] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:59.401620 17203 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.369091 17366 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.402697 17366 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:19:59.403184 17366 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.372491 17397 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.407934 10125 raft_consensus.cc:2410] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:19:59.407837 17397 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.394704 17631 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.410542 17397 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.411077 17631 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.402242 17267 consensus_queue.cc:260] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:59.374564 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 214652us
I20260709 13:19:59.421051 10020 leader_election.cc:400] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:59.421516 10020 leader_election.cc:403] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:19:59.377998 17697 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.379081 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.378069 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45470 (request call id 893) took 2205 ms. Trace:
I20260709 13:19:59.423532 10128 rpcz_store.cc:276] 0709 13:19:57.172867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.331620 (+158753us) service_pool.cc:224] Handling call
0709 13:19:59.377797 (+2046177us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b5400321c54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:59.377975 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870105856}
I20260709 13:19:59.425244 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.426743 10128 raft_consensus.cc:2410] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:19:59.372991 10892 rpcz_store.cc:276] 0709 13:19:58.094876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.103221 (+  8345us) service_pool.cc:224] Handling call
0709 13:19:58.886506 (+783285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.898824 (+ 12318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.270703 (+371879us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54001e7150. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:59.270886 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775131008}
I20260709 13:19:59.330888 17732 raft_consensus.cc:3060] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:59.357120 10129 raft_consensus.cc:2468] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 14.
I20260709 13:19:59.389153 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.399087 17735 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.434581 17735 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.437013 17622 raft_consensus.cc:3055] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:19:59.437608 17622 raft_consensus.cc:740] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.438186 17222 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:19:59.440194 10122 raft_consensus.cc:1240] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:59.441766 10122 raft_consensus.cc:1275] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:59.385556 17126 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:59.392675 17632 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:19:59.447707 17632 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.448145 17632 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:19:59.448474 17737 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:59.459731 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.454875 10168 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:59.458709 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 378698us
W20260709 13:19:59.463567 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 84594.4us
W20260709 13:19:59.473661 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 50761.5us
I20260709 13:19:59.473754 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:59.429400 17668 raft_consensus.cc:3060] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.464184 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:59.474591 10020 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:59.476042 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:19:59.429766 17267 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:59.390035 17701 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:19:59.493438 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48279.3us
I20260709 13:19:59.452538 17592 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:19:59.497601 10019 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.500245 10020 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:19:59.473145 17697 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:59.474010 17126 raft_consensus.cc:2804] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:59.450940 17737 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.438592 17622 consensus_queue.cc:260] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:59.508477 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:59.458645 17661 consensus_queue.cc:1059] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:19:59.492750 17631 raft_consensus.cc:2804] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:19:59.462774 17222 consensus_queue.cc:1048] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:19:59.498780 17711 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:59.501632 17397 raft_consensus.cc:2804] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:59.505704 17203 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:19:59.537225 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:59.538905 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.549160 10204 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:59.552924 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.558501 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.559522 10204 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.560091 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.565847 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:19:59.539999 17631 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.541746 17697 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:59.570295 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:19:59.542517 17711 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.571998 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:19:59.541939 17126 raft_consensus.cc:697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:59.545615 17735 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:59.573911 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:59.541671 17490 raft_consensus.cc:697] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:59.545213 17328 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:59.574687 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:59.545430 17397 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.582309 10899 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:19:59.578972 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 461411us
I20260709 13:19:59.581745 10892 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
W20260709 13:19:59.587774 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 421970us
I20260709 13:19:59.589005 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:59.599675 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:59.600813 10891 raft_consensus.cc:2393] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:19:59.601433 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:19:59.568172 17622 raft_consensus.cc:3060] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.563658 17203 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:59.555606 17760 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.577889 17631 raft_consensus.cc:3060] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.612660 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:19:59.607896 10865 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:59.590948 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:59.579129 17711 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.579084 17672 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:19:59.632558 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.632432 17490 consensus_queue.cc:237] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.583758 17737 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.600677 17397 raft_consensus.cc:3060] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.635591 17737 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.636771 17203 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.568945 17701 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.606076 17655 consensus_queue.cc:1059] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:19:59.621500 17651 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:59.621793 17697 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:59.649637 10900 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60878 (request call id 841) took 1033 ms. Trace:
I20260709 13:19:59.650205 10900 rpcz_store.cc:276] 0709 13:19:58.616004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.655074 (+ 39070us) service_pool.cc:224] Handling call
0709 13:19:58.655523 (+   449us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:58.903147 (+247624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.649224 (+746077us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54001e64d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:19:59.649548 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566350592}
I20260709 13:19:59.579281 17594 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:19:59.651499 17594 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.657133 17752 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:19:59.658170 17752 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:59.657582 17620 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.612597 17735 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:19:59.603991 17328 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.612340 17738 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.624985 10864 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:59.627609 17126 consensus_queue.cc:237] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.654770 17594 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.680341 17738 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.687992 17738 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.662314 17655 consensus_queue.cc:1243] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer d51ad94710c348889d417f7f00032948 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:19:59.673614 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48719.3us
I20260709 13:19:59.616448 10095 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:19:59.692833 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 76499.7us
I20260709 13:19:59.697104 10895 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 25. Current term is 26. Ops: []
I20260709 13:19:59.664563 17627 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:59.702602 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:59.703649 17735 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.705497 17651 raft_consensus.cc:3055] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:19:59.705972 17651 raft_consensus.cc:740] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.706640 17328 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.704222 10900 raft_consensus.cc:1240] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:59.659945 17661 consensus_queue.cc:1048] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:19:59.712239 10095 leader_election.cc:400] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:59.712831 10095 leader_election.cc:403] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:59.717041 10899 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:19:59.683889 17752 raft_consensus.cc:2804] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:19:59.727464 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:19:59.734006 10199 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2579) took 1794 ms (client timeout 2217 ms). Trace:
W20260709 13:19:59.734699 10199 rpcz_store.cc:269] 0709 13:19:57.930902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.034495 (+103593us) service_pool.cc:224] Handling call
0709 13:19:59.724818 (+1690323us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54001d0954. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:19:59.724981 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453262208}
I20260709 13:19:59.710175 17638 raft_consensus.cc:2804] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:19:59.752702 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.755702 10168 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:19:59.764798 10129 raft_consensus.cc:1275] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:59.735731 17731 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 13. Current term is 14.
I20260709 13:19:59.774379 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:19:59.775146 10899 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:59.777379 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.768384 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:59.778484 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:19:59.780581 10893 raft_consensus.cc:2393] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:59.716429 17464 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:19:59.746800 17752 raft_consensus.cc:697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:59.783493 17638 raft_consensus.cc:697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:19:59.785662 17655 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.765595 17732 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.786247 17764 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:19:59.706904 17651 consensus_queue.cc:260] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.764410 17628 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.790051 10129 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:19:59.792251 10204 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
W20260709 13:19:59.790306 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 455896us
I20260709 13:19:59.772665 17328 raft_consensus.cc:2804] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:59.773069 17737 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:19:59.750634 17583 consensus_queue.cc:1059] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:19:59.751574 17407 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 13:19:59.829012 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:19:59.833974 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.837697 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:59.771822 17632 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.773931 17592 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:19:59.876567 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:19:59.863432 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:19:59.822363 17668 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:19:59.868024 10863 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:19:59.878130 10129 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:19:59.789494 17366 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:59.882134 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:59.815460 17594 consensus_queue.cc:1059] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:19:59.896934 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:59.898010 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:19:59.919328 10894 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 27: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:19:59.869964 17762 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:59.881420 17641 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.876605 17328 raft_consensus.cc:697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:59.887334 17651 raft_consensus.cc:3060] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:19:59.892462 17346 raft_consensus.cc:3055] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:59.825807 17655 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.892037 17407 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:19:59.888832 17738 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.830765 17764 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.836534 17628 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:19:59.938680 10096 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:19:59.901486 17620 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:19:59.919596 17763 raft_consensus.cc:2804] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:19:59.955610 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:19:59.951871 17696 consensus_meta.cc:287] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.667s	user 0.014s	sys 0.002s
I20260709 13:19:59.937736 17464 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:19:59.962852 10897 raft_consensus.cc:1275] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:19:59.964440 10125 raft_consensus.cc:2468] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:19:59.972342 10129 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:19:59.937981 17346 raft_consensus.cc:740] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:19:59.942544 17704 raft_consensus.cc:3055] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:19:59.941197 17700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.017s	sys 0.016s
I20260709 13:19:59.944455 17407 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:19:59.993499 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 224453us
W20260709 13:19:59.993888 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.016013 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.993637 17766 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:00.007604 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:00.013145 17700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.018s	sys 0.016s
I20260709 13:19:59.938519 17641 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.993106 17726 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.007249 17763 raft_consensus.cc:697] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:19:59.920053 17752 consensus_queue.cc:237] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:00.006112 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 237710us
I20260709 13:19:59.958477 17738 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:19:59.938194 17638 consensus_queue.cc:237] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.007716 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.013271 17704 raft_consensus.cc:740] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:00.005374 17620 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.048317 17738 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.050985 17407 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.038738 17735 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:00.054494 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3486) took 1083 ms. Trace:
I20260709 13:20:00.085250 10123 rpcz_store.cc:276] 0709 13:19:58.970846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.469559 (+498713us) service_pool.cc:224] Handling call
0709 13:19:59.470148 (+   589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.495222 (+ 25074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.495276 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.502412 (+  7136us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.947862 (+445450us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:59.947939 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.948072 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.054181 (+106109us) raft_consensus.cc:1685] finished
0709 13:20:00.054212 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.054406 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25890,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":6783,"thread_start_us":411056,"threads_started":1,"wal-append.queue_time_us":532632}
I20260709 13:20:00.026742 17328 consensus_queue.cc:237] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:19:59.993599 17696 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.068986 17346 consensus_queue.cc:260] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:00.080047 17767 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:20:00.069533 17732 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.079850 17366 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:00.059983 17393 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.104923 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:00.108956 10123 raft_consensus.cc:2368] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
W20260709 13:20:00.113343 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.045100 17726 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.114156 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:00.126241 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
W20260709 13:20:00.128610 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.109943 17407 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:00.107306 17735 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:00.084255 17620 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.156037 17620 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.157548 17628 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.144369 17346 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:00.120884 17393 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:00.162573 17774 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:20:00.162073 17622 consensus_meta.cc:287] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.004s
I20260709 13:20:00.165655 10126 raft_consensus.cc:1275] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:20:00.166203 10126 pending_rounds.cc:85] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 10
I20260709 13:20:00.166538 10126 pending_rounds.cc:107] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:20:00.166849 10126 raft_consensus.cc:2889] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:00.128687 17464 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:20:00.143848 17267 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.007s
I20260709 13:20:00.135574 17632 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.097783 17330 consensus_queue.cc:1048] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:20:00.153841 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:00.149684 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102592us
I20260709 13:20:00.182683 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:00.199702 17780 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:20:00.201294 17613 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:20:00.201781 17397 consensus_meta.cc:287] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.002s
I20260709 13:20:00.201874 17613 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:00.202353 17397 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.172168 17656 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:00.247720 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65220.1us
I20260709 13:20:00.250986 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:00.208307 17779 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:20:00.267812 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.274765 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.146272 17704 consensus_queue.cc:260] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.185319 17622 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:00.207540 17738 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.206277 17492 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.306799 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.203909 17641 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.257968 17126 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:00.147246 17763 consensus_queue.cc:237] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:00.297969 17774 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:00.278528 17393 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:00.323653 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:00.342031 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.342343 10131 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:00.342798 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.338064 17788 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 13:20:00.339039 17786 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:20:00.346019 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3483) took 1377 ms. Trace:
I20260709 13:20:00.272966 17785 consensus_queue.cc:1048] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:20:00.202955 17613 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.308912 17697 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:00.320525 17704 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.352609 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3487) took 1362 ms. Trace:
I20260709 13:20:00.365758 10130 rpcz_store.cc:276] 0709 13:19:58.989945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.471685 (+481740us) service_pool.cc:224] Handling call
0709 13:19:59.472123 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.472382 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:19:59.472695 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.261913 (+789218us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b1800044fe0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba423a 00007fa27800ea2a
0709 13:20:00.352205 (+ 90292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.352268 (+    63us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:00.352332 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.352405 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.352548 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12303,"prepare.queue_time_us":9302,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":16571,"spinlock_wait_cycles":4001664,"thread_start_us":77645,"threads_started":1}
I20260709 13:20:00.369102 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3485) took 1399 ms. Trace:
I20260709 13:20:00.215999 17735 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.351444 17721 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.369899 10128 rpcz_store.cc:276] 0709 13:19:58.969370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.459771 (+490401us) service_pool.cc:224] Handling call
0709 13:19:59.460107 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.460500 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.460531 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.465175 (+  4644us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.791440 (+326265us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:19:59.824365 (+ 32925us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.859812 (+ 35447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.859912 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.368716 (+508804us) raft_consensus.cc:1685] finished
0709 13:20:00.368751 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.368998 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10860,"prepare.queue_time_us":32932,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":19872,"raft.queue_time_us":37167,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":28871,"spinlock_wait_cycles":11878144,"thread_start_us":261558,"threads_started":1,"wal-append.queue_time_us":475330}
W20260709 13:20:00.372625 17782 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:20:00.418664 17631 consensus_meta.cc:287] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.756s	user 0.013s	sys 0.000s
I20260709 13:20:00.419309 17631 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.422183 17631 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.367898 17697 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.427850 17764 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.432407 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.433660 10130 raft_consensus.cc:2376] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19. Re-sending same reply.
I20260709 13:20:00.434442 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.435927 10128 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:20:00.437656 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.439657 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.440634 17596 raft_consensus.cc:3055] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:00.441205 17596 raft_consensus.cc:740] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:00.441697 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3491) took 1428 ms. Trace:
I20260709 13:20:00.442142 10126 rpcz_store.cc:276] 0709 13:19:59.012951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.661346 (+648395us) service_pool.cc:224] Handling call
0709 13:19:59.661750 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.179391 (+517641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.441387 (+261996us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54001261d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:00.441618 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157632256}
I20260709 13:20:00.442811 17596 consensus_queue.cc:260] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:00.443990  9948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 123) took 26289 ms (client timeout 15000 ms). Trace:
I20260709 13:20:00.444700 17596 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:00.447572 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.351776 17783 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:00.388444 17726 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.391990 17397 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.347898 17793 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:20:00.446921 17787 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
W20260709 13:20:00.444482  9948 rpcz_store.cc:269] 0709 13:19:34.154841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.634916 (+1480075us) service_pool.cc:224] Handling call
0709 13:19:37.087187 (+1452271us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400109ad4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:20:00.443861 (+23356674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:55.556411 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:55.562337 (+  5926us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:55.578134 (+ 15797us) write_op.cc:435] Acquired schema lock
0709 13:19:55.579911 (+  1777us) tablet.cc:662] Decoding operations
0709 13:19:55.686229 (+106318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:55.686352 (+   123us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:55.686388 (+    36us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:19:55.688467 (+  2079us) tablet.cc:701] Row locks acquired
0709 13:19:55.688516 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:19:55.688745 (+   229us) write_op.cc:270] Start()
0709 13:19:55.688939 (+   194us) write_op.cc:276] Timestamp: P: 1783603195688661 usec, L: 0
0709 13:19:55.688980 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:19:55.809780 (+120800us) log.cc:844] Serialized 3566 byte log entry
0709 13:19:57.034182 (+1224402us) op_driver.cc:464] REPLICATION: finished
0709 13:19:57.343417 (+309235us) write_op.cc:301] APPLY: starting
0709 13:19:57.369157 (+ 25740us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:58.622741 (+1253584us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b1800346180. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa277d430ec 00007fa277d8fc0a 00007fa277d8f4f7 00007fa277d8f490 00007fa277d46f48 00007fa27ba5b1bb 00007fa27ba5b532 00007fa27baa2b93 00007fa27b94fde3
0709 13:19:58.680520 (+ 57779us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:58.680569 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:58.896056 (+215487us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:58.977430 (+ 81374us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 13:19:58.977499 (+    69us) write_op.cc:312] APPLY: finished
0709 13:19:58.977923 (+   424us) log.cc:844] Serialized 89 byte log entry
0709 13:19:59.304984 (+327061us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:59.335857 (+ 30873us) write_op.cc:454] Released schema lock
0709 13:19:59.368760 (+ 32903us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:59.390794 (+ 22034us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:59.390836 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5594851,"reported_tablets":22,"spinlock_wait_cycles":300340096,"child_traces":[["op",{"apply.queue_time_us":246411,"apply.run_cpu_time_us":119991,"apply.run_wall_time_us":2093866,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41675,"num_ops":9,"prepare.queue_time_us":104184,"prepare.run_cpu_time_us":32356,"prepare.run_wall_time_us":467636,"raft.queue_time_us":245625,"raft.run_cpu_time_us":31341,"raft.run_wall_time_us":145257,"replication_time_us":1344976,"spinlock_wait_cycles":6079232,"thread_start_us":725330,"threads_started":5,"wal-append.queue_time_us":857939,"wal-append.run_cpu_time_us":28811,"wal-append.run_wall_time_us":2706767}]]}
I20260709 13:20:00.313843 17738 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.462404 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:00.318576 17492 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.463768 17738 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.312834 17622 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.465301 17492 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.466013 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2601) took 1401 ms. Trace:
I20260709 13:20:00.466287 17622 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.467461 10123 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:20:00.468417 17697 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.469165 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.469918 10123 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:00.470410 10123 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:20:00.469159 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.475435 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.476366 10891 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
W20260709 13:20:00.477233 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.477759 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.478391 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.482049 10123 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:00.484417 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:00.486925 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:00.487509 17771 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.446s	user 0.002s	sys 0.000s
I20260709 13:20:00.488611 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.402994 17721 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.490705 17793 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.491215 17721 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:00.411761 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.491411 17793 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:00.492395 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.492280 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.493360 17738 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.494091 10019 leader_election.cc:400] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21.
I20260709 13:20:00.493942 17738 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:00.413089 17711 consensus_meta.cc:287] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.786s	user 0.014s	sys 0.000s
I20260709 13:20:00.489781 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:00.436012 17583 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:20:00.344229 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.440704 10128 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:20:00.479408 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:00.503592 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.353034 10122 rpcz_store.cc:276] 0709 13:19:58.968638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.442501 (+473863us) service_pool.cc:224] Handling call
0709 13:19:59.442856 (+   355us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:59.452933 (+ 10077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.452975 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:59.475830 (+ 22855us) log.cc:844] Serialized 78 byte log entry
0709 13:19:59.924746 (+448916us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:19:59.941619 (+ 16873us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.060820 (+119201us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b1800467740. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa27b9fc3c7 00007fa27800e9a7 00007fa27801e0bf 00007fa27801e038
0709 13:20:00.061144 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.068104 (+  6960us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.170542 (+102438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.243983 (+ 73441us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.345623 (+101640us) raft_consensus.cc:1685] finished
0709 13:20:00.345652 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.345917 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9359,"prepare.queue_time_us":17947,"raft.queue_time_us":21809,"raft.run_cpu_time_us":757,"raft.run_wall_time_us":42504,"spinlock_wait_cycles":20219136,"thread_start_us":511561,"threads_started":2,"wal-append.queue_time_us":697312}
I20260709 13:20:00.402882 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:00.455873 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 907882us
W20260709 13:20:00.485797 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.400161 17755 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.979s	user 0.001s	sys 0.004s
I20260709 13:20:00.449177 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:00.517948 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:00.518313 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3500) took 1239 ms. Trace:
I20260709 13:20:00.479873 10126 raft_consensus.cc:2393] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:00.518759 10129 rpcz_store.cc:276] 0709 13:19:59.278704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.517384 (+1238680us) service_pool.cc:224] Handling call
0709 13:20:00.517752 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.517965 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.517995 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.518048 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:00.518081 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.518140 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.518250 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.496882 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.500057 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.507587 10125 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.522149 10895 raft_consensus.cc:2410] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 22 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:20:00.466408 10205 rpcz_store.cc:276] 0709 13:19:59.064988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.277455 (+212467us) service_pool.cc:224] Handling call
0709 13:19:59.725091 (+447636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.725476 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.725504 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:59.725546 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:59.725583 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.725637 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.465807 (+740170us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54001d0950. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:00.465933 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1531,"raft.queue_time_us":445,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":16950,"spinlock_wait_cycles":1028418816}
I20260709 13:20:00.511855 10899 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:00.523743 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3499) took 1245 ms. Trace:
I20260709 13:20:00.524880 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.524718 10128 rpcz_store.cc:276] 0709 13:19:59.278365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.515185 (+1236820us) service_pool.cc:224] Handling call
0709 13:20:00.522931 (+  7746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.523220 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.523263 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.523329 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:00.523380 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.523469 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.523652 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:00.525754 10198 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:00.526302 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.537983 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.538848 10203 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:20:00.539767 10199 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:00.541574 10131 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:00.542018 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.398003 17613 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:00.550297 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.535733 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.563172 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.563486 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.560746 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:00.513108 17697 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.515206 17622 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:00.515678 17726 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.563606 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.512872 17633 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.501572 17583 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:00.570518 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.572353 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.565743 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:00.567869 10128 raft_consensus.cc:1275] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:20:00.586519 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:00.512501 17492 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:00.560006 17764 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.601106 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.528216 17651 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:00.540724 17731 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:00.581293 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:00.625771 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.535575 17397 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:00.573962 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51501.4us
I20260709 13:20:00.598574 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.599206 17793 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.632516 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:00.633805 10196 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:00.634104 17697 raft_consensus.cc:2804] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:00.635035 17697 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.602137 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.635511 17697 raft_consensus.cc:3060] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:00.598261 17761 consensus_queue.cc:1048] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:20:00.638463 17700 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:00.641901 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:00.510659 17711 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.649681 17668 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:00.650293 17668 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.650730 17668 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:00.651336 17711 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:00.652151 17711 consensus_queue.cc:1048] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:00.627054 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.642324 17700 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:00.654983 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.655794 10205 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:00.629434 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.657227 10197 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:20:00.659894 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:00.660734 10170 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.581965 17464 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:20:00.672780 17755 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:00.562258 17228 consensus_queue.cc:1048] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:20:00.672191 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.664788 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:00.605275 17492 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.590543 17726 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.623786 17731 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.676573 10122 raft_consensus.cc:1240] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:20:00.588814 17622 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:20:00.682627 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.687199 10198 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:20:00.689802 10197 raft_consensus.cc:2468] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 27.
W20260709 13:20:00.697917 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.671923 17793 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:00.582643 17222 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 13:20:00.594230 17633 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.695179 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:00.720248 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.625156 17764 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.687599 17464 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:00.718093 10170 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:00.700911 17397 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:00.628952 17651 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.717132 10864 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:00.729465 10170 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:00.734874 17330 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.735282 17620 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:20:00.735909 17799 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:00.739590 17622 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.742587 17738 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.732316 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:00.790565 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58352.6us
W20260709 13:20:00.743685 17346 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.584s	user 0.016s	sys 0.000s
I20260709 13:20:00.793769 17627 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:00.744879 17492 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.745456 17726 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.713716 17731 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.662374 17668 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.670286 10125 raft_consensus.cc:2468] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:00.809394 10125 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:00.734099 17397 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:00.772114 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54051us
W20260709 13:20:00.815237 10170 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:00.773312 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56281.2us
W20260709 13:20:00.817690 10170 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:00.722913 17407 consensus_meta.cc:287] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.570s	user 0.016s	sys 0.000s
I20260709 13:20:00.776775 17228 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:00.782230 10201 raft_consensus.cc:1275] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:20:00.729636 17631 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:00.804855 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3477) took 1839 ms. Trace:
I20260709 13:20:00.795054 17752 raft_consensus.cc:2804] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:20:00.811406 10122 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:00.832796 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.763481 17330 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.840036 10198 raft_consensus.cc:1240] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:00.733708 17464 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:00.830713 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.855962 10197 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:00.856590 10197 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:00.824527 10127 rpcz_store.cc:276] 0709 13:19:58.965464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.994022 (+ 28558us) service_pool.cc:224] Handling call
0709 13:19:58.994403 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.008039 (+ 13636us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:59.034238 (+ 26199us) log.cc:844] Serialized 14 byte log entry
0709 13:19:59.544014 (+509776us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.560684 (+ 16670us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.560864 (+   180us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:59.560900 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.560973 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.399465 (+838492us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:00.780273 (+380808us) raft_consensus.cc:1685] finished
0709 13:20:00.804467 (+ 24194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.804736 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25243,"prepare.queue_time_us":27141,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":2428,"raft.queue_time_us":87789,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":14405,"spinlock_wait_cycles":8483200,"thread_start_us":474471,"threads_started":2,"wal-append.queue_time_us":285277}
I20260709 13:20:00.804061 17628 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:00.822628 17397 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:00.875739 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.876895 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.879729 10197 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
W20260709 13:20:00.880620 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.881552 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.881862 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.882989 10197 raft_consensus.cc:2393] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 27: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:00.884177 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.874298 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.931372 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.931996 17764 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.932668 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:00.933360 10197 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:20:00.934458 10198 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:00.934754 17651 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.935248 17655 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:00.936658 17633 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:00.824262 17631 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.884178 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:00.690791 17711 consensus_queue.cc:1048] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.172s
I20260709 13:20:00.889897 10900 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:00.883576 17735 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:00.895715 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:00.900120 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.904227 17632 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.908118 17668 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:00.872558 17628 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:00.925052 10171 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.951596 10202 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 26.
I20260709 13:20:00.922917 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.931295 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:00.941605 10894 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:00.864277 17620 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.944993 17631 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:00.955199 17668 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:00.946563 17735 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:00.948714 17632 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:00.958339 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3504) took 1262 ms. Trace:
I20260709 13:20:00.965947 10203 raft_consensus.cc:2393] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:20:00.940773 17754 log.cc:927] Time spent T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 1.355s	user 0.000s	sys 0.013s
I20260709 13:20:01.014957 10123 rpcz_store.cc:276] 0709 13:19:59.695561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.559550 (+863989us) service_pool.cc:224] Handling call
0709 13:20:00.572599 (+ 13049us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.574330 (+  1731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.574373 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:00.584255 (+  9882us) log.cc:844] Serialized 26 byte log entry
0709 13:20:00.820589 (+236334us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:00.820637 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.820723 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.957984 (+137261us) raft_consensus.cc:1685] finished
0709 13:20:00.958017 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.958245 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1270,"raft.queue_time_us":2201,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":11793,"thread_start_us":137986,"threads_started":1,"wal-append.queue_time_us":288697}
W20260709 13:20:00.980422 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.020370 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.020510 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:01.021274 10123 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:00.958380 17696 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:01.046041 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.973539 17203 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.014039 17655 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.015302 17651 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.061512 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:01.062803 10125 raft_consensus.cc:2410] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:20:01.069036 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:01.009888 17620 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.076949 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:01.016839 17668 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:01.080312 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 206402us
I20260709 13:20:01.082597 17752 raft_consensus.cc:697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:01.040938 17731 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.065784 17696 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.100224 10200 raft_consensus.cc:1275] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:20:01.080189 17704 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.710s	user 0.014s	sys 0.000s
I20260709 13:20:01.102469 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.071926 17655 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.111006 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3475) took 2146 ms. Trace:
I20260709 13:20:01.112710 17655 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.111615 10124 rpcz_store.cc:276] 0709 13:19:58.964722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.986562 (+ 21840us) service_pool.cc:224] Handling call
0709 13:19:58.986893 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.987367 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.987401 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.007289 (+ 19888us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.333523 (+326234us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:59.333598 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.333704 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.681559 (+1347855us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:01.110601 (+429042us) raft_consensus.cc:1685] finished
0709 13:20:01.110639 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.110918 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30185,"raft.queue_time_us":1557,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":13775,"spinlock_wait_cycles":4238592,"thread_start_us":278578,"threads_started":1,"wal-append.queue_time_us":504178}
I20260709 13:20:01.059923 17203 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.115770 17203 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.118577 17808 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.119350 17808 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.120013 17817 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.120797 17817 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.120831 17632 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.123153 17752 consensus_queue.cc:237] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.075281 17651 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.126314 17651 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.105098 17696 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.130698 17696 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.131336 17638 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.141032 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:01.141577 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 322461us
W20260709 13:20:01.142210 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:01.144969 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.142659 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.155359 10123 raft_consensus.cc:2468] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:01.116513 17203 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:01.142990 17697 consensus_meta.cc:287] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.006s
I20260709 13:20:01.160719 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 918) took 1030 ms. Trace:
I20260709 13:20:01.161769 10125 rpcz_store.cc:276] 0709 13:20:00.130081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.064391 (+934310us) service_pool.cc:224] Handling call
0709 13:20:01.066732 (+  2341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.067044 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:01.067549 (+   505us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.160153 (+ 92604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.160238 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:01.160312 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.160427 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.160621 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":148,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":168,"thread_start_us":92104,"threads_started":1}
I20260709 13:20:01.122488 17808 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.161042 17203 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.172479 17647 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:01.172755 17203 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.170185 17620 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.176523 10125 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:01.177593 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3502) took 1630 ms. Trace:
I20260709 13:20:01.178146 10129 rpcz_store.cc:276] 0709 13:19:59.547058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.536753 (+989695us) service_pool.cc:224] Handling call
0709 13:20:00.537162 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.723634 (+186472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.723671 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.723719 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:00.723753 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.177299 (+453546us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54000a0854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:01.177359 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.177522 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428327680}
I20260709 13:20:01.180161 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3933) took 1375 ms. Trace:
I20260709 13:20:01.182924 10122 rpcz_store.cc:276] 0709 13:19:59.804759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.723839 (+919080us) service_pool.cc:224] Handling call
0709 13:20:00.811196 (+ 87357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.832220 (+ 21024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.179909 (+347689us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b540023b9d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:01.180071 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159376128}
I20260709 13:20:01.154641 17661 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.157692 17812 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.153951 17817 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.155050 17696 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.207971 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:01.209794 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.211509 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:01.147444 17638 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.210213 17655 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.218189 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 917) took 1116 ms. Trace:
W20260709 13:20:01.223943 17800 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:20:01.227597 17668 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.185078 17811 consensus_queue.cc:1048] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:20:01.229470 17731 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.230247 17735 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.213236 17819 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:01.211210 17393 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.267980 17814 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.236784 10123 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:20:01.180416 10125 raft_consensus.cc:1240] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:01.209014 17620 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.165478 17697 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:01.256596 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 154237us
I20260709 13:20:01.253429 17700 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:20:01.284107 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.217093 17752 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.250595 17628 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.268594 17393 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.256162 17655 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:01.299347 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76967.2us
I20260709 13:20:01.271529 17813 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.272817 17638 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.233383 17812 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.280884 10125 raft_consensus.cc:1275] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:20:01.333545 10125 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:20:01.285619 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.266918 17397 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.293713 17819 consensus_queue.cc:1059] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:20:01.294433 17735 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.252851 17696 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.296020 17700 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:01.297165 17393 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:01.300491 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.259598 17464 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.363595 17819 consensus_queue.cc:1243] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer db052bef7536453fa2d7f5a00e314a18 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:20:01.363934 10124 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:01.364548 17819 raft_consensus.cc:3055] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:01.364989 17819 raft_consensus.cc:740] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:01.365677 17696 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.365510 17464 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.366101 17819 consensus_queue.cc:260] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:01.366724 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66309.3us
I20260709 13:20:01.287240 17814 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.289166 17620 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.300168 17752 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:01.345762 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 60230.8us
I20260709 13:20:01.404737 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.367193 17700 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:01.402884 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.407868 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:01.429399 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61830.6us
I20260709 13:20:01.421026 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:01.425827 10202 raft_consensus.cc:1275] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:20:01.430279 10170 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:01.431185 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.431922 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.355665 17628 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:01.439286 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.446501 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:01.441731 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:01.455492 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.450119 17668 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:01.452548 17819 raft_consensus.cc:3060] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:01.478645 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.442041 17366 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.506453 17366 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.485841 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.509914 17696 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:01.336714 17813 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.433241 17631 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.472723 17622 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.513499 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 257606us
I20260709 13:20:01.515987 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3513) took 1006 ms. Trace:
I20260709 13:20:01.517479 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:01.462829 17823 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:20:01.531440 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.533602 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:01.527622 10123 rpcz_store.cc:276] 0709 13:20:00.509241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.301905 (+792664us) service_pool.cc:224] Handling call
0709 13:20:01.356922 (+ 55017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.357269 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:01.357747 (+   478us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.515586 (+157839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.515648 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:01.515698 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.515775 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.515908 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":206,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":1112,"spinlock_wait_cycles":7936,"thread_start_us":156215,"threads_started":1}
I20260709 13:20:01.544387 10129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3510) took 1204 ms. Trace:
I20260709 13:20:01.510861 17696 consensus_queue.cc:1059] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:20:01.546093 10129 rpcz_store.cc:276] 0709 13:20:00.340012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.181961 (+841949us) service_pool.cc:224] Handling call
0709 13:20:01.182360 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.182595 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:01.212654 (+ 30059us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.543908 (+331254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.543974 (+    66us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:01.544044 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.544136 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.544309 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35258,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":330397,"threads_started":1}
I20260709 13:20:01.551388 17697 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:01.558128 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.566278 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:01.567154 10891 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
I20260709 13:20:01.544853 17722 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:01.579298 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.583832 17812 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.572398 17397 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.536888 17622 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.690690 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:01.693334 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:01.704282 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:01.749374 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:01.750114 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:01.751273 10123 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.596846 17697 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:01.669678 17812 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:01.733531 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.741024 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:01.765209 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.765573 10891 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:20:01.588752 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 55241us
I20260709 13:20:01.748023 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:01.769191 10892 raft_consensus.cc:3060] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:01.757628 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.717335 17632 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.737591 17731 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.793411 10205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2607) took 1269 ms. Trace:
I20260709 13:20:01.794896 10205 rpcz_store.cc:276] 0709 13:20:00.524010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.524826 (+   816us) service_pool.cc:224] Handling call
0709 13:20:01.793057 (+1268231us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b54003548d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:01.793306 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159882624}
I20260709 13:20:01.796494 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:01.754050 17821 log.cc:927] Time spent T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.331s	user 0.000s	sys 0.002s
I20260709 13:20:01.800132 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3949) took 1484 ms. Trace:
I20260709 13:20:01.801045 10124 rpcz_store.cc:276] 0709 13:20:00.315897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.121938 (+806041us) service_pool.cc:224] Handling call
0709 13:20:01.122290 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.364520 (+242230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.799725 (+435205us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002d0d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:01.800043 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555137792}
I20260709 13:20:01.803010 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:01.805814 17824 log.cc:927] Time spent T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:20:01.807230 17810 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.810165 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:01.810910 10197 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:01.813689 17731 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.815891 17464 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.816468 17731 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.816874 17752 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.779891 17647 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.817648 17647 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.820168 17647 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:01.785804 17722 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:20:01.820940 17647 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.821437 17647 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:01.822340 17631 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.780476 17808 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.823611 17808 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.825091 17823 raft_consensus.cc:3055] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:01.825776 17823 raft_consensus.cc:740] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:01.826107 17808 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.828706 10124 raft_consensus.cc:2393] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:20:01.829216 17613 consensus_meta.cc:287] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 1.264s	user 0.015s	sys 0.002s
I20260709 13:20:01.830660 10130 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 17.
I20260709 13:20:01.792088 17813 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.832693 10124 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:01.781266 17812 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.834018 10863 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:01.834431 10124 raft_consensus.cc:1275] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:01.828526 17366 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:01.764101 17638 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:01.755929 17620 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.780828 17397 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:20:01.835968 17638 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.836169 17813 consensus_queue.cc:1048] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:01.836585 17638 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:01.845472 10124 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:20:01.848901 10124 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:20:01.858673 10124 pending_rounds.cc:85] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 9
I20260709 13:20:01.861567 10124 pending_rounds.cc:107] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:20:01.864012 10124 raft_consensus.cc:2889] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:01.856156 10197 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:01.865512 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.868026 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.869831 10197 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 17.
I20260709 13:20:01.873504 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:01.867758 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.899410 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.901124 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:01.848846 17697 raft_consensus.cc:3060] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:01.800280 17632 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:01.893009 10122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.038s	sys 0.085s
W20260709 13:20:01.877375 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 80943.1us
I20260709 13:20:01.894690 17752 consensus_queue.cc:1048] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:20:01.925179 17834 log.cc:927] Time spent T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:01.816545 17464 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.898732 17397 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.906903 17808 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:01.929905 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:01.879221 17622 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.918906 10892 raft_consensus.cc:2468] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:20:01.923375 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:01.933974 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:01.896013 17814 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.891642 17813 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
W20260709 13:20:01.949071 10122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.038s	sys 0.085s
I20260709 13:20:01.905006 17731 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.955299 10892 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:01.934785 10125 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
W20260709 13:20:01.934823 17627 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.011s	sys 0.011s
I20260709 13:20:01.963478 10095 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:01.962311 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:01.928797 17397 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:01.928997 17464 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:01.943739 17628 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:01.944113 17647 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:01.954825 17622 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.929023 17808 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:01.966549 17627 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.013s	sys 0.011s
I20260709 13:20:01.969434 17829 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:02.019852 17613 raft_consensus.cc:2804] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:02.043185 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:01.960584 17731 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:02.054390 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.032951 17829 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.030064 17627 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:20:02.065059 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.051836 10168 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.021960 17464 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:20:02.101600 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 128151us
I20260709 13:20:02.110558 17647 raft_consensus.cc:2804] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:02.110932 17841 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.112417 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3512) took 1771 ms. Trace:
I20260709 13:20:02.113741 10128 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:20:02.112887 10130 rpcz_store.cc:276] 0709 13:20:00.340602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.488248 (+147646us) service_pool.cc:224] Handling call
0709 13:20:02.112139 (+1623891us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003d4354. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:02.112333 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3084543744}
W20260709 13:20:02.115437 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.054239 17126 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:20:02.125494 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 350789us
I20260709 13:20:01.971935 10123 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:02.024839 17622 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:02.139050 17833 log.cc:927] Time spent T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:20:02.109179 17812 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.137326 10169 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:02.144119 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.064429 17829 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:02.137213 10168 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.151821 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:02.152642 10892 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:02.151185 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.134873 17126 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.173072 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:02.174199 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 140183us
I20260709 13:20:02.174995 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:02.180392 10898 raft_consensus.cc:2393] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:02.179234 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:02.137247 17647 raft_consensus.cc:697] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:02.149086 17668 consensus_meta.cc:287] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.659s	user 0.011s	sys 0.010s
I20260709 13:20:02.139079 17700 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.148727 17752 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.179104 17830 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.286s	user 0.000s	sys 0.002s
I20260709 13:20:02.181805 17126 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:02.183532 17628 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.111717 17841 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.183357 17632 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:01.807951 17810 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.184099 17661 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:02.183831 17843 raft_consensus.cc:2804] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:02.188659 17668 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:01.827011 17823 consensus_queue.cc:260] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.150988 17627 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.230654 10892 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:02.204072 17393 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
W20260709 13:20:02.233268 17828 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:20:02.204540 17700 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.207191 17752 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.237751 17397 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.249657 17763 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:02.216181 17628 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.243474 17661 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:02.280459 17802 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.428s	user 0.007s	sys 0.018s
I20260709 13:20:02.233270 17843 raft_consensus.cc:697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:02.310775 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 131663us
I20260709 13:20:02.280012 17668 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.318742 10865 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.332882 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.248988 17613 raft_consensus.cc:697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:01.244961 10127 rpcz_store.cc:276] 0709 13:20:00.101523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.859046 (+757523us) service_pool.cc:224] Handling call
0709 13:20:00.859662 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.859964 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:20:01.151042 (+291078us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.217797 (+ 66755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.217863 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:01.217915 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.217998 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.218128 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13173,"prepare.queue_time_us":39312,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":103,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":12574,"thread_start_us":321278,"threads_started":2}
I20260709 13:20:02.321427 10891 raft_consensus.cc:2468] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 25.
I20260709 13:20:02.284098 17823 raft_consensus.cc:3060] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:02.325373 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2609) took 1789 ms. Trace:
I20260709 13:20:02.345355 17847 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.345924 17838 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:20:02.346165 17847 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.342880 10204 rpcz_store.cc:276] 0709 13:20:00.536133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.536344 (+   211us) service_pool.cc:224] Handling call
0709 13:20:00.536913 (+   569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:00.559958 (+ 23045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.560000 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:01.244739 (+684739us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:20:01.245417 (+   678us) log.cc:844] Serialized 52 byte log entry
0709 13:20:01.476823 (+231406us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:01.476868 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.514378 (+ 37510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.312647 (+798269us) raft_consensus.cc:1685] finished
0709 13:20:02.312672 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.325245 (+ 12573us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22404,"raft.queue_time_us":22534,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6022,"spinlock_wait_cycles":195602688,"thread_start_us":191654,"threads_started":1,"wal-append.queue_time_us":514035}
W20260709 13:20:02.348537 17831 log.cc:927] Time spent T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 13:20:02.351261 17845 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:20:02.354679 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3846) took 1806 ms. Trace:
I20260709 13:20:02.355363 10196 rpcz_store.cc:276] 0709 13:20:00.548394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.639830 (+ 91436us) service_pool.cc:224] Handling call
0709 13:20:00.640294 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.640789 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.640824 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.845391 (+1204567us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b2c0011db58. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa2768fda11 00007fa27690d372 00007fa27803fa01 00007fa27800443a 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2
0709 13:20:01.845891 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.108571 (+262680us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:02.108626 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.108722 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.354195 (+245473us) raft_consensus.cc:1685] finished
0709 13:20:02.354230 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.354579 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":118,"spinlock_wait_cycles":11019008,"thread_start_us":248322,"threads_started":1,"wal-append.queue_time_us":358519}
W20260709 13:20:02.389405 17826 log.cc:927] Time spent T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:20:02.390774 17843 consensus_queue.cc:237] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:02.396133 17825 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.436s	user 0.002s	sys 0.008s
I20260709 13:20:02.398346 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3511) took 2058 ms. Trace:
I20260709 13:20:02.398854 10122 rpcz_store.cc:276] 0709 13:20:00.340304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.184011 (+843707us) service_pool.cc:224] Handling call
0709 13:20:01.212049 (+ 28038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.299442 (+ 87393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.299477 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.300075 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.952255 (+652180us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:01.952637 (+   382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.397871 (+445234us) spinlock_profiling.cc:243] Waited 86.9 ms on lock 0x7b5400126bd4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:02.397934 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.397999 (+    65us) raft_consensus.cc:1685] finished
0709 13:20:02.398010 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.398263 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5164,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":32869,"spinlock_wait_cycles":199755136,"thread_start_us":650180,"threads_started":1,"wal-append.queue_time_us":754779}
I20260709 13:20:02.404832 17628 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:02.407933 17819 consensus_meta.cc:287] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.889s	user 0.015s	sys 0.000s
I20260709 13:20:02.329636 17330 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:02.430029 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:02.456651 17850 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:02.457675 17850 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.420593 17813 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.410151 17848 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.292935 17647 consensus_queue.cc:237] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.393059  9946 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:02.407586 17668 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.438473 17808 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:02.435597 17464 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:02.506232 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.509557 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.452422 17622 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:02.511740 10203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50726 (request call id 984) took 1413 ms. Trace:
I20260709 13:20:02.512532 10203 rpcz_store.cc:276] 0709 13:20:01.098293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.098499 (+   206us) service_pool.cc:224] Handling call
0709 13:20:01.099033 (+   534us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:01.102660 (+  3627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.102708 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:01.108939 (+  6231us) log.cc:844] Serialized 52 byte log entry
0709 13:20:01.457396 (+348457us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:01.548077 (+ 90681us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.548561 (+   484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.548825 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.511341 (+962516us) raft_consensus.cc:1685] finished
0709 13:20:02.511381 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.511630 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3553,"prepare.queue_time_us":248942,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":934,"raft.queue_time_us":4353,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":31944,"thread_start_us":397113,"threads_started":2,"wal-append.queue_time_us":892733}
W20260709 13:20:02.528815 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 339700us
I20260709 13:20:02.475471 17704 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.545292 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.546060 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 181042us
W20260709 13:20:02.545974 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:02.557111 10095 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.562773 10900 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.570879 17622 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.477253 17813 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:02.575809 10170 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:02.576383 17858 consensus_queue.cc:1048] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:02.578872 17808 consensus_queue.cc:1048] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:02.571768 17622 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.581375 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:02.587019 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.589388 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.591060 10203 raft_consensus.cc:1240] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 22. Current term is 23. Ops: []
W20260709 13:20:02.592015 10022 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.592454 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.496376 17464 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:02.604624 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 74603us
I20260709 13:20:02.486104 17848 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.606436 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:02.608106 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.610256 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.603291 10095 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.614768 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.615406 10202 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:20:02.615577 10197 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:20:02.615960 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.616967 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.617791 10203 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:02.619073 10196 raft_consensus.cc:2393] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 24: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:02.583809 17855 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.621950 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.505872 17613 consensus_queue.cc:237] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.625927 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.561867 17704 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.605942 17841 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.632764 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:02.634037 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66729.5us
I20260709 13:20:02.608328 17783 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.633442 10892 raft_consensus.cc:2955] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 8 to 13, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:02.644862 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:02.646255 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.646433 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:02.649982 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 226125us
I20260709 13:20:02.629781 17397 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.629930 17812 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.620289 17852 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:02.629508 17627 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.613161 17330 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:20:02.673676 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:02.676863 10197 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:02.647708 17841 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:02.695721 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.695928 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.699744 10202 raft_consensus.cc:2410] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 20 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:20:02.696382 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:02.658903 17700 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:02.700526 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84662.7us
I20260709 13:20:02.702075 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.631441 17855 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:02.708036 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.698309 17841 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.712576 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.690569 17852 raft_consensus.cc:697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:02.662055 17842 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:02.739575 10196 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:02.666082 17397 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.652901 17783 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.740360 17856 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:02.676959 17812 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.748332 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:02.676506 17627 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.756100 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:02.757129 10123 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:02.746501 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.749351 10125 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 16.
I20260709 13:20:02.733451 17841 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:02.752357 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:02.752501 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:02.758817 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:02.766474 10123 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:20:02.770324 10123 raft_consensus.cc:1275] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:20:02.760021 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:02.771528 10125 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:20:02.741029 17628 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.784096 10124 raft_consensus.cc:1275] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:02.786422 10204 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:02.745262 17850 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:02.742925 17878 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:02.634923  9946 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:02.801745 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.806075 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.809789 10196 raft_consensus.cc:2393] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 26: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:02.810318 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.811561 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:02.793159 17863 log.cc:927] Time spent T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:20:02.812762 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.815202 10202 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:20:02.783612 17874 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.816699 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:02.791482 17638 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.953s	user 0.017s	sys 0.000s
I20260709 13:20:02.792347 17655 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.795504 17858 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.790534 17813 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.790459 17877 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:20:02.877947 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 167216us
I20260709 13:20:02.871474 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.876947 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:02.866653 17697 consensus_meta.cc:287] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.937s	user 0.017s	sys 0.000s
I20260709 13:20:02.883456 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:02.876561 17874 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:02.886772 17820 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 1.655s	user 0.009s	sys 0.016s
W20260709 13:20:02.886969 17860 log.cc:927] Time spent T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.204s	user 0.001s	sys 0.001s
I20260709 13:20:02.887316 17854 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:02.816350 17622 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.904295 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:02.792876 17879 consensus_queue.cc:1059] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:20:02.914561 17855 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:02.914933 17879 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:02.916795 17847 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:02.917078 17126 consensus_meta.cc:287] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.698s	user 0.014s	sys 0.000s
I20260709 13:20:02.917662 17126 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:02.920092 17835 log.cc:927] Time spent T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.993s	user 0.003s	sys 0.000s
I20260709 13:20:02.790834 17628 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.867424 17878 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.921799 17878 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.867853 17850 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:02.923774 10203 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:02.905057 10196 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:20:02.912951  9946 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:02.895483 17842 raft_consensus.cc:3060] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:02.893685 17697 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:02.947507 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3514) took 2438 ms. Trace:
I20260709 13:20:02.954255 10126 rpcz_store.cc:276] 0709 13:20:00.509554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.331077 (+821523us) service_pool.cc:224] Handling call
0709 13:20:01.331522 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.348074 (+ 16552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.348116 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.398006 (+ 49890us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.735328 (+337322us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:01.735382 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.735474 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.919315 (+1183841us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:02.923646 (+  4331us) raft_consensus.cc:1685] finished
0709 13:20:02.923669 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.947392 (+ 23723us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21229,"raft.queue_time_us":16769,"raft.run_cpu_time_us":1992,"raft.run_wall_time_us":50914,"spinlock_wait_cycles":6043008,"thread_start_us":273794,"threads_started":1,"wal-append.queue_time_us":497763}
I20260709 13:20:02.890534 17704 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:02.871515 17638 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:02.961357 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69086us
W20260709 13:20:02.964485 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46164us
I20260709 13:20:02.923530 17850 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.962802 10019 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:02.932125 17126 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:02.944233 17628 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.883369 17858 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.977214 10130 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
W20260709 13:20:02.972383 17872 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:20:02.985791 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:02.972471 17829 consensus_meta.cc:287] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.796s	user 0.009s	sys 0.005s
I20260709 13:20:02.907076 17877 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:20:02.986833 10124 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:20:02.989274 17885 raft_consensus.cc:3055] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:02.989032 17829 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:02.989845 17885 raft_consensus.cc:740] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:02.990916 17885 consensus_queue.cc:260] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:02.995986 10898 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:02.818711 17852 consensus_queue.cc:237] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.006062 17842 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:03.007517 17783 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:03.009783 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:03.002669 17883 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:20:02.995772 17878 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.012373 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:03.014724 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.046983 10126 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:20:03.030318  9946 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.035360 17812 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.035765 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:03.036546 17884 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:03.038887 17829 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:02.992403 17885 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:03.073594 17854 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:03.074513 17854 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:03.075670 10894 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 850) took 2024 ms (client timeout 2054 ms). Trace:
W20260709 13:20:03.076228 10894 rpcz_store.cc:269] 0709 13:20:01.050963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.052103 (+  1140us) service_pool.cc:224] Handling call
0709 13:20:03.075334 (+2023231us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540064f0d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:03.075548 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327530368}
W20260709 13:20:03.059791 17872 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:20:03.080317 17836 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.152s	user 0.001s	sys 0.003s
W20260709 13:20:03.081404 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.081832 10200 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57102 (request call id 2616) took 1675 ms. Trace:
I20260709 13:20:03.082448 10899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42822 (request call id 572) took 2397 ms. Trace:
I20260709 13:20:03.082944 17668 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.082321 10200 rpcz_store.cc:276] 0709 13:20:01.406891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.407085 (+   194us) service_pool.cc:224] Handling call
0709 13:20:01.433914 (+ 26829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.434283 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.434321 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.434985 (+   664us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.752283 (+317298us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:01.761934 (+  9651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.762023 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.079505 (+1317482us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:03.081446 (+  1941us) raft_consensus.cc:1685] finished
0709 13:20:03.081472 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.081752 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4228992,"thread_start_us":277801,"threads_started":1,"wal-append.queue_time_us":421760}
I20260709 13:20:03.086238 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.082957 10899 rpcz_store.cc:276] 0709 13:20:00.685518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.685708 (+   190us) service_pool.cc:224] Handling call
0709 13:20:00.709055 (+ 23347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:00.709641 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:20:01.014739 (+305098us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.095332 (+ 80593us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:01.096060 (+   728us) log.cc:844] Serialized 52 byte log entry
0709 13:20:01.096235 (+   175us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:20:01.104837 (+  8602us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.133817 (+ 28980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.133911 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:01.233598 (+ 99687us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b18004af920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:20:01.421496 (+187898us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18004af920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f17522 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:20:02.882108 (+1460612us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:03.082156 (+200048us) raft_consensus.cc:1685] finished
0709 13:20:03.082179 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.082334 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36925,"prepare.queue_time_us":532242,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":102,"raft.queue_time_us":83737,"raft.run_cpu_time_us":469,"raft.run_wall_time_us":20788,"spinlock_wait_cycles":46495104,"thread_start_us":326842,"threads_started":2,"wal-append.queue_time_us":125925}
I20260709 13:20:03.089283 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:03.090430 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:02.998834 17858 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.054484 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.054533 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:03.100271 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45874.2us
I20260709 13:20:02.994585 17628 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:03.059752 17841 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.102955 17840 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:03.099509 17697 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:03.108270 17840 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.108845 17840 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:03.113509  9946 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.114506 17731 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.116192 17810 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.116451 17850 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.060863 17638 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.057261 17884 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.062098 10197 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:20:03.062124 17887 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:20:03.062806 17397 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.093266 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:03.097504 17848 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.097357 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.098037 17668 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:03.101811 10122 raft_consensus.cc:3060] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:03.108470 17697 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:03.108470 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.163838 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.164870 10202 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:20:03.165934 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.060614 17752 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.129292 17874 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.130362 10197 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:03.141978 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 48820.8us
I20260709 13:20:03.169553 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:03.132787 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:03.160949 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.168159 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.175439 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.178987 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:03.176752 10202 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:03.183738 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.185180 10197 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.186102 17841 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.168454 17633 consensus_queue.cc:1048] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:20:03.168644 17668 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:03.188501 17668 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:03.187764  9946 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.189824  9946 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 13, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.134218 17628 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:03.168291 17697 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:03.132202 17884 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:03.193985 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:03.199708 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.160403 17879 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:03.208822 10122 raft_consensus.cc:2468] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:03.183218 17885 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:03.191975 17704 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:03.182987 17738 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:03.195199 17865 consensus_queue.cc:1048] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:20:03.199436 17812 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.228303 17812 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.209547 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:03.232434 10199 raft_consensus.cc:1275] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:20:03.232685 17763 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:03.233170 17763 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:03.235817  9946 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:03.237079 17823 consensus_meta.cc:287] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.829s	user 0.016s	sys 0.000s
I20260709 13:20:03.203553 17879 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:03.237982 17879 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:03.240566 17464 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.204545 17783 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:03.244401 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:03.246538 17738 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:03.247053 17738 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.247527 17738 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:03.249730 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3929) took 3446 ms. Trace:
I20260709 13:20:03.251412 17735 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:03.251329 17628 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.216732 17647 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:03.256953 17647 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.257371 17647 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:03.261456 10891 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60878 (request call id 853) took 1120 ms. Trace:
I20260709 13:20:03.262005 10891 rpcz_store.cc:276] 0709 13:20:02.140704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.143465 (+  2761us) service_pool.cc:224] Handling call
0709 13:20:03.261193 (+1117728us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540064e1d4. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:03.261372 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378381696}
I20260709 13:20:03.212235 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:03.263693 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 51562.1us
I20260709 13:20:03.217247 17704 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.192111 17397 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:20:03.271677 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.252043 17735 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:03.212921 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:03.276481 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 63647.5us
I20260709 13:20:03.265648 10203 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:20:03.277622 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.004s	sys 0.015s
I20260709 13:20:03.250298 10128 rpcz_store.cc:276] 0709 13:19:59.803576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.601865 (+798289us) service_pool.cc:224] Handling call
0709 13:20:02.113319 (+1511454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:02.114138 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.249429 (+1135291us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003d4350. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780038ad 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:03.249630 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3475010432}
W20260709 13:20:03.267972 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.278614 10096 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.273801 10168 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:03.194734 17126 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.278151 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.004s	sys 0.015s
I20260709 13:20:03.283077 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.284701 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.285323 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.285625 10123 raft_consensus.cc:2410] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:20:03.286334 10124 raft_consensus.cc:2393] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 21: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
W20260709 13:20:03.286471 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.286846 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.289278 10123 raft_consensus.cc:2393] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:03.290287 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.292269 10123 raft_consensus.cc:2393] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:03.292549 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.230954 17812 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.295660 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:03.296988 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.238314  9946 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.272455 17704 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:03.272718 17875 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:03.300952 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.294411 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.303429 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.421993 10124 raft_consensus.cc:2393] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:03.286399 17735 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:03.285351 17885 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.311291 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.352919 17852 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:03.324594 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.290717 17346 raft_consensus.cc:2804] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:03.308480 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.445626 10203 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:20:03.341710 17876 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:03.339486 17878 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:20:03.308496 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.322625 17875 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.293967 17126 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.390811 17783 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:03.382882 17874 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.391958  9946 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:03.353303 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.399507 17583 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.401437 10131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 926) took 1498 ms. Trace:
I20260709 13:20:03.402105 17840 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:03.403232 10170 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:03.406628 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.411067 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:03.411952 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:03.348863 17854 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.324797 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:03.441862 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.442668 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.450443 10125 raft_consensus.cc:2468] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:20:03.422027 17721 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 13:20:03.451756 17885 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.441226 17823 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.459525 17852 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.492441 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:03.497318 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 858959us
I20260709 13:20:03.498287 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:03.487576 10097 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.501173 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.497737 10130 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:20:03.520268 10129 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:03.521309 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:03.522766 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 169508us
I20260709 13:20:03.523460 10125 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:20:03.523597 10199 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:20:03.522846 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.525616 10130 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:03.496934 10131 rpcz_store.cc:276] 0709 13:20:01.902796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.953065 (+ 50269us) service_pool.cc:224] Handling call
0709 13:20:01.953392 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.408986 (+455594us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:20:02.530725 (+121739us) log.cc:844] Serialized 14 byte log entry
0709 13:20:02.646146 (+115421us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.661055 (+ 14909us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.661330 (+   275us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:02.661388 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.400917 (+739529us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001c6e54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:03.400982 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.401037 (+    55us) raft_consensus.cc:1685] finished
0709 13:20:03.401144 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.401349 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181901,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":7737,"raft.queue_time_us":204180,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":24627,"spinlock_wait_cycles":464879872,"thread_start_us":403970,"threads_started":3,"wal-append.queue_time_us":234403}
W20260709 13:20:03.524027 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.501168 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.532131 10124 raft_consensus.cc:2410] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:20:03.530666 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:03.473522 17783 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:03.417752 17763 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.502148 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:03.535211 10170 leader_election.cc:336] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.537585 10096 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.462366 17738 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.551955 17877 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.600534 17877 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.601722 17840 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:03.551894 17464 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:03.546406 10170 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:03.472005 17647 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:03.538235 17843 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.006s	sys 0.021s
W20260709 13:20:03.615021 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 79890.1us
W20260709 13:20:03.616132 10170 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.617238 10170 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.618294 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:03.618621 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:03.618777 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 330813us
I20260709 13:20:03.553339 10124 pending_rounds.cc:85] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 11
I20260709 13:20:03.597092 10129 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:03.597597 10128 raft_consensus.cc:2410] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:20:03.496764 17346 raft_consensus.cc:697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:03.481379 17875 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:03.608573 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71050.7us
I20260709 13:20:03.534173 17721 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:20:03.590701 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.522583 17823 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:03.614986 17843 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.006s	sys 0.026s
I20260709 13:20:03.620114 17882 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:03.625979 17877 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.622488 10124 pending_rounds.cc:107] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:20:03.637487 10130 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 13:20:03.620910 10170 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.761575 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:03.762944 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 141929us
W20260709 13:20:03.765220 10170 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.768251 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:03.750299 10096 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.769443 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:03.769770 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:03.754185 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.756388 10124 raft_consensus.cc:2889] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:03.758085 10127 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3525) took 1006 ms. Trace:
I20260709 13:20:03.762290 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.758944 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:03.776930 10127 rpcz_store.cc:276] 0709 13:20:02.751885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.755287 (+  3402us) service_pool.cc:224] Handling call
0709 13:20:02.755855 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.756333 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.756378 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.776955 (+ 20577us) log.cc:844] Serialized 26 byte log entry
0709 13:20:03.259898 (+482943us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18004ba9c0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274ef9e07 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b 00007fa277ba3d23 00007fa277ba40d1 00007fa277fbfa0f
0709 13:20:03.530835 (+270937us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:03.530886 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.530976 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.757075 (+226099us) raft_consensus.cc:1685] finished
0709 13:20:03.757138 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.757802 (+   664us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3038,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":15465,"spinlock_wait_cycles":7476608,"thread_start_us":270665,"threads_started":1,"wal-append.queue_time_us":896401}
I20260709 13:20:03.780974 10126 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:03.638410 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:03.744138  9946 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.783062 10129 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:03.631909 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:03.788733 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 161202us
W20260709 13:20:03.790050 10865 leader_election.cc:336] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.795619 10865 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:03.797058 10865 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.803468 17126 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.802850 17346 consensus_queue.cc:237] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:03.770543 10170 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.797554  9946 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.551800 17583 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:03.813156 10123 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:03.498548 17874 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.540534 17896 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:03.883965 10129 raft_consensus.cc:2468] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:03.884927 17896 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:03.886940 17647 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:03.898219 10127 raft_consensus.cc:1275] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:03.827088 17877 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.879330 17841 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.880618 17668 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.691s	user 0.013s	sys 0.001s
I20260709 13:20:03.645006 17721 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:03.672567 17879 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.904549 17738 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.709281 17884 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.667665 17763 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.694666 17852 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.699144 17885 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:03.673732 17897 log.cc:927] Time spent T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:20:03.917695 10170 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:03.894742 17874 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.826838 17854 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:03.927138 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:03.955646 17668 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.956305  9946 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 16, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 16 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:03.899833 17877 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.962180  9946 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.976493 17810 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:03.977720 17810 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:03.979101  9946 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 10, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 10 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:03.987465 17854 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:03.992355 17823 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:03.988024  9946 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.949985 17841 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:03.996354 17841 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.997225 17627 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.000538  9946 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.006242 17877 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:03.958323 17721 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.009903 17721 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:04.008129  9946 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 13, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 13 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:04.014704 17810 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.016940 10197 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:04.032048 17859 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:04.033560 17859 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.037564 17859 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:03.952790 17879 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:04.053808 17904 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.366s	user 0.004s	sys 0.000s
W20260709 13:20:04.043311 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 177031us
W20260709 13:20:04.060861 10170 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:04.062909 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 313113us
W20260709 13:20:04.048570 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.047292 17884 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:04.069952 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.075872 17925 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.001s
W20260709 13:20:04.076692 17927 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.003s	sys 0.000s
W20260709 13:20:04.095428 10019 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.507s (SENT) [suppressed 102 similar messages]
I20260709 13:20:04.101459 10196 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:04.108891 10895 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:04.110493 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:04.112735 17735 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:04.118583 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 54027.4us
I20260709 13:20:04.106491 17668 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:04.120798 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 292259us
I20260709 13:20:04.138715  9946 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.146863 17829 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:04.180495 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:04.184001 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 133756us
I20260709 13:20:04.194680  9946 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:04.197340 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 136563us
I20260709 13:20:04.242923 10197 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:20:04.247421 10197 pending_rounds.cc:85] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Aborting all ops after (but not including) 11
I20260709 13:20:04.245836 17735 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.271833 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:04.269179 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:04.276598 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 119537us
I20260709 13:20:04.300266 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.312374 10198 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:04.244784 17829 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.311080 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:04.298192 10197 pending_rounds.cc:107] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:20:04.326447 10197 raft_consensus.cc:2889] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:04.315517 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 220320us
I20260709 13:20:04.336772 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.337119 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:04.337990 10127 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 26.
I20260709 13:20:04.339921 17668 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.340812 17668 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.327641  9946 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.351475 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:04.352267 17870 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:20:04.352375 10127 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:04.352983  9946 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:04.354626 17668 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.355497 17876 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.356168 10197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3879) took 1676 ms. Trace:
I20260709 13:20:04.361462 10197 rpcz_store.cc:276] 0709 13:20:02.679679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.679913 (+   234us) service_pool.cc:224] Handling call
0709 13:20:02.680277 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.061960 (+381683us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b540011bfa0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f7bdc0 00007fa2780061c0 00007fa278007231 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c
0709 13:20:03.989575 (+927615us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b58004d99d0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa277f600aa 00007fa277fe2447 00007fa277ffafe7 00007fa277ffc7de 00007fa278006b49 00007fa278007416 00007fa278003d8f 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4
0709 13:20:04.351848 (+362273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.355907 (+  4059us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":790406400}
I20260709 13:20:04.356185 17876 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.357127 17885 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:04.378387 17668 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:04.378643 10127 raft_consensus.cc:2468] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:20:04.378901 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:04.379403 17668 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.387306 10201 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
W20260709 13:20:04.390008 10201 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2621) took 2943 ms (client timeout 1665 ms). Trace:
I20260709 13:20:04.391299 10124 raft_consensus.cc:3060] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:04.392172 17857 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.393064 17857 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.347946 17879 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.373875 17735 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:04.390599 10201 rpcz_store.cc:269] 0709 13:20:01.446588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.459542 (+ 12954us) service_pool.cc:224] Handling call
0709 13:20:04.389680 (+2930138us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54001ffc54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:04.389890 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6464890624}
W20260709 13:20:04.402275 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 55625.8us
I20260709 13:20:04.389997 17668 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.424801 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3973) took 1052 ms. Trace:
I20260709 13:20:04.418394 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:04.407688 17879 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:04.439702 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 165207us
I20260709 13:20:04.442180 17879 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.442299 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:04.443825 10128 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:04.410701 17911 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:04.467761 10198 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:04.443313 17735 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:04.483968 17783 consensus_meta.cc:287] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.803s	user 0.019s	sys 0.004s
I20260709 13:20:04.483801 17907 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:04.484763 17783 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.487344 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:04.497797 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 80141.8us
I20260709 13:20:04.430923 10125 rpcz_store.cc:276] 0709 13:20:03.360803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.541710 (+180907us) service_pool.cc:224] Handling call
0709 13:20:03.542129 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.885664 (+343535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.885724 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.886665 (+   941us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.220984 (+334319us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:04.221025 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.237932 (+ 16907us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540036ae54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:04.237998 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.395342 (+157344us) raft_consensus.cc:1685] finished
0709 13:20:04.395368 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.413112 (+ 17744us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26546,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":246,"spinlock_wait_cycles":787972352,"thread_start_us":333652,"threads_started":1,"wal-append.queue_time_us":415183}
W20260709 13:20:04.500455 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45986.8us
I20260709 13:20:04.450583 17885 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.536828 17881 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:20:04.537492 17704 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.014s
I20260709 13:20:04.537843 17583 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 13:20:04.539830 17783 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:04.516933 17879 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.586966 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.602479 17668 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:04.602078 17857 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.609833 17885 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.614394 17885 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.615562 17885 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:04.619606 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 177564us
I20260709 13:20:04.619633 17885 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.628863 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.634985 17879 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.637866 17879 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.526356 17661 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:04.639703 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 98287.9us
W20260709 13:20:04.646245 17721 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.636s	user 0.022s	sys 0.001s
I20260709 13:20:04.685174 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.686071 10197 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:20:04.593670 17704 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.672840 17876 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.668435 17911 consensus_queue.cc:1048] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:20:04.697283 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:04.705703 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 376975us
I20260709 13:20:04.707873 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:04.714473 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:04.716014 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:04.717139 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 439425us
I20260709 13:20:04.666774 17881 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:04.725687 10205 raft_consensus.cc:2368] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
I20260709 13:20:04.723685 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:04.735774 10128 raft_consensus.cc:2468] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:04.736155 10201 raft_consensus.cc:2410] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 13:20:04.701203 17661 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.753840 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:04.740752 17735 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.762831 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:04.745970 17829 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:04.761824 10168 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:04.781036 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.783209 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:04.789388 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 52052.1us
I20260709 13:20:04.794528 17881 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:04.795543 17881 raft_consensus.cc:2804] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:04.796099 17881 raft_consensus.cc:697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:04.797237 17881 consensus_queue.cc:237] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:04.805491 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58347.5us
I20260709 13:20:04.811935 10124 raft_consensus.cc:2468] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
W20260709 13:20:04.812397 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 132848us
I20260709 13:20:04.785327 17829 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:04.815519 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:04.814263 17812 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:04.821019 10197 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
W20260709 13:20:04.811681 17875 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 1.084s	user 0.016s	sys 0.005s
W20260709 13:20:04.837973 17697 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.591s	user 0.010s	sys 0.018s
I20260709 13:20:04.838644 17697 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.841310 17697 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:04.832295 17875 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.850884 10204 raft_consensus.cc:1240] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:20:04.852072 17875 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:04.825862 17876 consensus_queue.cc:1059] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:04.829146 17704 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:04.867542 17393 consensus_queue.cc:1048] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:20:04.882682 10199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.050s	sys 0.079s
W20260709 13:20:04.883276 10199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.051s	sys 0.079s
I20260709 13:20:04.865792 17721 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:04.882064 17697 raft_consensus.cc:3055] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:20:04.895104 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:04.896813 17697 raft_consensus.cc:740] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:04.907931 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:04.912232 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:04.922935 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92911us
I20260709 13:20:04.936609 10020 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
W20260709 13:20:04.937559 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46832.6us
I20260709 13:20:04.936954 17697 consensus_queue.cc:260] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.938644 17735 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:04.939293 17735 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:04.940383 17704 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:20:04.941030 17704 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.940536 17735 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:04.941431 17704 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:04.904593 17876 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:20:04.947402 17936 log.cc:927] Time spent T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 13:20:04.954684 17661 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:04.957446 17697 raft_consensus.cc:3060] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:04.934468 17721 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:04.983768 17938 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:20:04.984755 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 109787us
W20260709 13:20:04.985862 10096 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:04.986374 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:04.987394 17896 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:20:05.000828 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 243695us
W20260709 13:20:05.002876 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95495.1us
I20260709 13:20:05.004757 17721 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:05.019583 10122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 3980) took 1400 ms. Trace:
I20260709 13:20:05.011466 17852 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:20:05.029166 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 85473.3us
I20260709 13:20:05.024379 10122 rpcz_store.cc:276] 0709 13:20:03.613976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.773002 (+159026us) service_pool.cc:224] Handling call
0709 13:20:03.773773 (+   771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.926236 (+152463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.926272 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.119780 (+193508us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.324582 (+204802us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:04.324646 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.324740 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.014358 (+689618us) raft_consensus.cc:1685] finished
0709 13:20:05.014395 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.014606 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111609,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":308,"thread_start_us":315259,"threads_started":2,"wal-append.queue_time_us":337730}
W20260709 13:20:05.037029 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 129391us
I20260709 13:20:05.043394 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:05.044745 17948 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:05.045288 17948 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.045655 17948 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:05.059227 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:05.063556 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 88099.8us
I20260709 13:20:05.064857 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:05.066601 17697 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.067420 17885 raft_consensus.cc:2804] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:05.066686 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:05.070262 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 84488.5us
I20260709 13:20:05.080444 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.081677 10197 raft_consensus.cc:2393] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 18: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:20:05.068735 10095 connection.cc:256] RPC call timeout handler was delayed by -1.3044828720001078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260709 13:20:05.086529 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:05.079543 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:05.094362 17950 log.cc:927] Time spent T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:05.097599 10895 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 24. Current term is 25. Ops: []
I20260709 13:20:05.097848 17366 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:05.098251 17366 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:05.099409 17366 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.100663 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.101042 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.101424 10895 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:05.104722 10894 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:05.106216 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.119359 17876 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:05.122007 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 117765us
I20260709 13:20:05.123382 10196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57108 (request call id 3894) took 1003 ms. Trace:
W20260709 13:20:05.124318 10096 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.123972 10196 rpcz_store.cc:276] 0709 13:20:04.120257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.126021 (+  5764us) service_pool.cc:224] Handling call
0709 13:20:04.181979 (+ 55958us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:04.280963 (+ 98984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.281009 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:04.347050 (+ 66041us) log.cc:844] Serialized 52 byte log entry
0709 13:20:04.785230 (+438180us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:04.793292 (+  8062us) log.cc:844] Serialized 14 byte log entry
0709 13:20:04.904454 (+111162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.904546 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.122826 (+218280us) raft_consensus.cc:1685] finished
0709 13:20:05.122847 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.123058 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23940,"prepare.queue_time_us":8518,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":49012,"raft.run_cpu_time_us":526,"raft.run_wall_time_us":48382,"thread_start_us":472284,"threads_started":2,"wal-append.queue_time_us":513494}
I20260709 13:20:05.070281 17885 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:05.127599 17885 raft_consensus.cc:3060] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:05.129294 17366 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:05.143815 17697 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.148651 10168 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.151755 17852 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:05.153589 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.154680 17946 raft_consensus.cc:2804] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:05.156065 17946 raft_consensus.cc:697] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:05.154886 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:05.158711 17947 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:05.159911 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 91099.2us
I20260709 13:20:05.160075 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:05.161103 17947 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:05.161531 17876 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:05.161783 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.161072 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:05.155864 17704 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.154729 17852 raft_consensus.cc:2764] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:20:05.148187 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 81682.2us
W20260709 13:20:05.167413 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.083657 17948 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.164047 10168 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.157225 17697 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:05.166326 17896 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:05.185348 10168 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.192610 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:05.193369 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 66266.4us
W20260709 13:20:05.194422 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.195883 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.194887 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:05.199708 17704 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:05.172397 17876 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:20:05.201164 10096 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.201470 17876 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:05.201772 10096 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:05.189167 17896 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:05.203066 17896 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:05.208637 17870 raft_consensus.cc:2804] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:05.135363 17907 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.074s
I20260709 13:20:05.194764 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:05.157748 17946 consensus_queue.cc:237] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.193131 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:05.213901 10891 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:05.216557 17697 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.237934 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.239759 10891 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:05.241899 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:05.219798 17852 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:05.217630 17697 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.225291 17661 raft_consensus.cc:2804] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:20:05.247047 10168 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.236430 17870 raft_consensus.cc:697] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:05.248581 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:05.246670 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:05.260514 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 65864.3us
I20260709 13:20:05.262025 10127 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:05.261513 10130 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:05.269773 17948 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:05.273738 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.274765 10204 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 18.
W20260709 13:20:05.280715 10169 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.252629 17876 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:05.252406 17735 raft_consensus.cc:3060] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:05.260725 17661 raft_consensus.cc:697] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:05.293354 17661 consensus_queue.cc:237] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.294500 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:05.295676 10126 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:05.303718 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 115188us
I20260709 13:20:05.304273 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:05.304777 17881 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:05.306089 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:05.306505 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.307148 10865 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:05.307965 17876 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:05.308755 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.308208 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:05.305409 17881 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.306856 10125 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:05.356966 17948 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:05.357297 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48590.1us
I20260709 13:20:05.358367 17907 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.359045 17907 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:05.361964 17907 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:05.370033 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:05.373085 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 63199.4us
I20260709 13:20:05.375267 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:05.376266 10122 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:05.362357 17881 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:05.382915 10019 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:05.382196 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.384804 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.363559 17852 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:05.392340 10125 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 27.
I20260709 13:20:05.394942 10198 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 13:20:05.462324 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:05.463349 10895 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 25: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:05.464632 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:05.465657 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 44395.1us
W20260709 13:20:05.468070 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 266932us
W20260709 13:20:05.465622 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 182851us
W20260709 13:20:05.475267 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 290696us
I20260709 13:20:05.467384 10864 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:05.464854 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.492240 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:05.493167 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
W20260709 13:20:05.518743 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.531520 10895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.500962 17876 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.503378 17829 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:05.566213 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46880.5us
I20260709 13:20:05.573390 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.559800 17783 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:20:05.607658 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 225369us
I20260709 13:20:05.472920 17870 consensus_queue.cc:237] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.610432 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:05.615936 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 106739us
W20260709 13:20:05.627246 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58923.8us
I20260709 13:20:05.629257 10891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.633051 10891 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:05.636811 17812 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:05.632684 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.637979 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:05.637557 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:05.643162 17875 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:05.644122 17875 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.646138 17812 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.646520 17875 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:05.612548 17783 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.637655 17583 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:20:05.629607 10096 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.574455 17876 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:05.670675 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 326507us
I20260709 13:20:05.626564 17964 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:05.668776 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 105894us
I20260709 13:20:05.660214 17721 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:05.669448 17783 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:05.646926 17812 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:05.692488 17959 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:20:05.693267 10020 leader_election.cc:400] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
I20260709 13:20:05.700557 10020 leader_election.cc:403] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:20:05.693591 17960 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:20:05.670773 17583 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:05.708326 17583 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:05.694238 17965 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:20:05.695453 17721 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:05.708688 17955 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 13:20:05.707468 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:05.707760 17963 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:20:05.731741 10168 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:05.734352 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 97661us
W20260709 13:20:05.727785 17953 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.328s	user 0.000s	sys 0.002s
I20260709 13:20:05.767899 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.776489 10197 raft_consensus.cc:2468] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 24.
I20260709 13:20:05.770629 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.721832 17697 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:05.780205 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45125.9us
I20260709 13:20:05.784173 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:05.684274 17964 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:05.789494 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.792610 10126 raft_consensus.cc:2468] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:20:05.794390 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:05.776571 17960 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:20:05.798960 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 188138us
I20260709 13:20:05.776767 17875 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:20:05.804463 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.806020 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:05.809679 17980 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:20:05.880302 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62960.5us
I20260709 13:20:05.880983 10198 raft_consensus.cc:1240] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:05.818158 17975 consensus_queue.cc:1059] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
W20260709 13:20:05.893998 17956 log.cc:927] Time spent T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.289s	user 0.000s	sys 0.002s
W20260709 13:20:05.897956 17982 log.cc:927] Time spent T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:20:05.901014 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:05.837278 17697 raft_consensus.cc:3060] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:05.924606 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 227254us
W20260709 13:20:05.891605 17981 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:20:05.926928 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:05.930091 10199 consensus_meta.cc:287] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.586s	user 0.016s	sys 0.002s
I20260709 13:20:05.932646 10199 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 25.
I20260709 13:20:05.878156 17875 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:05.949783 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:05.939886 17983 log.cc:927] Time spent T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.166s	user 0.003s	sys 0.000s
I20260709 13:20:05.948206 17948 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:20:05.969576 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.905465 17980 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:05.969599 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:05.970129 10168 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:05.981189 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:05.936480 17812 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:05.980566 17896 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.777s	user 0.017s	sys 0.000s
W20260709 13:20:06.014331 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 53523.5us
I20260709 13:20:05.939997 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:05.933130 17984 raft_consensus.cc:3055] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:06.007047 17980 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:06.008944 17876 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:06.050654 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 223915us
I20260709 13:20:06.022960 17859 raft_consensus.cc:2804] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:20:06.055529 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 327351us
W20260709 13:20:06.056159 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 75083.4us
I20260709 13:20:06.058019 10197 raft_consensus.cc:2410] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 13:20:06.054171 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:06.062935 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:06.061975 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 273462us
W20260709 13:20:06.062551 17735 consensus_meta.cc:287] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.005s
I20260709 13:20:06.054639 17879 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:06.026506 17896 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:06.092423 10122 consensus_meta.cc:287] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Time spent flushing consensus metadata: real 0.716s	user 0.018s	sys 0.000s
I20260709 13:20:06.099397 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:06.071177 17911 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.010s
I20260709 13:20:06.121333 10122 raft_consensus.cc:2410] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 13:20:06.134789 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 169091us
I20260709 13:20:06.179610 10128 raft_consensus.cc:1275] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:06.127275 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:06.185204 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 83504.5us
W20260709 13:20:06.127803 17990 log.cc:927] Time spent T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:20:06.104476 17911 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.008s	sys 0.010s
I20260709 13:20:06.095083 10200 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
W20260709 13:20:06.096083 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:06.180038 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:06.211410 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115415us
I20260709 13:20:06.134382 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.080348 17859 raft_consensus.cc:697] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:06.213575 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 79314.8us
I20260709 13:20:06.067787 17994 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:06.218499 10203 raft_consensus.cc:2468] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:20:06.100422 17829 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:06.106515 17735 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:06.073602 17947 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:06.184235 10131 raft_consensus.cc:1275] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:06.077589 17962 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:06.209095 10127 raft_consensus.cc:1275] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:20:06.207376 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 422668us
I20260709 13:20:06.213747 10130 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:06.181567 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.053740 17984 raft_consensus.cc:740] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:06.219964 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.225308 10128 raft_consensus.cc:1275] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:20:06.225541 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.237681 10129 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:06.236618 17971 raft_consensus.cc:2804] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:06.253181 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:06.222069 17829 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:06.092381 17875 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.225937 17962 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.278656 17962 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:06.284231 17896 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:06.289361 18001 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.289986 17881 consensus_queue.cc:1048] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:06.290246 18001 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.292096 10201 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:06.292900 17881 consensus_queue.cc:1048] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:06.293783 17735 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.294276 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:06.294425 17735 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.294955 10204 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:06.295631 10204 raft_consensus.cc:2410] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:20:06.295425 17974 consensus_queue.cc:1048] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:06.296891 17735 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:06.298187 17996 log.cc:927] Time spent T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:20:06.270368 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 99314.3us
I20260709 13:20:06.263990 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.298696 17969 consensus_queue.cc:1048] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:20:06.301760 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:06.293150 18001 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:06.315851 17998 log.cc:927] Time spent T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:20:06.318120 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 226184us
I20260709 13:20:06.319569 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:06.307287 10170 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.312098 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 52621.6us
W20260709 13:20:06.313709 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49855.6us
I20260709 13:20:06.259789 17876 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:06.325212 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:06.267925 17971 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:06.302856 17879 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:06.268509 17964 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:06.352063 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:06.363443 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:06.363827 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:06.367508 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:06.351854 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51534.3us
W20260709 13:20:06.368201 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48837.6us
I20260709 13:20:06.390508 10126 raft_consensus.cc:2368] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
W20260709 13:20:06.389901 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 170555us
W20260709 13:20:06.371392 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 46307.1us
W20260709 13:20:06.399518 10096 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.400225 10096 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:06.404057 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.405380 10122 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:20:06.405691 17721 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:06.299505 17812 consensus_queue.cc:1059] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:20:06.407398 17812 raft_consensus.cc:3055] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:06.408061 17812 raft_consensus.cc:740] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:06.409456 17812 consensus_queue.cc:260] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.413808 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.349045 17999 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:06.415887 17999 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:06.417490 17999 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.420046 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:06.422832 18000 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:20:06.424130 10130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45986 (request call id 3591) took 1049 ms. Trace:
I20260709 13:20:06.375073 17971 raft_consensus.cc:3060] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:06.424854 10130 rpcz_store.cc:276] 0709 13:20:05.374694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.398297 (+ 23603us) service_pool.cc:224] Handling call
0709 13:20:06.423763 (+1025466us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54003d4854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:06.424023 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684136832}
W20260709 13:20:06.427870 17979 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.544s	user 0.005s	sys 0.000s
W20260709 13:20:06.428754 17885 consensus_meta.cc:287] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 1.300s	user 0.011s	sys 0.007s
I20260709 13:20:06.429350 17885 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.432526 10123 raft_consensus.cc:2410] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:20:06.434501 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3580) took 1134 ms. Trace:
I20260709 13:20:06.435076 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.435225 10124 rpcz_store.cc:276] 0709 13:20:05.299862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.345338 (+ 45476us) service_pool.cc:224] Handling call
0709 13:20:05.377389 (+ 32051us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.388370 (+ 10981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.388408 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.467038 (+ 78630us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.572317 (+105279us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:05.572397 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.572524 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.418977 (+846453us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa277b9f86e 00007fa277b9ebb6 00007fa277b9e7d8 00007fa277ba8221 00007fa277ba81d9 00007fa277ba8169
0709 13:20:06.433999 (+ 15022us) raft_consensus.cc:1685] finished
0709 13:20:06.434033 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.434385 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205402,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":23065,"spinlock_wait_cycles":65006464,"thread_start_us":48398,"threads_started":1,"wal-append.queue_time_us":280305}
I20260709 13:20:06.439461 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.441028 10124 raft_consensus.cc:2410] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:20:06.352399 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:06.451694 10125 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 18: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:06.462162 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:06.456182 17881 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:06.470981 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 71533.6us
I20260709 13:20:06.471012 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:06.357491 17988 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:06.471437 17881 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.442723 17735 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:06.475858 17964 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:06.476812 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:06.260646 17947 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:06.480257 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91219.8us
I20260709 13:20:06.368625 17975 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:06.487684 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:06.492549 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:06.493042 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.466264 17879 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:06.502669 10096 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.503252 10096 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.511730 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:06.513540 10196 raft_consensus.cc:2393] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 17: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:20:06.514508 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.479417 17988 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.502358 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.532037 10019 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:06.534057 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.534693 17992 log.cc:927] Time spent T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.439s	user 0.002s	sys 0.000s
W20260709 13:20:06.538436  8925 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:06.541471 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:06.595222 10127 raft_consensus.cc:2368] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:06.548666 17885 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:06.600276 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 98059.6us
I20260709 13:20:06.602166 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.548898 17991 raft_consensus.cc:2804] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:06.489984 17812 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:06.603657 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 71795.2us
I20260709 13:20:06.604696 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.605723 17994 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:06.472323 17984 consensus_queue.cc:260] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:06.550572 17783 consensus_meta.cc:287] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.845s	user 0.009s	sys 0.004s
I20260709 13:20:06.490778 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.548533 17980 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.362241 17829 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.553598 17973 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:06.530478 17879 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.555172 17962 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.557058 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.560520 17907 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:20:06.568001 10126 raft_consensus.cc:2393] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 19: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:20:06.637570 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 80635.5us
W20260709 13:20:06.637791 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 147118us
W20260709 13:20:06.580513 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 48015us
W20260709 13:20:06.577090 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.641374 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:06.597329 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.635282 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.497587 17876 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.110s
W20260709 13:20:06.648010 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 70963us
W20260709 13:20:06.649026 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.593811 17988 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:06.630590 17976 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.010s	sys 0.041s
W20260709 13:20:06.656807 10170 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.637301 17583 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.918s	user 0.011s	sys 0.003s
W20260709 13:20:06.660763 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 58695.1us
I20260709 13:20:06.603673 17991 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.662262 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:06.636946 17885 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:06.478641 17859 consensus_queue.cc:237] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:06.660789 17976 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.011s	sys 0.041s
I20260709 13:20:06.609005 17735 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:06.666778 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.649149 17661 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:06.636845 17907 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:20:06.671660 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 176907us
I20260709 13:20:06.672991 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:06.636898 17964 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:06.653548 17975 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:06.657109 17994 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.681921 10127 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:06.632961 17783 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.697253 10020 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.705118 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:06.698449 18013 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:06.665488 17991 raft_consensus.cc:3060] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:06.712328 10126 raft_consensus.cc:2410] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:20:06.682577 17984 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:06.713548 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 108955us
I20260709 13:20:06.724265 10129 raft_consensus.cc:1240] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:20:06.734669 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:06.741845 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 85943.4us
I20260709 13:20:06.668287 17583 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.721763 17976 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:06.753304 10170 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.760927 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:06.760643 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.767982 10019 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.776880 18004 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:06.777240 17987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.004s	sys 0.011s
W20260709 13:20:06.777812 17987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.004s	sys 0.012s
I20260709 13:20:06.777840 18005 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:20:06.784174 18012 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:20:06.788097 17962 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:06.790525 17829 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:06.791319 17949 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.792124 17949 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.795421 17907 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.797235 17907 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:06.723006 17994 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.799695 17879 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:06.801236 17697 consensus_meta.cc:287] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.867s	user 0.013s	sys 0.002s
I20260709 13:20:06.801509 17994 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:06.802418 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:06.802997 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:06.719795 18013 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.776017 17946 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:06.801792 17697 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
W20260709 13:20:06.832984 17977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.003s	sys 0.013s
I20260709 13:20:06.821686 17879 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:06.846565 17973 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:06.846457 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 98469.8us
I20260709 13:20:06.853986 18013 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:06.854328 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 101766us
W20260709 13:20:06.856315 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83566.5us
I20260709 13:20:06.837080 17962 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:06.858856 17977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.003s	sys 0.015s
I20260709 13:20:06.878263 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:06.872531 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 1.832s (SENT)
I20260709 13:20:06.842200 17980 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:06.882949 17988 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:06.855849 17971 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:06.894889 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.860001 17973 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:06.880385 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.916203 10019 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.897873 18004 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:20:06.917887 10096 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.920164 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:06.923643 10096 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:06.940210 17911 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:06.948413 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47916.4us
I20260709 13:20:06.929554 17879 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:06.933696 18004 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:07.007879 17988 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:06.953688 10096 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:06.968453 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 96014.3us
I20260709 13:20:06.951371 10203 raft_consensus.cc:1275] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:20:06.969090 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:06.965387 10199 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:06.979938 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:06.983561 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:07.010960 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:06.983966 17971 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:06.984828 17997 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.731s	user 0.000s	sys 0.007s
I20260709 13:20:06.962364 17991 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:06.993000 17661 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:06.967360 18022 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:20:07.002532 17393 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.004312 17949 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:07.019565 10893 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:20:06.964385 18020 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.145s	user 0.003s	sys 0.000s
I20260709 13:20:07.024019 10899 raft_consensus.cc:2393] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:06.962046 17947 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:07.055658 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.056034 10202 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:07.039422 17988 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:07.059419 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92640.7us
W20260709 13:20:07.061870 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 108288us
I20260709 13:20:07.038373 17393 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.071065 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:07.073153 18030 log.cc:927] Time spent T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
W20260709 13:20:07.083343 10021 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.084461 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:07.086530 10896 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 13:20:07.075804 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.087306 10896 pending_rounds.cc:85] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Aborting all ops after (but not including) 18
I20260709 13:20:07.087877 10199 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
I20260709 13:20:07.088229 10896 pending_rounds.cc:107] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 13:20:07.088753 10896 raft_consensus.cc:2889] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.092157 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.113703 17393 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:07.074821 18004 consensus_queue.cc:1048] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:20:07.119366 10096 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:07.121595 10096 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:07.123046 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.125499 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:07.126449 17988 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.126987 18022 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:07.127544 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:07.127691 18022 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.128564 10897 raft_consensus.cc:2393] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c in current term 20: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:07.130487 18022 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.127168 17988 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.136341 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.139844 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:07.141476 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.145455 10204 raft_consensus.cc:2368] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:07.151029 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.169363 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:07.174866 10095 leader_election.cc:400] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:07.175575 10095 leader_election.cc:403] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:20:07.178534 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:07.179512 10197 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:07.180943 10864 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.143889 17947 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.188799 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.183394 17977 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:07.189224 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.182893 18035 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:20:07.185565 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:07.148815 18004 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.190460 10897 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:07.196580 17988 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.201830 17976 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:07.218150 10204 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:20:07.218875 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.221282 10196 raft_consensus.cc:2410] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:20:07.183466 17697 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:20:07.227952 10019 leader_election.cc:400] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 18. Current term is 19.
I20260709 13:20:07.231068 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.188036 17962 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.233038 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.193192 17947 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:07.182248 17911 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:20:07.249099 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.249567 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.252345 10126 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:07.246457 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:07.258363 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.260022 10122 raft_consensus.cc:2410] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:20:07.261495 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.262482 10128 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.245042 17947 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:07.282076 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49133.2us
I20260709 13:20:07.292514 10019 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.294549 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:07.275256 17812 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:07.276508 17951 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:07.244730 17962 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.322412 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:07.275399 17973 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.334429 17973 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:07.337510 17962 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.305262 17583 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:07.291428 17991 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.339792 10124 raft_consensus.cc:1275] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:07.340307 17881 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:07.341214 17852 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.342196 17852 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.345387 17907 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:07.346027 17907 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:07.284505 17870 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
W20260709 13:20:07.348340 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 127574us
W20260709 13:20:07.350529 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:07.275530 18038 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 13:20:07.353834 17783 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:07.315609 17951 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.356618 17951 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:07.365295 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:07.366145 10892 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:07.372398 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.377543 10197 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:07.345496 17988 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:07.382786 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 90345.3us
I20260709 13:20:07.384145 10019 leader_election.cc:400] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:20:07.385535 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.380015 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.398428 10020 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.375764 17852 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.402384 17988 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:07.406795 17984 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:07.458537 10128 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:20:07.464767 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 79368.8us
I20260709 13:20:07.466464 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.472390 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.477614 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.480019 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:07.476012 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69391.4us
W20260709 13:20:07.484081 10863 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Timed out: RequestConsensusVote RPC to 127.8.183.66:41681 timed out after 2.119s (SENT)
I20260709 13:20:07.484658 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.485172 10128 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 25: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:07.470188 17984 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:07.476444 17879 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.494536 10130 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: []
W20260709 13:20:07.498135 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 198184us
I20260709 13:20:07.504825 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.484540 17988 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:07.518082 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.520459 10019 leader_election.cc:400] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:07.520892 10019 leader_election.cc:403] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:20:07.522053 10128 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:07.494207 18001 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.537356 10019 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.544852 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.547994 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.559324 10019 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.570757 10019 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.575610 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.536136 17876 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:07.544072 17971 raft_consensus.cc:3060] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:07.579880 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.589010 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:07.594717  8925 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:07.597375 17907 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.599252 17907 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:07.603760 18003 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:07.604533 18003 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:07.605858 18003 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.618652 17876 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:07.620003 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.621379 10125 raft_consensus.cc:2393] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 25: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:07.569037 18033 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:07.565608 17988 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:07.577418 18016 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.632452 10896 raft_consensus.cc:1275] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:07.588587 17991 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.638463 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.582185 17879 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:07.611331 17812 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:07.624722 18003 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:07.644949 10130 raft_consensus.cc:2393] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 22: Already voted for candidate 01bfaf1ad56a4deea886bbc64dec466c in this term.
I20260709 13:20:07.624127 17977 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:07.623574 17876 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.647749 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.578926 18001 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:20:07.645520 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.662693 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.667657 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:07.671887 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92109.9us
W20260709 13:20:07.673604 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 189568us
I20260709 13:20:07.684481 10130 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:20:07.675374 18047 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.201s	user 0.000s	sys 0.002s
I20260709 13:20:07.646811 17991 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.690791 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:07.692250 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 193291us
I20260709 13:20:07.649132 17987 consensus_queue.cc:1048] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:20:07.643435 18016 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.640522 17984 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.698899 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.704221 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.705189 10128 raft_consensus.cc:2368] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:20:07.663023 17876 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.708596 10122 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:20:07.709093 10122 pending_rounds.cc:85] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Aborting all ops after (but not including) 11
I20260709 13:20:07.709422 10122 pending_rounds.cc:107] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:20:07.709681 10122 raft_consensus.cc:2889] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.714016 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:07.735596 18051 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:20:07.736629 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:07.738188 18049 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 13:20:07.739707 18057 log.cc:927] Time spent T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:20:07.743223 18052 log.cc:927] Time spent T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.001s	sys 0.001s
I20260709 13:20:07.754132 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.761251 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.762692 10892 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:07.770512 10197 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:07.731156 18064 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:07.772464 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:07.776983 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.787290 18001 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.788039 18001 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:07.790553 18001 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:07.801859 17991 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:07.802974 10866 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.803686 18016 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:07.807628 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:07.812773 18061 log.cc:927] Time spent T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:20:07.826495 10131 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:07.794916 17859 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:20:07.846045 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 102206us
I20260709 13:20:07.859786 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:07.835269 17991 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:07.908160 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 88701.8us
I20260709 13:20:07.799755 17947 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.916054 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.859867 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:07.853940 18022 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.850180 18063 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.957540 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:07.961661 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:07.952630 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:07.813853 18064 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.875095 17859 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:07.844375 17951 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:07.970837 10127 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
W20260709 13:20:07.951777 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 91996.1us
W20260709 13:20:07.981973 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 56660.1us
I20260709 13:20:07.985109 10130 raft_consensus.cc:1240] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:20:07.942044 18005 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:08.008239 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.966235 17721 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:20:08.076050 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 93204.5us
W20260709 13:20:07.989861 10170 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:08.085009 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76925us
I20260709 13:20:08.086400 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:07.918581 17991 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:08.080519 18022 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:08.092893 18066 log.cc:927] Time spent T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:20:08.129860 17991 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:08.129884 18022 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:08.103749 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 113929us
W20260709 13:20:08.132179 10170 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.132704 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:07.984179 18063 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:08.143319 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57032.9us
I20260709 13:20:08.145790 10863 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:08.078524 18038 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:08.120720 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:08.120544 17366 consensus_queue.cc:1048] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:08.125200 18064 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:08.162364 10130 raft_consensus.cc:1275] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:20:08.163863 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:08.126113 18062 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.423s	user 0.002s	sys 0.000s
W20260709 13:20:08.094137 18068 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:20:08.109642 17951 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:08.160050 10129 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:20:08.078796 17721 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:08.159842 18074 log.cc:927] Time spent T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:20:08.156886 10170 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.170121 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:08.171365 10198 raft_consensus.cc:1240] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:08.175309 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:08.166213 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.205338 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.206261 10203 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:20:08.155783 17948 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:08.158329 18060 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.214597 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.217940 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:08.218292 10197 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 17.
I20260709 13:20:08.218677 10893 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:08.168311 18022 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.249440 18022 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:08.184994 18033 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 13:20:08.254104 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:08.255613 17991 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.258284 18022 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:08.223654 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.167138 17583 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:08.170939 18070 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:08.230506 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:08.274055 10127 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:08.239634 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 82992.7us
W20260709 13:20:08.274902 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49918.2us
W20260709 13:20:08.276602 10170 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.221565 17859 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:08.206621 18001 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:08.247707 18062 log.cc:927] Time spent T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:08.200273 17721 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:08.175314 17907 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:20:08.228299 17971 consensus_meta.cc:287] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.000s
I20260709 13:20:08.265098 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:08.256384 17991 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:08.272796 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.283229 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:08.284309 17971 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:08.291136 18022 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:08.295074 18064 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:08.317286 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.329054 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.288233 18001 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:08.338274 18003 consensus_meta.cc:287] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.001s
W20260709 13:20:08.365041 10170 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.369290 10893 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:08.330166 18022 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.333640 18064 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.377125 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:08.332507 18070 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:08.391999 10170 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:08.393519 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 92818.2us
I20260709 13:20:08.396231 10127 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
W20260709 13:20:08.408401 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 201869us
I20260709 13:20:08.415987 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.332835 17583 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 13:20:08.327751 17812 consensus_queue.cc:1059] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:20:08.389501 18022 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:08.411698 10170 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:08.425276 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48311.9us
I20260709 13:20:08.426301 17366 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:08.398381 18064 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:08.442636 10170 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.367184 18071 consensus_queue.cc:1048] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.109s
W20260709 13:20:08.455317 10170 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.459591 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:08.475955 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:08.464900 17366 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:08.468067 17951 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:08.474954 18063 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:08.522841 18076 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:20:08.526224 18077 log.cc:927] Time spent T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 13:20:08.528034 18081 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:20:08.555338 18078 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.191s	user 0.000s	sys 0.002s
I20260709 13:20:08.505070 18070 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:08.559811 17991 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:08.577157 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.584824 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.585947 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:08.587534 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:08.594758 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:08.595304  8925 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:08.598423 17976 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:08.603180 17991 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:08.603899 17366 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:08.606825 17876 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:08.608767 17721 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:08.620896 10892 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Rejecting Update request from peer e9a849ac7eda426abfca97cf8b15da0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:08.621563 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:08.623510 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:08.640189 10202 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:08.619649 18001 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:08.636335 17879 raft_consensus.cc:3055] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:20:08.643639 17984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.008s
I20260709 13:20:08.665166 17947 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:08.650902 17879 raft_consensus.cc:740] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:20:08.668196 18089 log.cc:927] Time spent T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.002s
W20260709 13:20:08.654929 17984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.010s	sys 0.008s
W20260709 13:20:08.673846 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 204517us
W20260709 13:20:08.675886 18088 log.cc:927] Time spent T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:20:08.679004 17879 consensus_queue.cc:260] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:08.680778 17879 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:08.637775 18022 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:08.695385 10170 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.696388 10170 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:08.698568 10170 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:08.701144 10170 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.706084 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:08.711361 17984 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:08.682247 18087 log.cc:927] Time spent T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:20:08.713126 18090 log.cc:927] Time spent T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:20:08.715080 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:08.708726 17661 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:08.715646 10170 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:08.737092 10170 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.737725 10170 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:08.743726 10170 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.745016 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:08.738250 17661 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:20:08.754683 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.745738 18071 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:08.747267 17984 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:08.753479 18048 log.cc:927] Time spent T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 1.236s	user 0.006s	sys 0.009s
I20260709 13:20:08.751475 17366 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:08.783138 18063 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:08.793696 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 4012) took 1030 ms. Trace:
I20260709 13:20:08.798996 10125 rpcz_store.cc:276] 0709 13:20:07.763322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.763729 (+   407us) service_pool.cc:224] Handling call
0709 13:20:07.783514 (+ 19785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:07.799320 (+ 15806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:07.799369 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.849716 (+ 50347us) log.cc:844] Serialized 26 byte log entry
0709 13:20:08.172947 (+323231us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:08.217373 (+ 44426us) log.cc:844] Serialized 14 byte log entry
0709 13:20:08.759163 (+541790us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b180044eae0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274e91c0a 00007fa27e5d1219 00007fa274f19a40 00007fa274f144e4 00007fa274f14455 00007fa277ffc060 00007fa27800e716 00007fa27801f287 00007fa27801f208
0709 13:20:08.759438 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.759541 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:08.759593 (+    52us) raft_consensus.cc:1685] finished
0709 13:20:08.784924 (+ 25331us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.793600 (+  8676us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58760,"prepare.queue_time_us":51313,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":594431,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":248,"spinlock_wait_cycles":9438592,"thread_start_us":204144,"threads_started":1,"wal-append.queue_time_us":391563}
W20260709 13:20:08.832468 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.792402 18050 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:08.894661 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 278274us
I20260709 13:20:08.895740 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:08.909361 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64699us
W20260709 13:20:08.916790 10170 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:08.923580 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:08.932662 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:08.933728 10899 raft_consensus.cc:2368] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:08.846071 18004 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:08.809299 17984 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:08.960445 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:08.972072 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.984629 10124 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 952) took 1285 ms. Trace:
I20260709 13:20:08.985321 10124 rpcz_store.cc:276] 0709 13:20:07.698903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.699080 (+   177us) service_pool.cc:224] Handling call
0709 13:20:07.703639 (+  4559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.707297 (+  3658us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:20:07.918252 (+210955us) log.cc:844] Serialized 14 byte log entry
0709 13:20:08.974009 (+1055757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:08.974079 (+    70us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:20:08.984266 (+ 10187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.984391 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.984552 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359479,"prepare.queue_time_us":369353,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":476882,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":30134,"thread_start_us":805727,"threads_started":3,"wal-append.queue_time_us":540161}
I20260709 13:20:08.893703 18050 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:08.987963 10123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 4013) took 1205 ms. Trace:
W20260709 13:20:08.990528 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.969295 17989 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:08.912684 18064 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:08.969576 18022 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:09.003213 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 193798us
W20260709 13:20:09.006127 10096 leader_election.cc:336] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.718s (SENT)
I20260709 13:20:09.006830 10096 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.007227 10124 raft_consensus.cc:1275] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:20:08.992722 18100 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
I20260709 13:20:08.998528 18016 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:08.993556 10123 rpcz_store.cc:276] 0709 13:20:07.782606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.782779 (+   173us) service_pool.cc:224] Handling call
0709 13:20:07.801795 (+ 19016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.802073 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:07.946496 (+144423us) log.cc:844] Serialized 14 byte log entry
0709 13:20:08.987447 (+1040951us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:08.987530 (+    83us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:08.987605 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.987694 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.987861 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":433843,"prepare.queue_time_us":287725,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":530739,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":19614,"thread_start_us":754043,"threads_started":3,"wal-append.queue_time_us":458739}
I20260709 13:20:08.975659 18004 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.009063 17977 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:09.017385 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:09.054311 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.054862 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.016573 17987 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:09.030435 18098 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:20:09.072556 17721 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:09.061153 17881 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:09.010156 17852 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:20:09.080528 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 74460.3us
W20260709 13:20:09.082070 10096 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Timed out: RequestConsensusVote RPC to 127.8.183.65:43347 timed out after 1.770s (SENT)
I20260709 13:20:09.082577 10096 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.066749 18102 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:09.084831 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 73009.8us
I20260709 13:20:09.088042 10892 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:09.088078 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:09.089246 10198 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:09.099866 10200 raft_consensus.cc:1240] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer 01bfaf1ad56a4deea886bbc64dec466c for earlier term 20. Current term is 21. Ops: []
W20260709 13:20:09.098999 10865 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 114 similar messages]
I20260709 13:20:09.107606 10126 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45970 (request call id 4017) took 1197 ms. Trace:
I20260709 13:20:09.068691 18003 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.108280 10126 rpcz_store.cc:276] 0709 13:20:07.910553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.924344 (+ 13791us) service_pool.cc:224] Handling call
0709 13:20:07.943459 (+ 19115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.943688 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:08.152560 (+208872us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.107065 (+954505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.107208 (+   143us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:09.107263 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.107365 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.107543 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":420843,"prepare.queue_time_us":146388,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":524882,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":12530,"thread_start_us":714827,"threads_started":3,"wal-append.queue_time_us":292215}
I20260709 13:20:09.124732 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:09.125602 10200 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:09.120290 18064 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.121523 17984 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.102772 18056 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.144438 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.064792 17977 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:09.143512 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:09.155968 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:09.157128 10198 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:09.159168 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:09.151150 18056 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.167399 18056 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:09.162375 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.172196 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.120401 18050 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:09.099587 17987 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:09.183934 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84660us
I20260709 13:20:09.134117 17991 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.144625 17812 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:09.117794 17366 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:20:09.135464 18063 consensus_queue.cc:1059] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:20:09.154531 17984 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:09.150954 17879 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.203603 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.208485 10095 leader_election.cc:400] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:09.131546 18003 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.212990 10122 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 16.
I20260709 13:20:09.212945 10095 leader_election.cc:403] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:20:09.221662 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.196638 17857 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:09.224947 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.227167 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.231587 10122 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 19.
I20260709 13:20:09.200680 18075 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:09.230245 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:09.233279 10124 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:09.233989 10019 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.240301 10899 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:09.242305 17852 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.212960 17812 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.255426 10019 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.260600 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.264658 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:09.265179 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 183136us
I20260709 13:20:09.287873 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.230228 17947 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775) with higher term. Message: Invalid argument: T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f for earlier term 21. Current term is 22.
I20260709 13:20:09.300364 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:09.291082 10019 leader_election.cc:400] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
I20260709 13:20:09.198736 17991 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.302487 10019 leader_election.cc:403] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 13:20:09.257390 18099 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.412s	user 0.000s	sys 0.003s
W20260709 13:20:09.259011 18110 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:20:09.220890 17984 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.274781 18001 raft_consensus.cc:3055] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:20:09.324608 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:09.280329 17977 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.249818 18109 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.265483 17876 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:09.275091 17812 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:09.338771 10096 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.343858 10096 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.232710 17879 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.305091 18101 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:09.265681 18063 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:20:09.358791 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.371963 10124 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 18.
I20260709 13:20:09.373855 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:09.374788 10124 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
W20260709 13:20:09.359903 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 197559us
I20260709 13:20:09.272058 17852 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.327365 18115 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:09.332116 18001 raft_consensus.cc:740] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:09.324462 17973 consensus_queue.cc:1048] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:20:09.393896 10124 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:09.367540 17812 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.413161 10124 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:20:09.412410 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 79871.3us
W20260709 13:20:09.416206 10096 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.416849 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.375447 17977 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:09.419847 10020 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:09.388859 17876 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:09.441504 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 154241us
I20260709 13:20:09.444394 10019 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.358335 18109 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.457893 10127 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:09.447855 10128 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45470 (request call id 955) took 1189 ms. Trace:
I20260709 13:20:09.451020 17987 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.452984 17984 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.453876 18003 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.368093 18101 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.429664 18116 raft_consensus.cc:3060] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:09.458866 10128 rpcz_store.cc:276] 0709 13:20:08.258095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.434881 (+176786us) service_pool.cc:224] Handling call
0709 13:20:08.456385 (+ 21504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.969087 (+512702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.969146 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.969898 (+   752us) log.cc:844] Serialized 52 byte log entry
0709 13:20:09.038764 (+ 68866us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b48000b22d8. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27cf2daa1 00007fa27cf24b37 00007fa27cf249ca 00007fa27d01c26f 00007fa27d01bf37 00007fa27d03055c 00007fa27d030508 00007fa27d030488 00007fa27d030443 00007fa27d02f6ec 00007fa27bf59a96
0709 13:20:09.094716 (+ 55952us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:20:09.287681 (+192965us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.291970 (+  4289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.292454 (+   484us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.444543 (+152089us) raft_consensus.cc:1685] finished
0709 13:20:09.447556 (+  3013us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.447750 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382246,"prepare.queue_time_us":169166,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":487375,"raft.run_cpu_time_us":3057,"raft.run_wall_time_us":112746,"spinlock_wait_cycles":52891392,"thread_start_us":395489,"threads_started":3,"wal-append.queue_time_us":194261}
W20260709 13:20:09.470628 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:09.482300 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 105331us
I20260709 13:20:09.440089 17947 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:09.497706 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 107880us
I20260709 13:20:09.464334 18124 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.472817 18102 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:09.465251 18115 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.505039 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:09.478475 18004 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:09.434100 17876 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:09.506536 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90360.2us
W20260709 13:20:09.529062 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.511498 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.508033 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.443790 17812 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.530315 10204 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:20:09.542255 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.537155 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:09.543514 10897 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:20:09.545840 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 126047us
I20260709 13:20:09.543519 10200 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:09.499748 17984 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.549369 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.500152 18063 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.516571 18102 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:09.499768 18125 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:09.528169 18064 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:09.548197 18110 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:20:09.578213 17879 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:09.580227 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 51219.8us
W20260709 13:20:09.581662 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.582219 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.480901 18001 consensus_queue.cc:260] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.574553 18063 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.594841 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.595912 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:09.597190  8925 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 10081)
W20260709 13:20:09.598179 18114 log.cc:927] Time spent T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 13:20:09.604107 18121 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:20:09.610244 10096 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.611743 18063 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:09.611763 10096 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.612958 10096 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.614120 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:09.618296 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:09.619506 10020 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:09.620886 18126 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:09.621487 18126 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.621920 18126 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:09.536085 18115 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.625653 18109 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:09.626286 17812 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.627079 17991 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.629290 17876 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.635492 17852 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:09.645380 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 59626.4us
I20260709 13:20:09.652091 18016 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:09.671869 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 60129.8us
W20260709 13:20:09.673063 10096 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.673607 10096 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:09.676365 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 56938.7us
I20260709 13:20:09.653913 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 
I20260709 13:20:09.587483 18125 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:09.522678 18124 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.560947 17984 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:09.707974 10096 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.709105 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:09.710361 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 173353us
I20260709 13:20:09.684343 18001 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:09.719089 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.680656 17987 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:09.728610 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.742586 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:09.699209 18120 log.cc:927] Time spent T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.063s	user 0.006s	sys 0.006s
I20260709 13:20:09.694880 18128 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.754415 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.717319 18016 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.734638 18003 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:20:09.765785 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 57861.4us
W20260709 13:20:09.764950 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 320612us
W20260709 13:20:09.766870 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.767575 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.767927 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:09.769312 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 115502us
I20260709 13:20:09.742285 17812 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:09.774331 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 273102us
W20260709 13:20:09.776551 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.751754 18105 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.779992 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.777210 10096 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.742967 17947 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:09.694403 18102 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.784879 17812 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.771261 18003 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.786824 10019 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.788775 10899 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:09.807402 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:09.750265 17949 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:20:09.810859 18075 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.778512 18016 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:09.814528 10019 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.785195 18115 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:09.807348 17987 consensus_queue.cc:1175] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d51ad94710c348889d417f7f00032948" responder_term: 18 status { last_received { term: 18 index: 14 } last_committed_idx: 14 last_received_current_leader { term: 18 index: 14 } }
I20260709 13:20:09.796577 18128 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:09.816246 17984 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.786855 17947 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:09.824482 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.802186 18003 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:09.797535 18102 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:09.826210 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.831058 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.831668 10096 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.825070 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.790247 18132 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:09.787460 18125 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:09.830080 18128 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:09.801867 17812 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.818714 18063 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:09.839759 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.841655 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.841703 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.832851 18016 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:09.843258 18016 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.832988 17721 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:20:09.843320 18136 consensus_queue.cc:1059] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:20:09.848603 18075 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:09.836678 18135 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:20:09.855672 10096 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.856341 10096 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:09.865494 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.866078 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:09.867329 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.874087 18075 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:09.878077 18101 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.867910 10096 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.886225 18124 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.889865 17947 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:09.894234 17947 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:09.835445 17984 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:09.826329 18070 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
W20260709 13:20:09.898928 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:09.908129 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 66513.6us
I20260709 13:20:09.911751 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.910768 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.911610 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.912708 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.833767 18118 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:09.928483 10019 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.929090 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.940591 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.940822 10020 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:09.843948 18016 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:09.943547 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.944491 10205 raft_consensus.cc:2393] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 22: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:09.964380 10019 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:09.965884 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52760.2us
I20260709 13:20:09.970077 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.971904 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:09.960544 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:09.994123 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 95247.9us
I20260709 13:20:09.995880 18128 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:20:09.996014 10096 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:09.996753 10096 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:09.996538 18134 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 13:20:09.991597 10128 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:09.923791 18136 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:09.941959 18132 raft_consensus.cc:3055] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:09.994783 17812 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:09.958964 18004 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:10.013404 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 49176.4us
I20260709 13:20:10.009678 10125 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45986 (request call id 3655) took 1016 ms. Trace:
I20260709 13:20:10.018601 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:09.960016 18001 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:09.959636 18124 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.019060 10125 rpcz_store.cc:276] 0709 13:20:08.993264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.994855 (+  1591us) service_pool.cc:224] Handling call
0709 13:20:09.007854 (+ 12999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.179891 (+172037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.179929 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.188137 (+  8208us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.353572 (+165435us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:20:09.407302 (+ 53730us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.407970 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.467407 (+ 59437us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b54003d4d54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa278004e01 00007fa27800383e 00007fa27cfc8419 00007fa27653fbd4 00007fa27653fb62 00007fa27653fa91 00007fa27653fa0c 00007fa27653ecc2 00007fa27633f48c 00007fa27633e8c6
0709 13:20:09.467490 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.630280 (+162790us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x7b18002a2920. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa27dddfde1 00007fa27ddec5b1 00007fa274f173f3 00007fa274f197c1 00007fa274f19779 00007fa274f19709 00007fa274f196d1 00007fa274f189cd 000000000038f354 000000000038f199 00007fa274efa83a
0709 13:20:10.009416 (+379136us) raft_consensus.cc:1685] finished
0709 13:20:10.009444 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.009599 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119712,"prepare.queue_time_us":58372,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":3306,"raft.queue_time_us":112687,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":14796,"spinlock_wait_cycles":149965312,"thread_start_us":130496,"threads_started":1,"wal-append.queue_time_us":250546}
I20260709 13:20:10.041709 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.045535 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.046988 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.056960 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.061331 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:10.062005 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.009905 18132 raft_consensus.cc:740] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:10.079916 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.015822 18022 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:10.084826 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:10.072021 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.075938 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:10.086360 10128 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:10.001545 18118 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.060715 17812 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.096458 17812 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.060974 17876 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:10.060446 18125 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:10.013003 18128 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
W20260709 13:20:10.076900 18123 log.cc:927] Time spent T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
W20260709 13:20:10.069216 18116 consensus_meta.cc:287] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Time spent flushing consensus metadata: real 0.570s	user 0.014s	sys 0.000s
I20260709 13:20:10.092036 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.060621 18124 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.095345 18038 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.125948 18038 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.126842 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.127277 18105 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:10.078102 18064 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:10.127942 18105 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.128337 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.128290 18128 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.129339 10898 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:10.099566 17876 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.130841 18105 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.067752 18122 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.132220 18116 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 21. Current term is 22.
W20260709 13:20:10.078460 18121 log.cc:927] Time spent T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:20:10.090638 18022 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.145676 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.110163 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:10.117425 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 121437us
W20260709 13:20:10.151183 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:10.152649 10096 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.153162 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.121809 18104 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:10.131892 18016 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.178666 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:10.183637 10096 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.188723 10096 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:10.190373 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104451us
I20260709 13:20:10.171001 17812 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:10.191504 18126 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: Time spent flushing consensus metadata: real 0.569s	user 0.016s	sys 0.000s
I20260709 13:20:10.188717 18116 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:10.228338 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44747.8us
I20260709 13:20:10.232607 18104 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:10.234910 18118 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:10.186033 18132 consensus_queue.cc:260] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.190183 17949 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:10.245322 10128 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:10.193300 18016 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:10.244939 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:10.257421 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.257963 10096 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.225160 18064 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:10.265246 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.266121 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.246852 17812 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:10.281251 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.281899 10096 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.297786 10863 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:10.300220 18132 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:10.247045 18126 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.250535 18119 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 13:20:10.293565 18104 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:10.334061 10019 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.252465 18116 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.359627 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:10.410933 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 76976.1us
I20260709 13:20:10.326612 17947 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.308785 18016 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:10.411600 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75594.5us
I20260709 13:20:10.416198 10019 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.419090 17879 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:10.419476 10019 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.420735 17879 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.420846 10019 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.422220 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.423233 17984 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.423452 17879 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.425827 10019 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.262599 17949 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.325587 18118 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:10.275535 18064 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.308336 18022 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.449666 10204 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 23: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:10.451547 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.452567 10128 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:20:10.470069 18104 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:20:10.471436 18143 log.cc:927] Time spent T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:20:10.471058 18104 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.294193 17812 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:10.474505 17812 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.475862 18104 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.496097 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.427975 18125 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.441959 10019 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.443262 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.444171 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.445195 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.446180 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:10.440543 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.427105 17879 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:10.498558 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.503074 10096 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:10.504402 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 62604.5us
I20260709 13:20:10.505355 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:10.497956 18146 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.333s	user 0.002s	sys 0.000s
I20260709 13:20:10.449381 18118 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:10.510747 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:10.502233 18003 consensus_meta.cc:287] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.002s
I20260709 13:20:10.514969 10019 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:10.507910 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 67441.3us
I20260709 13:20:10.510565 18122 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.521972 17949 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:10.509383 18022 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.523334 18116 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:10.519407 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.536440 10096 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:10.537437 10019 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.508548 18016 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.512914 18118 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:10.517336 18125 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:10.554524 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:10.538669 18116 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:10.570629 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.571637 10126 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
I20260709 13:20:10.572348 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.542747 18104 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.529140 18003 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.559517 17879 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.568259 18126 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.565783 18122 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.596163 10897 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:10.596017 18122 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:10.597754  8925 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:10.598497 18122 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.595358 18003 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.640141 18064 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.657581 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:10.658689 10128 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:10.672731 17812 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.673975 17812 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.677932 17812 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.680488 18124 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.683045 18157 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.686162 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:10.687832 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:10.690721 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:10.690430 18157 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.642095 18118 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:10.662483 18154 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:20:10.736114 10122 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:10.738127 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.736219 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:10.737398 10096 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.740661 10096 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.747882 10128 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
W20260709 13:20:10.750969 10096 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.751922 10096 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:10.753942 10096 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.755955 10096 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.747925 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:10.767026 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.769455 10096 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.778730 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:20:10.781009 10096 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.782559 10096 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.783748 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.787425 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:10.788298 10020 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.789551 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.787526 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.761041 18102 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:10.757545 18101 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.794962 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:10.799762 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.771907 18004 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.771248 17812 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:10.822424 10897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42822 (request call id 599) took 1015 ms. Trace:
I20260709 13:20:10.822962 10897 rpcz_store.cc:276] 0709 13:20:09.806884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.807033 (+   149us) service_pool.cc:224] Handling call
0709 13:20:10.822173 (+1015140us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54004aef54. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:10.822353 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807875328}
I20260709 13:20:10.823691 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.808501 10205 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:10.768610 17876 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.776512 18104 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:10.830816 10204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57102 (request call id 2683) took 1097 ms. Trace:
I20260709 13:20:10.831419 10204 rpcz_store.cc:276] 0709 13:20:09.727394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.728217 (+   823us) service_pool.cc:224] Handling call
0709 13:20:10.824446 (+1096229us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54001ff254. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:20:10.824579 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1599140224}
I20260709 13:20:10.835273 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.836869 17876 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.837539 17876 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.840209 17876 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.841732 17879 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.842340 17879 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.844705 17879 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:10.845314 17879 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:10.846892 10020 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:10.848662 18157 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.854926 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:10.858316 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 77346.2us
W20260709 13:20:10.859596 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.822687 17812 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.860795 10096 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.865666 10897 raft_consensus.cc:1240] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:10.876807 10863 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:10.852979 18118 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.857098 18101 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.857906 18122 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:10.867519 17812 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:10.890935 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.893550 10129 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 19.
I20260709 13:20:10.870186 18038 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:10.898540 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:10.900704 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:10.891872 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.873538 17947 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.908525 10204 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 23: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:10.908499 10096 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.909446 10131 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:10.913280 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:10.914862 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:10.931977 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:10.932274 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:10.880448 18064 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:20:10.940336 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:10.941378 10128 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 23: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
W20260709 13:20:10.944124 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 282516us
I20260709 13:20:10.890337 18159 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:10.947907 18159 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:10.949052 18159 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:10.953619 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:10.963762 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:10.965179 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 73345.6us
W20260709 13:20:10.967103 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:10.978189 10096 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:10.976950 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.008184 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.013181 18128 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:11.046716 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111238us
W20260709 13:20:11.051947 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 106092us
W20260709 13:20:11.053270 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86211.3us
I20260709 13:20:11.054701 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:11.056609 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 143436us
I20260709 13:20:11.053975 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:11.056529 18151 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:20:11.015206 17947 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:11.068400 18157 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:11.070874 17973 consensus_queue.cc:1059] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:20:11.106604 18157 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.107019 18157 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:11.071734  9948 catalog_manager.cc:5697] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 21 to 24, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 24 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:11.108544 17721 raft_consensus.cc:3055] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:11.072034 17661 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:11.109233 17721 raft_consensus.cc:740] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:11.072541 18161 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 13:20:11.088914 18160 log.cc:927] Time spent T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:20:11.045485 17876 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:11.097942 18164 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:11.115536 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45391us
I20260709 13:20:11.058298 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.005882 18016 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:11.123795 17876 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.124564 17876 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.128504 18128 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:11.129343 18128 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.130177 17812 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.060976 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:11.143214 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 82309.9us
W20260709 13:20:11.143551 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 127614us
I20260709 13:20:11.144454 10020 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:11.146109 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:11.146865 10096 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:11.149500 10096 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:11.150359 10096 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.136713 17876 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.162338 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.164932 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.110494 17721 consensus_queue.cc:260] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.167076 10020 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.169277 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.156189 18064 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:11.155234 18016 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:11.163044 18171 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:11.165242 18104 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:11.169566 17879 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:11.173447 18159 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:20:11.203388 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144708us
I20260709 13:20:11.172070 18132 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.175969 18063 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:11.187225 17721 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:11.222579 18064 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:20:11.224784 18124 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:11.240419 18128 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.222087 18132 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.244177 18164 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:11.209582 18165 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:11.266568 18166 log.cc:927] Time spent T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.325s	user 0.004s	sys 0.000s
I20260709 13:20:11.275373  9948 catalog_manager.cc:5697] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 18 to 24, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 24 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:11.277673  9948 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:11.289527  9948 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 17 to 20, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:20:11.325809 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 176368us
I20260709 13:20:11.349745  9948 catalog_manager.cc:5697] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:11.373413 18174 log.cc:927] Time spent T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:20:11.382099  9946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 208) took 68780 ms (client timeout 15000 ms). Trace:
I20260709 13:20:11.400260 18132 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.401993  9948 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 5 to 10, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 15 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:20:11.422735 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 256569us
W20260709 13:20:11.382712  9946 rpcz_store.cc:269] 0709 13:19:02.601657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.602011 (+   354us) service_pool.cc:224] Handling call
0709 13:20:11.381962 (+68779951us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:05.504410 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:05.737784 (+233374us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:05.778907 (+ 41123us) write_op.cc:435] Acquired schema lock
0709 13:20:05.791659 (+ 12752us) tablet.cc:662] Decoding operations
0709 13:20:06.603770 (+812111us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:06.603896 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:06.603929 (+    33us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:20:06.605915 (+  1986us) tablet.cc:701] Row locks acquired
0709 13:20:06.605979 (+    64us) write_op.cc:260] PREPARE: finished
0709 13:20:06.606189 (+   210us) write_op.cc:270] Start()
0709 13:20:06.606381 (+   192us) write_op.cc:276] Timestamp: P: 1783603206606103 usec, L: 0
0709 13:20:06.606429 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:20:06.712355 (+105926us) log.cc:844] Serialized 8624 byte log entry
0709 13:20:07.158547 (+446192us) op_driver.cc:464] REPLICATION: finished
0709 13:20:07.204969 (+ 46422us) write_op.cc:301] APPLY: starting
0709 13:20:07.205094 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:08.187769 (+982675us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:08.266723 (+ 78954us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:09.455182 (+1188459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:09.455653 (+   471us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:20:09.455749 (+    96us) write_op.cc:312] APPLY: finished
0709 13:20:09.456373 (+   624us) log.cc:844] Serialized 196 byte log entry
0709 13:20:09.552127 (+ 95754us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:09.552403 (+   276us) write_op.cc:454] Released schema lock
0709 13:20:09.559857 (+  7454us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:09.598825 (+ 38968us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:09.598869 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":169,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":46026,"apply.run_cpu_time_us":103065,"apply.run_wall_time_us":2380692,"cfile_cache_hit":90,"cfile_cache_hit_bytes":95911,"num_ops":22,"prepare.queue_time_us":258629,"prepare.run_cpu_time_us":26617,"prepare.run_wall_time_us":1511549,"raft.queue_time_us":83254,"raft.run_cpu_time_us":5540,"raft.run_wall_time_us":24998,"replication_time_us":551741,"thread_start_us":371708,"threads_started":5,"wal-append.queue_time_us":392229,"wal-append.run_cpu_time_us":54379,"wal-append.run_wall_time_us":2147951}]]}
W20260709 13:20:11.427806 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 243889us
I20260709 13:20:11.438649 10125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:11.439460 10125 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:11.440613 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 271423us
I20260709 13:20:11.447556 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.450867 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.449473 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.424412  9948 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:11.452867 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.450076 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.434345 18132 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.437960 18128 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.466050 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.466403 18128 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.438253 17879 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.463089 10201 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:11.467549 17879 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.468974 18128 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.466050  9948 catalog_manager.cc:5697] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:11.470069 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.470227 17879 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.438103 17947 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:11.470804  9948 catalog_manager.cc:5697] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 13 to 25, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 25 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:11.471596 17947 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.473340  9948 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 17 to 20, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:11.474421 17947 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.477095 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.417747 18157 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.495922 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.498040 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.499321 10128 raft_consensus.cc:2410] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:20:11.498957 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:11.499902 10201 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:11.497495 10125 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:20:11.501080 10204 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:11.507537 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.511919 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.513872 10019 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:11.492175 17879 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.529520 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.498155 18124 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.499518 18128 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:11.521322 18179 log.cc:927] Time spent T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:20:11.560423 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 120264us
W20260709 13:20:11.560429 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46712.3us
I20260709 13:20:11.563625 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:11.565248 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 115333us
I20260709 13:20:11.567425 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.520359 18063 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.493626 18132 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.589386 10201 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:20:11.590284 10204 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:11.591012 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.592032 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.560170 18124 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:11.598421  8925 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:11.606717 10199 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
I20260709 13:20:11.609728 10896 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:11.562623 18128 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:11.620431 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 48834.8us
I20260709 13:20:11.578153 18185 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:11.578249 18183 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:11.621778 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.569139 17879 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.591846 17876 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:11.642855 10126 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:11.578272 18063 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.653483 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.656894 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.662077 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:11.665189 10126 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:11.663082 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 46092.4us
I20260709 13:20:11.646993 18070 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:11.666075 18157 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.667275 18102 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:11.659480 18193 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:11.720386 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:11.726297 18198 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.666901 18187 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:20:11.679286 17876 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:20:11.729768 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 60038.6us
W20260709 13:20:11.734730 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 81341.5us
I20260709 13:20:11.693917 18102 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:11.740120 18102 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:11.719763 17947 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:11.746351 17947 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.747301 18200 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:11.747583 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 145567us
I20260709 13:20:11.749130 17947 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.683723 18070 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.752310 18070 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.752657 18102 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.754082 18132 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.756072 18172 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:11.757042 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.756879 18172 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.726557 10126 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:20:11.760537 18132 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:11.763217 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:11.764362 18172 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.766952 18186 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:11.729039 18193 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:11.769575 18102 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:11.770141 18063 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.770457 18122 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:11.770704 17879 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:11.772045 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.778509 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:11.777666 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:11.779351 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.779966 10896 raft_consensus.cc:2393] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:11.773186 10863 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:11.780612 10199 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:11.781352 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.790252 10202 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:11.802054 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.814793 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.816318 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:11.814900 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:11.832219 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 172666us
I20260709 13:20:11.791000 17876 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:11.789309 18171 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:11.866312 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:11.865191 10197 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
W20260709 13:20:11.867659 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 300390us
I20260709 13:20:11.810458 18157 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:20:11.867084 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:11.870119 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:11.811975 17879 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.882763 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.883812 10123 raft_consensus.cc:2410] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 13:20:11.881968 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:11.887266 10865 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:11.886577 10199 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:20:11.886998 18118 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:11.856849 18004 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:11.903451 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.906131 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.917760 10128 raft_consensus.cc:2368] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20.
I20260709 13:20:11.920085 18122 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.920759 18122 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:11.863744 18185 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:11.925128 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.926234 10128 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 20.
I20260709 13:20:11.932071 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:11.819087 18198 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:11.887488 18157 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:11.927726  9943 maintenance_manager.cc:419] P db0c98fae3aa4b45a39c8914c5e641cc: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033342
I20260709 13:20:11.952066 10168 leader_election.cc:400] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20.
I20260709 13:20:12.030843 10168 leader_election.cc:403] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:12.031595 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88159.2us
I20260709 13:20:11.902168 17879 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:12.024765 10170 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:12.034924 10170 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.035867 17366 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 for earlier term 19. Current term is 20.
I20260709 13:20:11.878846 18171 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.030897 18122 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:12.029615 18157 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.050184 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:12.052309 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 170371us
I20260709 13:20:12.018504 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.954939 10019 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:11.930212 18004 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:12.029229  9939 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 806f35e66c234d569cbafd8b3981518f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 5 but the committed config has opid_index of 10
I20260709 13:20:12.015856 17875 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:11.945998 18159 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:11.925609 18204 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.940665 17812 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.053133 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:11.933387 18128 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:12.058904 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:11.961251 18198 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.036720 17366 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:11.998026 18173 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
W20260709 13:20:12.066460 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 111633us
I20260709 13:20:11.959040 18124 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.036399 17879 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:12.038718 18171 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:12.095538 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 218496us
I20260709 13:20:12.096958 10095 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:12.065001 18122 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.098685 10020 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.102281 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.114981 10204 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
W20260709 13:20:12.085363 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:12.115700 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:12.075927 18177 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:12.107702 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.124281 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.125042 10198 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:12.111553 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.083257 18183 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.117352 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.121815 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.142112 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.134496 10198 raft_consensus.cc:2368] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:12.146241 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.148910 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.149645 10129 raft_consensus.cc:1813] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Handling vote request from an unknown peer 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:12.151010 10129 raft_consensus.cc:2368] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:12.124445 18177 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:12.128429 18212 log.cc:927] Time spent T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:20:12.121965 17812 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:12.148404 18063 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:12.128773 18171 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:12.452195 18171 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:12.156363 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:12.153682 18211 log.cc:927] Time spent T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: Append to log took a long time: real 0.081s	user 0.000s	sys 0.002s
I20260709 13:20:12.082827 17875 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.088255 18204 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.150866 17879 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:12.491077 17875 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:12.167586 18177 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:12.146736 18122 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.127104 18183 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:12.318373 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 204636us
I20260709 13:20:11.964480 18185 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.411651 17876 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.416857 18209 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:12.420966 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.422281 18004 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.421111 18198 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.538973 18198 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.539144 17876 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.541987 18198 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:12.150846 18200 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.542909 18200 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.442889 18016 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:12.543565 17876 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.544081 18016 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.447279 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:12.453703 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 297457us
I20260709 13:20:12.549574 10019 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.551280 10019 leader_election.cc:400] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:12.551632 10019 leader_election.cc:403] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:12.552212 10202 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:12.453811 18171 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.454440 18063 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:12.103543 18173 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.520346 17879 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:12.520553 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 99682.4us
I20260709 13:20:12.525079 18122 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.528661 18204 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.531857 18183 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:12.564121 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.592293 10123 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:12.593855 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.564437 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.595222 18016 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:12.558938 18186 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:20:12.596184 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.572926 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.573297 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:12.573791 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.573925 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:12.600104 10201 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 21.
W20260709 13:20:12.567626 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49741.3us
W20260709 13:20:12.606860  8925 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:12.613489 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:12.614142 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.616044 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.580829 18132 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:12.579339 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:12.582779 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 65528.6us
I20260709 13:20:12.633862 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:12.585652 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.558820 18063 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:12.588523 18193 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.589632 18173 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.637405 18193 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.640065 18171 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:12.640318 18193 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.642098 10124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.643545 10124 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 22: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:12.578217 10020 leader_election.cc:400] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Vote denied by peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:12.610036 18198 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:12.656631 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 78649us
I20260709 13:20:12.623924 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:12.584414 18209 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.688113 18209 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.691730 18200 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.684571 18198 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:12.696447 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:12.712579 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 88809.4us
W20260709 13:20:12.738857 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 164070us
W20260709 13:20:12.748412 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:12.750293 10096 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:12.753193 10020 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.752993 10019 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:12.753005 10096 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:12.754408 10096 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:12.756604 18016 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:12.763373 17876 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:12.765461 18204 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.766153 18204 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.767333 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.768167 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.768725 18204 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.744354 18193 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:12.769701 18193 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.770119 18193 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:12.779688 18198 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:12.780450 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78082.5us
I20260709 13:20:12.792727 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:12.783025 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:12.795990 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.800987 10020 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.803354 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.803411 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:12.798382 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:12.802609 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.791531 18170 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:12.815691 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:12.795300 18185 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:12.807896 18198 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:12.824101 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:12.822921 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.793299  9942 maintenance_manager.cc:643] P db0c98fae3aa4b45a39c8914c5e641cc: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.762s	user 0.042s	sys 0.034s Metrics: {"bytes_written":4499,"delete_count":0,"lbm_write_time_us":1180,"lbm_writes_lt_1ms":5,"reinsert_count":0,"update_count":176}
I20260709 13:20:12.815183 18064 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:12.826164 18226 log.cc:927] Time spent T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:20:12.847061  9943 maintenance_manager.cc:419] P db0c98fae3aa4b45a39c8914c5e641cc: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.230820
I20260709 13:20:12.845309 18124 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:12.845577 18200 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.864017 18200 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:12.873216 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 76206.5us
I20260709 13:20:12.822593 18177 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.884757 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:12.901463 10863 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.896957 18132 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681) with higher term. Message: Invalid argument: T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 01bfaf1ad56a4deea886bbc64dec466c for earlier term 19. Current term is 20.
I20260709 13:20:12.921945 10020 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.925539 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:12.927875 18177 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.907665 18016 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:12.938714 18122 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:12.907351 18187 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:12.939790 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.941736 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:12.944950 18122 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:12.946115 18193 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.954339 10200 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:12.956198 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:12.957054 10202 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:20:12.956910 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:12.959741 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 112175us
W20260709 13:20:12.961859 10170 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:12.962389 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:12.958981 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:12.963891 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44816.2us
W20260709 13:20:12.965449 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:12.954928 17876 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:12.954133 18193 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:12.974467 10200 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:12.958055 18198 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.985075 10020 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948; no voters: 
I20260709 13:20:12.967229 18064 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.988560 18177 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:13.006927 18177 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:13.008109 18177 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.989483 18198 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:12.999735 18064 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:13.053642 18064 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:13.054471  9948 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 124) took 14584 ms (client timeout 15000 ms). Trace:
I20260709 13:20:13.058573 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.065311 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.037616 18198 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:13.075296 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.073567 10202 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
I20260709 13:20:13.082679 18200 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:13.085350 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:13.116981 10126 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:13.085614 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:13.118211 10096 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:13.090167 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:13.119168 10127 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:13.091571 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:13.092191 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:13.104254 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:13.111368 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.115804 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.122133 18050 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:13.131706 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 46106.1us
W20260709 13:20:13.132872 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:13.133466 10096 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:13.136021 18171 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:13.141316 10019 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.146139 10127 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 23.
W20260709 13:20:13.135087 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.160873 10899 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:13.163367 18063 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:13.166179 18118 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.166908 18118 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:13.149261 18171 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:13.169492 10019 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.169997 18118 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:13.171844 10019 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:13.174135 10201 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:13.174734 10020 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.176255 18132 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:13.176319 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:13.177912 10202 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:13.180913 18171 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:13.182137 18177 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:13.182829 18177 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Becoming Leader. State: Replica: 01bfaf1ad56a4deea886bbc64dec466c, State: Running, Role: LEADER
I20260709 13:20:13.183892 10130 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:13.183885 18177 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:13.186554 18198 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:13.177534 18064 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:13.193413 18064 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
W20260709 13:20:13.195722 10096 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:13.210114 18171 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:20:13.055024  9948 rpcz_store.cc:269] 0709 13:19:58.475907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.462530 (+1986623us) service_pool.cc:224] Handling call
0709 13:20:13.054361 (+12591831us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:11.944922 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:11.960366 (+ 15444us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:11.960416 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:20:11.960438 (+    22us) tablet.cc:662] Decoding operations
0709 13:20:11.961699 (+  1261us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:11.961806 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:11.961843 (+    37us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:20:11.962946 (+  1103us) tablet.cc:701] Row locks acquired
0709 13:20:11.962972 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:20:11.995651 (+ 32679us) write_op.cc:270] Start()
0709 13:20:11.997489 (+  1838us) write_op.cc:276] Timestamp: P: 1783603211963106 usec, L: 0
0709 13:20:11.997536 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:20:11.998411 (+   875us) log.cc:844] Serialized 3954 byte log entry
0709 13:20:12.469909 (+471498us) op_driver.cc:464] REPLICATION: finished
0709 13:20:12.474369 (+  4460us) write_op.cc:301] APPLY: starting
0709 13:20:12.474461 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:12.591030 (+116569us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:12.593517 (+  2487us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:12.781709 (+188192us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:12.781998 (+   289us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=40,mrs_lookups=0
0709 13:20:12.782051 (+    53us) write_op.cc:312] APPLY: finished
0709 13:20:12.782404 (+   353us) log.cc:844] Serialized 97 byte log entry
0709 13:20:12.782626 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:12.782790 (+   164us) write_op.cc:454] Released schema lock
0709 13:20:12.782849 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:12.812752 (+ 29903us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:12.813684 (+   932us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":185014,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":4090,"apply.run_cpu_time_us":21379,"apply.run_wall_time_us":324509,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45847,"cfile_init":1,"lbm_read_time_us":31714,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3,"num_ops":10,"prepare.queue_time_us":286695,"prepare.run_cpu_time_us":66401,"prepare.run_wall_time_us":234432,"raft.queue_time_us":17750,"raft.run_cpu_time_us":1576,"raft.run_wall_time_us":7362,"replication_time_us":472107,"spinlock_wait_cycles":31616,"thread_start_us":363878,"threads_started":4,"wal-append.queue_time_us":236013}]]}
I20260709 13:20:13.216892 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.216995 10096 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:13.220367 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.227836 18170 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:13.235208 10126 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bfaf1ad56a4deea886bbc64dec466c in term 22.
I20260709 13:20:13.239859 18118 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:13.269512 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.301396 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.303618 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.311707 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.314014 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.315449 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.309679  9954 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.347792 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.360124 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.361702 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.355144  9954 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.384506  9954 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.397078  9954 catalog_manager.cc:5697] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.403834 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.409909  9954 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.432734  9954 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.444631 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.446449 18238 log.cc:927] Time spent T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:13.434540  9954 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.473681 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.491148  9954 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 18 to 23, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 23 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:13.546217  9954 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.558642 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 154804us
I20260709 13:20:13.577139 18132 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.578128 18132 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:13.590962 10201 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:20:13.594584 18122 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:13.596825 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.605284 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:13.605554 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52096.1us
W20260709 13:20:13.608639  8925 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:13.615859 10019 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.612957 10123 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:20:13.610431  9954 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 13 to 16, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 16 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:20:13.634608 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64875.3us
W20260709 13:20:13.635871 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.636216  9954 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.643952 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 105959us
I20260709 13:20:13.613091 18132 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:13.634405 18063 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:13.677934  9954 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.639928 18064 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:13.684690  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 144065us
I20260709 13:20:13.692227 10130 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:13.636328 18198 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:20:13.698987 10202 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer 01bfaf1ad56a4deea886bbc64dec466c: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:13.709081  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 16 to 20, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:13.710889  9954 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.713636  9954 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 10 to 13, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 13 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.717994 18063 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:13.718784 18063 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:13.721669 18063 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:13.735324 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.742043 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 78688.5us
I20260709 13:20:13.731477  9954 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.750574 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.753572  9954 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.688052 18064 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:13.731441 18132 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:20:13.771786 18249 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
W20260709 13:20:13.780887  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:20:13.757576  9954 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.796494  9954 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.822320 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.824198 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.826383  9954 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 11 to 16, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 16 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.843837 18250 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.844616 18250 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:13.849493 18250 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:13.853756 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:13.862263 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.864261 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:13.825938 18064 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:13.874799 10020 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:13.889441 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:13.873184 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:13.908203 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:13.909543 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.913070 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.897185 18122 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:13.953787 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.970183 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:13.973361 10019 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:13.977329 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:13.985710 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:13.984443 18198 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:14.046911 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 73406us
I20260709 13:20:14.117929 18063 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:14.127692 18132 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.129125 18132 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:14.152451 10865 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 218 similar messages]
I20260709 13:20:14.155242 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:14.152515 18132 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:14.166512 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:14.186372 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 91407.7us
W20260709 13:20:14.197862 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 115964us
I20260709 13:20:14.130661 18063 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.219666 18063 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:14.224747 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:14.234246 18132 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.235577 18132 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.227196 18063 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:14.259344 18260 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.261152 18260 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.265559 18260 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.275244 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.282461 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:14.284415 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:14.285416 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.307344 18260 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:14.308300 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.308645 18260 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.307556 18132 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.312669 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.316901 18177 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.318226 18177 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.320645 18260 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:14.350055 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:14.354182 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.378163 18177 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.413625 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.414925 10020 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:14.416700 10020 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:14.416971 18198 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:14.417843 18122 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:14.419697 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.424319 18064 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:14.436685 18064 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:14.437533 18064 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.452653 18063 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.453649 18132 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.454483 18132 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:14.453792 18063 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.457315 18132 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:14.459580 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:14.476183 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.483995 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.498831 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.503825 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.518177 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.541286 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:14.548136 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:14.550415 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:14.567874 18064 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.582279 10020 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.589356 18063 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.590251 18122 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:14.592522 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.593262 10020 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:14.592589 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:14.595526 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:14.600252 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:14.600942 18122 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.601135 10020 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:14.601558 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:14.609531  8925 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:14.663932 18064 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:14.664124 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 63118.6us
I20260709 13:20:14.665755 10020 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:14.666769 18064 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:14.667912 10020 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.668915 18063 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.669539 10020 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.671226 18063 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:14.762255 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:14.770735 18064 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.771843 18064 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.781633 18063 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.782467 18063 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.785213 18063 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.790499 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.802073 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.806625 18064 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.814812 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.820438 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:14.841902 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:14.845196 10020 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:14.843294 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:14.843235 18063 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:14.853186 18122 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.853394 18063 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.864975 10019 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.866065 18122 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:14.873266 18063 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.873622 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.884331 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.903352 10020 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:14.905122 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:14.905447 18064 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.908825 18064 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.909651 18064 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.918169 18064 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.920682 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.922235 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:14.932791 10019 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:14.934437 18122 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:14.934751 18260 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.935649 18260 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.938655 18260 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:14.946201 18260 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.941197 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.965543 18260 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:14.996940 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.000764 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:14.976588  9942 maintenance_manager.cc:643] P db0c98fae3aa4b45a39c8914c5e641cc: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.126s	user 0.107s	sys 0.067s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":25384,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":60921,"dirs.run_cpu_time_us":951,"dirs.run_wall_time_us":9206,"lbm_read_time_us":3857,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":19,"lbm_write_time_us":2463,"lbm_writes_lt_1ms":10,"peak_mem_usage":265196,"reinsert_count":0,"thread_start_us":113499,"threads_started":2,"update_count":187}
I20260709 13:20:15.011103  9943 maintenance_manager.cc:419] P db0c98fae3aa4b45a39c8914c5e641cc: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:20:15.014358 18260 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:15.015455 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.017465 10020 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.018831 18064 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.019590  9942 maintenance_manager.cc:643] P db0c98fae3aa4b45a39c8914c5e641cc: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 13:20:15.020670 18063 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:15.019606 18064 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.030058 18064 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:15.030171 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.034415 10020 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.037379 18260 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:15.062458 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.089962 10020 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.094727 18260 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:15.248176  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:20:15.336246  9946 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 11 to 16, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 16 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:15.378306 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:15.399665  9946 catalog_manager.cc:5697] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.403674  9946 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.425740  9946 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:15.401849  9954 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 209) took 33644 ms (client timeout 15000 ms). Trace:
W20260709 13:20:15.435750  9954 rpcz_store.cc:269] 0709 13:19:41.758004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.392194 (+634190us) service_pool.cc:224] Handling call
0709 13:20:15.401600 (+33009406us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:14.011896 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:14.012665 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:14.012717 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:20:14.012742 (+    25us) tablet.cc:662] Decoding operations
0709 13:20:14.099809 (+ 87067us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:14.100044 (+   235us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:14.100139 (+    95us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:20:14.141155 (+ 41016us) tablet.cc:701] Row locks acquired
0709 13:20:14.141244 (+    89us) write_op.cc:260] PREPARE: finished
0709 13:20:14.141768 (+   524us) write_op.cc:270] Start()
0709 13:20:14.141989 (+   221us) write_op.cc:276] Timestamp: P: 1783603214141599 usec, L: 0
0709 13:20:14.142054 (+    65us) op_driver.cc:348] REPLICATION: starting
0709 13:20:14.160137 (+ 18083us) log.cc:844] Serialized 7854 byte log entry
0709 13:20:14.443043 (+282906us) op_driver.cc:464] REPLICATION: finished
0709 13:20:14.488989 (+ 45946us) write_op.cc:301] APPLY: starting
0709 13:20:14.489176 (+   187us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:14.718629 (+229453us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:14.718764 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:14.989840 (+271076us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:14.990400 (+   560us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:20:14.990471 (+    71us) write_op.cc:312] APPLY: finished
0709 13:20:15.042738 (+ 52267us) log.cc:844] Serialized 180 byte log entry
0709 13:20:15.049642 (+  6904us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:15.049877 (+   235us) write_op.cc:454] Released schema lock
0709 13:20:15.049986 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:15.050680 (+   694us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:15.050717 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1136,"reported_tablets":52,"spinlock_wait_cycles":19840,"child_traces":[["op",{"apply.queue_time_us":45477,"apply.run_cpu_time_us":34333,"apply.run_wall_time_us":561638,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87567,"cfile_init":1,"lbm_read_time_us":553,"lbm_reads_lt_1ms":4,"num_ops":20,"prepare.queue_time_us":114954,"prepare.run_cpu_time_us":41459,"prepare.run_wall_time_us":221672,"raft.queue_time_us":137857,"raft.run_cpu_time_us":2306,"raft.run_wall_time_us":26077,"replication_time_us":300744,"thread_start_us":304392,"threads_started":4,"wal-append.queue_time_us":139788}]]}
I20260709 13:20:15.434051  9946 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.460999  9946 catalog_manager.cc:5697] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 23 to 24, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 24 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.462734 18063 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.480129 18063 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.482867 18063 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:15.476547 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:15.483671 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.485478 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.488296 10020 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.490031 18063 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:15.499615  9946 catalog_manager.cc:5697] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.512163  9946 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 16 to 20, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.537739  9946 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.553369  9946 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 16 to 20, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.565831  9946 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.587275 18063 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.593326 18063 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:15.596330 18063 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:15.597613 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:15.600184 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.602039 10019 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.599174  9946 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 15 to 19, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 19 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.603148 18063 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:15.605305  9946 catalog_manager.cc:5697] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:20:15.610215  8925 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 10081)
I20260709 13:20:15.622881  9946 catalog_manager.cc:5697] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 7 to 17. New cstate: current_term: 17 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.632580  9946 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 18 to 19, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 19 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.639580  9946 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:15.652123 18063 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.652935 18063 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.665612 18063 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:15.666628 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.670439 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.675490 18063 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.676527 18063 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.681185 10020 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.683219 18260 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:15.684489 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.686048  9946 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 22 to 24, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 24 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.687561 18260 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:15.711899 18260 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.692492 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.715032 18260 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:15.716495 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.698841 18063 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:15.721004 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.722543  9946 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c reported cstate change: term changed from 16 to 18, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 18 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.723469 10020 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.729506 10020 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.730922 18260 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:15.731805 18260 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:15.725865 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.791018 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.800137 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.846524 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.858354 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.888157 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.896682 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.906708 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.912220 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.918820 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.973495 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:15.985920 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:15.986722 18063 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.987528 18063 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:15.991909 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:15.997803 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:15.999799 10020 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:15.997361 18063 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.000773 18270 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:16.027765 18063 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:16.028527 18063 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.033612 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.039294 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.030844 18063 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.047336 10020 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.053568 18270 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:16.054380 18260 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.055253 18260 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.066799 18260 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.073231 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.081631 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:16.084645 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.085717 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.087281 18270 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:16.091194 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.153046 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:16.179163 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.240631 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.310106 18260 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.310978 18260 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.313936 18260 leader_election.cc:290] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:16.315019  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:20:16.328179 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.331853 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.334856 10019 leader_election.cc:304] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.344627 18063 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:16.347714  9954 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.349507  9954 catalog_manager.cc:5697] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 6 to 21. New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.351008  9954 catalog_manager.cc:5697] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.352703  9954 catalog_manager.cc:5697] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.354287  9954 catalog_manager.cc:5697] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.364765  9954 catalog_manager.cc:5697] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.366341  9954 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.367725  9954 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 16 to 22, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 22 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.369635  9954 catalog_manager.cc:5697] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:20:16.371948  9946 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 225) took 11269 ms (client timeout 15000 ms). Trace:
W20260709 13:20:16.378381  9946 rpcz_store.cc:269] 0709 13:20:05.102405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.577412 (+6475007us) service_pool.cc:224] Handling call
0709 13:20:16.371854 (+4794442us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:15.812149 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:15.812859 (+   710us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:15.812926 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:20:15.812949 (+    23us) tablet.cc:662] Decoding operations
0709 13:20:15.814593 (+  1644us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:15.814690 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:15.814722 (+    32us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:20:15.856178 (+ 41456us) tablet.cc:701] Row locks acquired
0709 13:20:15.856201 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:20:15.856387 (+   186us) write_op.cc:270] Start()
0709 13:20:15.856545 (+   158us) write_op.cc:276] Timestamp: P: 1783603215856329 usec, L: 0
0709 13:20:15.856570 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:20:15.857458 (+   888us) log.cc:844] Serialized 7060 byte log entry
0709 13:20:15.900203 (+ 42745us) op_driver.cc:464] REPLICATION: finished
0709 13:20:15.923372 (+ 23169us) write_op.cc:301] APPLY: starting
0709 13:20:15.923696 (+   324us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:16.006235 (+ 82539us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:16.006366 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:16.236269 (+229903us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:16.236629 (+   360us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:20:16.236677 (+    48us) write_op.cc:312] APPLY: finished
0709 13:20:16.237138 (+   461us) log.cc:844] Serialized 164 byte log entry
0709 13:20:16.237386 (+   248us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:16.237626 (+   240us) write_op.cc:454] Released schema lock
0709 13:20:16.250238 (+ 12612us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:16.250787 (+   549us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:16.256352 (+  5565us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79596,"reported_tablets":53,"spinlock_wait_cycles":10624,"child_traces":[["op",{"apply.queue_time_us":19887,"apply.run_cpu_time_us":30014,"apply.run_wall_time_us":327405,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79223,"num_ops":18,"prepare.queue_time_us":20136,"prepare.run_cpu_time_us":6277,"prepare.run_wall_time_us":65238,"replication_time_us":43430,"thread_start_us":51261,"threads_started":2}]]}
I20260709 13:20:16.371068  9954 catalog_manager.cc:5697] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 21, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 21 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.391451  9954 catalog_manager.cc:5697] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.400319  9954 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.408119  9954 catalog_manager.cc:5697] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.433079  9954 catalog_manager.cc:5697] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 16 to 21, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 21 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.444761  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:16.449756  8925 tablet_replica.cc:333] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.453320  8925 raft_consensus.cc:2243] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.457808 18063 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.455042  8925 raft_consensus.cc:2272] T 87da05694091470284f66a8181207e2f P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.458743 18063 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:16.472020  9954 catalog_manager.cc:5697] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.473472 18063 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:16.474759  8925 tablet_replica.cc:333] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.476275 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:16.475587 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.486411 10020 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.488688 18063 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:16.497946 18270 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.498719  8925 raft_consensus.cc:2243] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.504922  8925 pending_rounds.cc:70] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:16.498994 18270 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:16.478935  9954 catalog_manager.cc:5697] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.532147  9954 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 8 to 13, term changed from 12 to 20, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68), VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 20 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.509092  8925 pending_rounds.cc:77] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 24 index: 16 } timestamp: 7305638680382316544 op_type: NO_OP noop_request { }
I20260709 13:20:16.545450  8925 raft_consensus.cc:2889] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:16.547616  8925 raft_consensus.cc:2272] T ece395e49d7e474182be2d25642b90de P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:16.557346 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.559923 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.584156 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.546703  9954 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 15 to 18, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 18 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.588512 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:16.593973  8925 tablet_replica.cc:333] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
W20260709 13:20:16.595588 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.600911 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.603055  8925 raft_consensus.cc:2243] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.604293  8925 raft_consensus.cc:2272] T 7a82842926e645c08dde847bfe182c31 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.608589  8925 tablet_replica.cc:333] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.610493  8925 raft_consensus.cc:2243] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.611760  8925 pending_rounds.cc:70] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 2 pending ops.
I20260709 13:20:16.612092  8925 pending_rounds.cc:77] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 16 index: 12 } timestamp: 7305638638151643136 op_type: NO_OP noop_request { }
I20260709 13:20:16.612516  8925 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:16.612864  8925 pending_rounds.cc:77] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 14 index: 11 } timestamp: 7305638574539956224 op_type: NO_OP noop_request { }
I20260709 13:20:16.613282  8925 raft_consensus.cc:2889] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:16.557457 18270 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:16.617833 18270 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.618538 18270 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:16.619647 10020 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:16.621856 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.622610 18063 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:16.599514 18260 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.623919 18260 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.626425 18260 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.627439 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.623436  9954 catalog_manager.cc:5697] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 10 to 20. New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.633380  8925 raft_consensus.cc:2272] T cebbf225b866442ab895577b0496de1b P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.638588  8925 tablet_replica.cc:333] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.622139 18270 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:16.640875 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:16.630812  9954 catalog_manager.cc:5697] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 10 to 23, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 23 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.675143  9954 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.646327 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.647004 18281 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.677560  9954 catalog_manager.cc:5697] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.678928 18281 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.679355  9954 catalog_manager.cc:5697] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 22 to 24, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 24 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.681674 18281 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.681280  9954 catalog_manager.cc:5697] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.682868 10020 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.684391  8925 raft_consensus.cc:2243] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.688551 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.706137 18281 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:16.707401  8925 raft_consensus.cc:2272] T 87d36b9784d64ff48d0d4f0224266506 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.713287 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.705257  9954 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 16 to 25, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 25 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.716434 10020 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:16.720008 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.733283  8925 tablet_replica.cc:333] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.735067 10020 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.744191 18270 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:16.736135 18281 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:16.744778 18260 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.745613 18260 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.747865  8925 raft_consensus.cc:2243] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.748328 18260 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.749221 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.751677 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:16.762457 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.763433  8925 raft_consensus.cc:2272] T 9a895dc8dee14dfba216f443121474f4 P 01bfaf1ad56a4deea886bbc64dec466c [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.768396 10020 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.777679 18281 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:16.740166  9954 catalog_manager.cc:5697] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 11 to 19, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 19 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:16.781004 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.781847  9954 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 11 to 19, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 19 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:20:16.783080 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.783584  8925 tablet_replica.cc:333] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.787449  8925 raft_consensus.cc:2243] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:16.794832  8925 raft_consensus.cc:2272] T 95b81db48ec24878966db719b2790127 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.783704  9954 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 19 to 24, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 24 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:16.807186 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.812697  9954 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.821460  8925 tablet_replica.cc:333] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.826593  8925 raft_consensus.cc:2243] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:20:16.828390 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.829913 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:16.836372  9954 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.838550  9954 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.830058  8925 raft_consensus.cc:2272] T 2c048cf8265e4264b35912a8caac9936 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.847692  9954 catalog_manager.cc:5697] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 10 to 23, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 23 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.849888  9954 catalog_manager.cc:5697] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 19 to 21, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 21 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.851536  8925 tablet_replica.cc:333] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.851832  9954 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 13 to 20, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 20 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:16.853062  8925 raft_consensus.cc:2243] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.853917  8925 raft_consensus.cc:2272] T 66968e5dbd3040d290ce9716c146d46a P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.853751  9954 catalog_manager.cc:5697] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
W20260709 13:20:16.871730 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.873739 18281 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.875991 18281 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:16.876737 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.881945 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:16.884658 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:16.883231 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:16.891249  8925 tablet_replica.cc:333] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.884280 18260 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.892168 18260 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:16.878983 18281 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:16.895821 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:16.896063 18260 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:16.903425  9954 catalog_manager.cc:5697] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.919462  9954 catalog_manager.cc:5697] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.920831  9954 catalog_manager.cc:5697] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:16.922245  9954 catalog_manager.cc:5697] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.892772 10020 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:16.892607  8925 raft_consensus.cc:2243] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:16.925680  8925 pending_rounds.cc:70] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:16.926584 18260 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:20:16.928017 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:16.927707  9954 catalog_manager.cc:5697] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 13 to 20, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:16.933162  8925 pending_rounds.cc:77] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305638620362485760 op_type: NO_OP noop_request { }
I20260709 13:20:16.935009  8925 raft_consensus.cc:2889] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:16.936110 10020 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:16.937119 18281 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:16.935497  8925 raft_consensus.cc:2272] T a0c93dea5060480eb0618f168078f4e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:16.943883  8925 tablet_replica.cc:333] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:16.970722  8925 raft_consensus.cc:2243] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:16.961431 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:16.972115 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:16.987293  8925 raft_consensus.cc:2272] T 6a7190358a9f475d82e5f9875677bccf P 01bfaf1ad56a4deea886bbc64dec466c [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.007473  8925 tablet_replica.cc:333] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.008776  8925 raft_consensus.cc:2243] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:17.024359 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:17.025367 18260 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.026082 18260 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.029717 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:17.032383 18260 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:17.033653 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:17.035869 10020 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:17.036906 18281 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:17.050155  8925 pending_rounds.cc:70] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 2 pending ops.
I20260709 13:20:17.050680  8925 pending_rounds.cc:77] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 22 index: 12 } timestamp: 7305638704223641600 op_type: NO_OP noop_request { }
I20260709 13:20:17.052017  8925 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.052405  8925 pending_rounds.cc:77] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 21 index: 11 } timestamp: 7305638650861084672 op_type: NO_OP noop_request { }
I20260709 13:20:17.052821  8925 raft_consensus.cc:2889] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.053215  8925 raft_consensus.cc:2272] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.058462  8925 tablet_replica.cc:333] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.059787  8925 raft_consensus.cc:2243] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.063987  8925 raft_consensus.cc:2272] T ee79852a0bc04642b58b690930f78267 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:17.066792 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:17.069972  8925 tablet_replica.cc:333] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.074265  8925 raft_consensus.cc:2243] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.075495 18260 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.076399 18260 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:17.079101 18260 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:17.080420 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:17.084889 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:17.085770 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:17.083695  8925 raft_consensus.cc:2272] T 806f35e66c234d569cbafd8b3981518f P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.087925 10019 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:17.089902 18260 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:17.091087 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:17.101209  8925 tablet_replica.cc:333] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.142567  8925 raft_consensus.cc:2243] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.144286  8925 raft_consensus.cc:2272] T 5391e406039d4af8bf2b17f1b2034ef5 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.188450  9946 catalog_manager.cc:5697] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 22 to 27. New cstate: current_term: 27 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.188426  8925 tablet_replica.cc:333] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.191815  9946 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 18 to 20, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 20 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:17.196866  9946 catalog_manager.cc:5697] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 13 to 22. New cstate: current_term: 22 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.191798  8925 raft_consensus.cc:2243] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.200227  8925 raft_consensus.cc:2272] T f09ed53b9cff42a29c1d6c06e009ee94 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.214501  9946 catalog_manager.cc:5697] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 24 to 25, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 25 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:17.216117  8925 tablet_replica.cc:333] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.237880  8925 raft_consensus.cc:2243] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.238935  8925 raft_consensus.cc:2272] T e14489a089594151b58748865ea36e81 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.246588 18260 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.247658 18260 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:17.259791 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:17.262319  9946 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43646 (request call id 125) took 3573 ms. Trace:
I20260709 13:20:17.278394  8925 tablet_replica.cc:333] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.279915  8925 raft_consensus.cc:2243] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.283254  8925 raft_consensus.cc:2272] T 5853a69cde724bc3a143613bf68a63fc P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.275903  9946 rpcz_store.cc:276] 0709 13:20:13.685954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.384380 (+2698426us) service_pool.cc:224] Handling call
0709 13:20:17.259343 (+874963us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:17.220898 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:17.221639 (+   741us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:17.221680 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:20:17.221700 (+    20us) tablet.cc:662] Decoding operations
0709 13:20:17.222401 (+   701us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:17.222465 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:17.222501 (+    36us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:20:17.222941 (+   440us) tablet.cc:701] Row locks acquired
0709 13:20:17.222963 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:20:17.223161 (+   198us) write_op.cc:270] Start()
0709 13:20:17.223305 (+   144us) write_op.cc:276] Timestamp: P: 1783603217223081 usec, L: 0
0709 13:20:17.223330 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:20:17.224017 (+   687us) log.cc:844] Serialized 1670 byte log entry
0709 13:20:17.226319 (+  2302us) op_driver.cc:464] REPLICATION: finished
0709 13:20:17.226951 (+   632us) write_op.cc:301] APPLY: starting
0709 13:20:17.227059 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:17.229733 (+  2674us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:17.229846 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:17.232494 (+  2648us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:17.232732 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:20:17.232791 (+    59us) write_op.cc:312] APPLY: finished
0709 13:20:17.233106 (+   315us) log.cc:844] Serialized 49 byte log entry
0709 13:20:17.234260 (+  1154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.234415 (+   155us) write_op.cc:454] Released schema lock
0709 13:20:17.234473 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:17.234982 (+   509us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.235013 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":24090,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":335,"apply.run_cpu_time_us":7104,"apply.run_wall_time_us":7985,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16688,"num_ops":4,"prepare.queue_time_us":113,"prepare.run_cpu_time_us":3452,"prepare.run_wall_time_us":5215,"replication_time_us":2778}]]}
I20260709 13:20:17.291272  8925 tablet_replica.cc:333] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.292755  8925 raft_consensus.cc:2243] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.293551  8925 raft_consensus.cc:2272] T c5af05c8079540ec84b0cc3dc1a9e051 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.298216  8925 tablet_replica.cc:333] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.299712  8925 raft_consensus.cc:2243] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.300900  8925 pending_rounds.cc:70] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.301213  8925 pending_rounds.cc:77] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 19 index: 10 } timestamp: 7305638646481399808 op_type: NO_OP noop_request { }
I20260709 13:20:17.260663 18260 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:17.294291 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:17.297959  9954 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 203) took 4022 ms. Trace:
I20260709 13:20:17.301718  8925 raft_consensus.cc:2889] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.313898  8925 raft_consensus.cc:2272] T a169a099da7f404bb7804561c7aca3e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:17.313122 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:17.313665  9954 rpcz_store.cc:276] 0709 13:20:13.275489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.492522 (+2217033us) service_pool.cc:224] Handling call
0709 13:20:17.297821 (+1805299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:17.011255 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:17.011955 (+   700us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:17.012003 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:20:17.012021 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:17.025236 (+ 13215us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:17.025362 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:17.025391 (+    29us) tablet.cc:685] Acquiring locks for 40 operations
0709 13:20:17.031998 (+  6607us) tablet.cc:701] Row locks acquired
0709 13:20:17.032022 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:20:17.032243 (+   221us) write_op.cc:270] Start()
0709 13:20:17.033467 (+  1224us) write_op.cc:276] Timestamp: P: 1783603217032181 usec, L: 0
0709 13:20:17.033496 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:20:17.034488 (+   992us) log.cc:844] Serialized 15490 byte log entry
0709 13:20:17.049080 (+ 14592us) op_driver.cc:464] REPLICATION: finished
0709 13:20:17.100244 (+ 51164us) write_op.cc:301] APPLY: starting
0709 13:20:17.100359 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:17.122655 (+ 22296us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:17.122772 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:17.148733 (+ 25961us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:17.149260 (+   527us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=80,key_file_lookups=80,delta_file_lookups=0,mrs_lookups=0
0709 13:20:17.149317 (+    57us) write_op.cc:312] APPLY: finished
0709 13:20:17.149996 (+   679us) log.cc:844] Serialized 340 byte log entry
0709 13:20:17.151034 (+  1038us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.151451 (+   417us) write_op.cc:454] Released schema lock
0709 13:20:17.151647 (+   196us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:17.152185 (+   538us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.152215 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29604,"reported_tablets":52,"spinlock_wait_cycles":46720,"child_traces":[["op",{"apply.queue_time_us":50795,"apply.run_cpu_time_us":49908,"apply.run_wall_time_us":51914,"cfile_cache_hit":162,"cfile_cache_hit_bytes":171007,"num_ops":40,"prepare.queue_time_us":45672,"prepare.run_cpu_time_us":9258,"prepare.run_wall_time_us":23654,"replication_time_us":15375,"spinlock_wait_cycles":19840,"thread_start_us":86271,"threads_started":2}]]}
I20260709 13:20:17.326303  8925 tablet_replica.cc:333] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.328343  8925 raft_consensus.cc:2243] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.337764  8925 raft_consensus.cc:2272] T 161f2d2f2674494bbffbd5ee927cf5e6 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.342182  8925 tablet_replica.cc:333] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
W20260709 13:20:17.363428 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 81710.9us
I20260709 13:20:17.366564  8925 raft_consensus.cc:2243] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.385723  8925 raft_consensus.cc:2272] T dfacd39092d24ba5a33d0cd9451b91a1 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:17.388526  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 86227.2us
W20260709 13:20:17.392882  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:20:17.399852 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:17.417994 10020 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62017.3us
I20260709 13:20:17.429828 10019 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:17.433313 18281 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:17.441588 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:17.445186  8925 tablet_replica.cc:333] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.471976  8925 raft_consensus.cc:2243] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:17.477480 10019 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 44961.8us
I20260709 13:20:17.484022  8925 pending_rounds.cc:70] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.484691  8925 pending_rounds.cc:77] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 13 index: 10 } timestamp: 7305638650382536704 op_type: NO_OP noop_request { }
I20260709 13:20:17.485215  8925 raft_consensus.cc:2889] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.485715  8925 raft_consensus.cc:2272] T a122c89ca68646b8bf88dbb1d5dcbfa8 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.492833  8925 tablet_replica.cc:333] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.497720  8925 raft_consensus.cc:2243] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.500016  8925 raft_consensus.cc:2272] T 986d51d657bc4fac8f4199fbc38600c1 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.515488  8925 tablet_replica.cc:333] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.516716  9948 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:17.516784  8925 raft_consensus.cc:2243] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.520092  8925 pending_rounds.cc:70] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 2 pending ops.
I20260709 13:20:17.520483  8925 pending_rounds.cc:77] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 8 } timestamp: 7305638622489612288 op_type: NO_OP noop_request { }
I20260709 13:20:17.522046  8925 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.522404  8925 pending_rounds.cc:77] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 16 index: 7 } timestamp: 7305638576084340736 op_type: NO_OP noop_request { }
I20260709 13:20:17.522789  8925 raft_consensus.cc:2889] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.523190  8925 raft_consensus.cc:2272] T 82960478409442549781547254b9d8ea P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.527093  8925 tablet_replica.cc:333] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
W20260709 13:20:17.533712  9951 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 202) took 20151 ms (client timeout 15000 ms). Trace:
W20260709 13:20:17.534454  9951 rpcz_store.cc:269] 0709 13:19:57.380647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.381037 (+   390us) service_pool.cc:224] Handling call
0709 13:20:17.532257 (+20151220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1389,"reported_tablets":52}
I20260709 13:20:17.528306  8925 raft_consensus.cc:2243] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.538599  8925 pending_rounds.cc:70] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 2 pending ops.
I20260709 13:20:17.539026  8925 pending_rounds.cc:77] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 20 index: 10 } timestamp: 7305638675349377024 op_type: NO_OP noop_request { }
I20260709 13:20:17.539811  8925 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.540220  8925 pending_rounds.cc:77] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 9 } timestamp: 7305638622906990592 op_type: NO_OP noop_request { }
I20260709 13:20:17.540684  8925 raft_consensus.cc:2889] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.541117  8925 raft_consensus.cc:2272] T c502c18433fb46e2ac2a42d7d8746ce0 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.572656 18291 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.573390 18291 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.576138  8925 tablet_replica.cc:333] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.577308  8925 raft_consensus.cc:2243] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.577874 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:17.578312  8925 pending_rounds.cc:70] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 2 pending ops.
I20260709 13:20:17.578624  8925 pending_rounds.cc:77] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 20 index: 14 } timestamp: 7305638684426375168 op_type: NO_OP noop_request { }
I20260709 13:20:17.579002  8925 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.579378  8925 pending_rounds.cc:77] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 13 } timestamp: 7305638638145961984 op_type: NO_OP noop_request { }
I20260709 13:20:17.579771  8925 raft_consensus.cc:2889] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.580188  8925 raft_consensus.cc:2272] T d97ed0ec5f214d95abb9e01d368a04b5 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.580098 18291 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:17.582120 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:17.584017  8925 tablet_replica.cc:333] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.585430  8925 raft_consensus.cc:2243] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.586434  8925 pending_rounds.cc:70] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.587508 10020 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:17.587870  8925 pending_rounds.cc:77] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305638675351175168 op_type: NO_OP noop_request { }
I20260709 13:20:17.588445 18291 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:17.589214  8925 raft_consensus.cc:2889] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.589627  8925 raft_consensus.cc:2272] T bfb08df714384523b94354944a523e54 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.593187  8925 tablet_replica.cc:333] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.594099  8925 raft_consensus.cc:2243] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.594998  8925 raft_consensus.cc:2272] T 79954d719dc94974b4864ff715e9ef07 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.598436  8925 tablet_replica.cc:333] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.599514  8925 raft_consensus.cc:2243] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.600237  8925 pending_rounds.cc:70] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.600512  8925 pending_rounds.cc:77] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 12 index: 8 } timestamp: 7305638629695467520 op_type: NO_OP noop_request { }
I20260709 13:20:17.600920  8925 raft_consensus.cc:2889] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.601332  8925 raft_consensus.cc:2272] T f3220b9037e44cae95e49af43de3523a P 01bfaf1ad56a4deea886bbc64dec466c [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.609831  8925 tablet_replica.cc:333] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.611078  8925 raft_consensus.cc:2243] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.612519  8925 raft_consensus.cc:2272] T 2869566cb59547689f49f6e04511ed8a P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.631173  8925 tablet_replica.cc:333] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.632300  8925 raft_consensus.cc:2243] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.633765  8925 raft_consensus.cc:2272] T 231b54e25cca40f6a3b0625d68e97cf7 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.638406  8925 tablet_replica.cc:333] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.639496  8925 raft_consensus.cc:2243] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.640442  8925 raft_consensus.cc:2272] T c3664f114a52481c8f97e06e9b2d2068 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.658869  8925 tablet_replica.cc:333] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.659941  8925 raft_consensus.cc:2243] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.660692  8925 pending_rounds.cc:70] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.660972  8925 pending_rounds.cc:77] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305638687038447616 op_type: NO_OP noop_request { }
I20260709 13:20:17.661362  8925 raft_consensus.cc:2889] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.661762  8925 raft_consensus.cc:2272] T aca8a04bf8df4212bfdf5fd990503c0a P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.665375  8925 tablet_replica.cc:333] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.699183  8925 raft_consensus.cc:2243] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.700387  8925 raft_consensus.cc:2272] T 1c7c855af77d40ab995a18b5609de8f5 P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.704682  8925 tablet_replica.cc:333] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.705888  8925 raft_consensus.cc:2243] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.707036  8925 raft_consensus.cc:2272] T 371654d6e0a94cba825fa080a9998940 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.711514  8925 tablet_replica.cc:333] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.712569  8925 raft_consensus.cc:2243] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.713673  8925 pending_rounds.cc:70] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.713964  8925 pending_rounds.cc:77] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305638660964114432 op_type: NO_OP noop_request { }
I20260709 13:20:17.714382  8925 raft_consensus.cc:2889] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.714847  8925 raft_consensus.cc:2272] T 764b8122cc0a4ccc97816d729adafabe P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.715741 18286 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:17.716513 18286 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:17.722347 18291 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.722648 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:17.723062 18291 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.726953 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:17.728711  8925 tablet_replica.cc:333] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.734015 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "01bfaf1ad56a4deea886bbc64dec466c" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:17.739966 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:17.740634 10096 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:17.741761 18288 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:17.741621 10019 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 01bfaf1ad56a4deea886bbc64dec466c; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:17.743147 18289 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:17.741236  8925 raft_consensus.cc:2243] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.745326  8925 pending_rounds.cc:70] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.745643  8925 pending_rounds.cc:77] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 14 index: 10 } timestamp: 7305638665207758848 op_type: NO_OP noop_request { }
I20260709 13:20:17.746006  8925 raft_consensus.cc:2889] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.746411  8925 raft_consensus.cc:2272] T d22230bc6afb42ceac52420e42db0292 P 01bfaf1ad56a4deea886bbc64dec466c [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.726992 18291 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:17.749982  8925 tablet_replica.cc:333] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.750207 18286 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:17.751035  8925 raft_consensus.cc:2243] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.754365  8925 pending_rounds.cc:70] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.754720  8925 pending_rounds.cc:77] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305638675397599232 op_type: NO_OP noop_request { }
I20260709 13:20:17.756645 18286 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:17.756837  8925 raft_consensus.cc:2889] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.757581  8925 raft_consensus.cc:2272] T 8fd90bb718ac49bf84e09fe79d462240 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.757373 18286 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.761221  8925 tablet_replica.cc:333] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
W20260709 13:20:17.764153 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:17.766441 18286 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:17.766738 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:17.769009 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:17.769358  8925 raft_consensus.cc:2243] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.771018 18286 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:17.770656  8925 raft_consensus.cc:2272] T d07e888fde29492b84e575218308b6ff P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.789690  8925 tablet_replica.cc:333] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.790776  8925 raft_consensus.cc:2243] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.791857  8925 raft_consensus.cc:2272] T 4d6c92980e764b60a913b8c5533ed9ad P 01bfaf1ad56a4deea886bbc64dec466c [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.798595  8925 tablet_replica.cc:333] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.800369  8925 raft_consensus.cc:2243] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.802029  8925 raft_consensus.cc:2272] T 2219e1af873d4a00966a8c5975eb7987 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.804162 18241 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:17.804884 18241 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.805523  8925 tablet_replica.cc:333] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.806438  8925 raft_consensus.cc:2243] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.807098  8925 raft_consensus.cc:2272] T cf88b6d5c6504b3cb2cbd21cff68e57e P 01bfaf1ad56a4deea886bbc64dec466c [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.808929 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:17.809731 10123 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:20:17.811944 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:17.812438  8925 tablet_replica.cc:333] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.813465  8925 raft_consensus.cc:2243] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:17.815953 10170 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:17.817252 18241 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:17.820858 18292 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:17.824692 18292 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:17.825261 18292 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:17.778507  9948 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47238 (request call id 210) took 8288 ms. Trace:
I20260709 13:20:17.834110  8925 pending_rounds.cc:70] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.833251  9948 rpcz_store.cc:276] 0709 13:20:09.489610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.265532 (+3775922us) service_pool.cc:224] Handling call
0709 13:20:17.778403 (+4512871us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:17.615822 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:17.616392 (+   570us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:17.616454 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:20:17.616468 (+    14us) tablet.cc:662] Decoding operations
0709 13:20:17.616839 (+   371us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:17.616910 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:17.616938 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:17.617164 (+   226us) tablet.cc:701] Row locks acquired
0709 13:20:17.617177 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:20:17.617332 (+   155us) write_op.cc:270] Start()
0709 13:20:17.617456 (+   124us) write_op.cc:276] Timestamp: P: 1783603217617283 usec, L: 0
0709 13:20:17.617473 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:20:17.618377 (+   904us) log.cc:844] Serialized 538 byte log entry
0709 13:20:17.628368 (+  9991us) op_driver.cc:464] REPLICATION: finished
0709 13:20:17.630014 (+  1646us) write_op.cc:301] APPLY: starting
0709 13:20:17.630154 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:17.631277 (+  1123us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:17.631392 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:17.632217 (+   825us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:17.632420 (+   203us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:17.632499 (+    79us) write_op.cc:312] APPLY: finished
0709 13:20:17.632750 (+   251us) log.cc:844] Serialized 25 byte log entry
0709 13:20:17.633765 (+  1015us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.633877 (+   112us) write_op.cc:454] Released schema lock
0709 13:20:17.633928 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:17.634371 (+   443us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:17.634405 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35198,"reported_tablets":52,"spinlock_wait_cycles":12032,"child_traces":[["op",{"apply.queue_time_us":1341,"apply.run_cpu_time_us":3577,"apply.run_wall_time_us":4348,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4172,"num_ops":1,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":2890,"prepare.run_wall_time_us":2888,"replication_time_us":10751}]]}
I20260709 13:20:17.835068  8925 pending_rounds.cc:77] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 25 index: 16 } timestamp: 7305638652536135680 op_type: NO_OP noop_request { }
I20260709 13:20:17.851821  8925 raft_consensus.cc:2889] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.852417  8925 raft_consensus.cc:2272] T 82406372d02c4dd59d4ef5c32436e148 P 01bfaf1ad56a4deea886bbc64dec466c [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.853945  9946 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 23, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65). New cstate: current_term: 23 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:17.855830  9946 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:17.863940  9946 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "01bfaf1ad56a4deea886bbc64dec466c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } }
I20260709 13:20:17.883517  8925 tablet_replica.cc:333] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.884719  8925 raft_consensus.cc:2243] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.885777  8925 pending_rounds.cc:70] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:17.886093  8925 pending_rounds.cc:77] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 23 index: 13 } timestamp: 7305638683550777344 op_type: NO_OP noop_request { }
I20260709 13:20:17.886534  8925 raft_consensus.cc:2889] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:17.889701 18292 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:17.899252 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:17.886971  8925 raft_consensus.cc:2272] T 2c485d13f66d4ce8bdca72b60d242842 P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.914433 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:17.915305 10123 raft_consensus.cc:3060] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:17.916818 10170 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:17.906881  8925 tablet_replica.cc:333] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.927302 18292 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:17.928301  8925 raft_consensus.cc:2243] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:20:17.929899  8925 raft_consensus.cc:2272] T 3173f90d855c4371aeafe17b5f760205 P 01bfaf1ad56a4deea886bbc64dec466c [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.927233 10123 raft_consensus.cc:2468] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:17.937333 10168 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:17.942432 18292 raft_consensus.cc:2804] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:17.946735 18292 raft_consensus.cc:697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:17.948061 18292 consensus_queue.cc:237] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:17.945768  8925 tablet_replica.cc:333] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.960096  8925 raft_consensus.cc:2243] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:17.961046  8925 raft_consensus.cc:2272] T 2188d855354542c984f249b94175c660 P 01bfaf1ad56a4deea886bbc64dec466c [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:17.988878  8925 tablet_replica.cc:333] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:17.989933  8925 raft_consensus.cc:2243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:17.997644  9949 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 224) took 55885 ms (client timeout 15000 ms). Trace:
W20260709 13:20:18.036273  9949 rpcz_store.cc:269] 0709 13:19:22.112639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.141226 (+ 28587us) service_pool.cc:224] Handling call
0709 13:20:17.997398 (+55856172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7458,"reported_tablets":53}
I20260709 13:20:17.990931  8925 raft_consensus.cc:2272] T 5e46fa8cb2404116bf566b4ef9bbb2cf P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:18.043258  8925 tablet_replica.cc:333] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:18.044287  8925 raft_consensus.cc:2243] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:18.045087  8925 raft_consensus.cc:2272] T b09e10904af344988a1a11ae9e576799 P 01bfaf1ad56a4deea886bbc64dec466c [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:18.059537  8925 tablet_replica.cc:333] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:18.060541  8925 raft_consensus.cc:2243] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:18.061818  8925 raft_consensus.cc:2272] T 2169fe5ebfe5416c94b7d9095d9af6da P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:18.088848  9952 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 197) took 148380 ms (client timeout 15000 ms). Trace:
I20260709 13:20:18.078987  8925 tablet_replica.cc:333] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:18.113920 18269 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:18.115643 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.140300  8925 raft_consensus.cc:2243] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:18.139154  9952 rpcz_store.cc:269] 0709 13:17:49.707895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:49.708257 (+   362us) service_pool.cc:224] Handling call
0709 13:20:18.088599 (+148380342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12463,"reported_tablets":52}
I20260709 13:20:18.141387  8925 pending_rounds.cc:70] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Trying to abort 1 pending ops.
I20260709 13:20:18.141748  8925 pending_rounds.cc:77] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305638671079108608 op_type: NO_OP noop_request { }
I20260709 13:20:18.142257  8925 raft_consensus.cc:2889] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:18.142666  8925 raft_consensus.cc:2272] T a353f267acea40eaaf03befcb32892f6 P 01bfaf1ad56a4deea886bbc64dec466c [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:18.139564 18269 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:18.139917 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.148121 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:18.149044 10123 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:20:18.150473 18269 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:18.155102 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:18.157428  8925 tablet_replica.cc:333] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:18.158712  8925 raft_consensus.cc:2243] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:18.162221 10865 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.165161  8925 raft_consensus.cc:2272] T 73fb38c54abe4cd4bb552eb544020f47 P 01bfaf1ad56a4deea886bbc64dec466c [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:18.169047 18269 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:18.169776 18269 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.170213 18269 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:18.216252  8925 tablet_replica.cc:333] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c: stopping tablet replica
I20260709 13:20:18.217624  8925 raft_consensus.cc:2243] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:18.218708  8925 raft_consensus.cc:2272] T 75ddb7f5e3204a51b3dafe4903888dcc P 01bfaf1ad56a4deea886bbc64dec466c [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:18.219779 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.221038 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.238772 18269 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:18.257192 18269 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:18.270291 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:18.271246 10128 raft_consensus.cc:3060] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:18.293303 18070 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.294139 18070 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:18.297824 18070 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:18.298022 10865 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.308231 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:18.318096 10128 raft_consensus.cc:2468] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:20:18.320044 10863 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:18.321655 18269 raft_consensus.cc:2804] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:18.322274 18269 raft_consensus.cc:697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 23 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:18.323342 18269 consensus_queue.cc:237] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:18.337369 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.320238 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.343436 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 93256.8us
W20260709 13:20:18.360881  9953 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 200) took 76189 ms (client timeout 15000 ms). Trace:
W20260709 13:20:18.370713  9953 rpcz_store.cc:269] 0709 13:19:02.171389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.289166 (+117777us) service_pool.cc:224] Handling call
0709 13:20:18.360618 (+76071452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31696,"reported_tablets":52,"spinlock_wait_cycles":111744}
W20260709 13:20:18.365859 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.408329 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.427134 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.472446 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.473553 10170 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.474159 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:18.475481 18294 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:18.484670 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.520311 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:18.531781 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.532774 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.533563 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.539641  9945 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47236 (request call id 201) took 52939 ms (client timeout 15000 ms). Trace:
I20260709 13:20:18.542380 18288 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:18.543354 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.543473 18288 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:18.548198 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:18.550284 18288 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:18.550570 10128 raft_consensus.cc:1275] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:20:18.551582 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:18.551486  9945 rpcz_store.cc:269] 0709 13:19:25.599912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.600373 (+   461us) service_pool.cc:224] Handling call
0709 13:20:18.539352 (+52938979us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1488096,"reported_tablets":52}
I20260709 13:20:18.552182 10096 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:18.552920 18070 consensus_queue.cc:1048] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:18.555053 18288 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:18.555423  9953 catalog_manager.cc:5697] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 19 to 20, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 20 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.563342  9946 catalog_manager.cc:5697] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 22 to 23, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 23 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.609336 18297 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.610298 18297 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:18.630105 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:18.636495  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:20:18.637691 18297 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:18.639528 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:18.640679 10201 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
W20260709 13:20:18.644759 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.647106 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:18.658878 18297 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:18.659577 18297 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
W20260709 13:20:18.669356 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.667421 18297 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:18.689459 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.731487 18070 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.732143 18070 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:18.740411 18070 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:18.749107 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58588.4us
W20260709 13:20:18.754601  9947 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 222) took 98810 ms (client timeout 15000 ms). Trace:
W20260709 13:20:18.764827  9947 rpcz_store.cc:269] 0709 13:18:39.943837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.947721 (+  3884us) service_pool.cc:224] Handling call
0709 13:20:18.754302 (+98806581us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":490743,"reported_tablets":53}
I20260709 13:20:18.729481 18297 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:18.780838 18297 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:18.788208 10865 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:18.806895 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:18.815701 10170 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.818086 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:18.818954 10201 raft_consensus.cc:3060] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:18.822399  9950 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47224 (request call id 223) took 77275 ms (client timeout 15000 ms). Trace:
W20260709 13:20:18.822979  9950 rpcz_store.cc:269] 0709 13:19:01.547144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.549558 (+  2414us) service_pool.cc:224] Handling call
0709 13:19:01.836873 (+287315us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5400002490. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274ee0640 00007fa27ddb7b76 00007fa27dec1c3e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:20:18.822134 (+76985261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25711,"reported_tablets":53,"spinlock_wait_cycles":56137600}
I20260709 13:20:18.826139 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:18.827088 10128 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:18.833200 10201 raft_consensus.cc:2468] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:20:18.850325 10864 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:18.856156 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:18.858803 18241 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:18.859328 18241 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.859686 18241 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:18.866227 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:18.867611 18245 raft_consensus.cc:2804] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:18.868391 18245 raft_consensus.cc:697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:18.870502 18245 consensus_queue.cc:237] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:18.879376 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:18.956419 18070 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.961946 18241 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:18.974630 18241 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:18.981441 10170 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:18.961115  9945 catalog_manager.cc:5697] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 20 to 21, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 21 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.994338 18241 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.995103 18241 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.003319 18305 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.004112 18305 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:18.913443 18288 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:18.904420 18286 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.006652 18070 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.015447 18241 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:19.024325 18305 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:19.026234 18070 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:19.028741 10128 raft_consensus.cc:1275] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:19.015842 18288 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.050081 18311 consensus_queue.cc:1048] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:19.050751 18288 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.052698 18288 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.015648 18286 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.053326 18288 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.055440 18286 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.055747 18288 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.056419 18288 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.056929 18288 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:19.059211 10002 auto_rebalancer.cc:629] Failed to schedule move for tablet dfacd39092d24ba5a33d0cd9451b91a1: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:19.060617 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:19.075896 18288 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.100605 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:19.101845 10201 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
W20260709 13:20:19.115487 10096 connection.cc:573] client connection to 127.8.183.65:43347 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:20:19.116387 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:20:19.118456 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:20:19.119019 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:20:19.118673 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:19.119993 10201 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:19.130290 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:19.131282 10201 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 18.
I20260709 13:20:19.139371 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:19.141408 10201 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:19.142560 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.143972 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.144974 18288 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:19.145581 18288 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.146008 18288 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:19.157541 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.158489 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:19.159329 10127 raft_consensus.cc:3060] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:19.162920 10170 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 429 similar messages]
W20260709 13:20:19.165007 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:19.169401 18315 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:19.170003 18315 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.170414 18315 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:19.176981 18316 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:19.177626 18316 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.178089 18316 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:19.180708 18315 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.183714 18315 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:19.184988 10865 connection.cc:573] client connection to 127.8.183.65:43347 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:20:19.185019 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.191359 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:19.193488 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:19.191874 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:19.195571 10170 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:20:19.195685 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:19.205914 10200 raft_consensus.cc:3060] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:19.196367 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
W20260709 13:20:19.206039 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.219885 10130 raft_consensus.cc:2393] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 24: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:19.221745 18316 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.223218 10128 raft_consensus.cc:2393] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 19: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:19.229152 18316 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.232317 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:19.233142 10196 raft_consensus.cc:3060] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:19.211426 10127 raft_consensus.cc:2468] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
I20260709 13:20:19.199875 18315 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:19.218314 10201 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:19.247974 18318 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.258100 18318 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:19.258924 10022 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4516) could send its response
W20260709 13:20:19.260226 10022 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4517) could send its response
W20260709 13:20:19.261224 10022 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4518) could send its response
I20260709 13:20:19.261668 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:19.262665 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:19.240535 18315 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.263754 10196 raft_consensus.cc:2468] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:20:19.260524 18318 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.263840 10200 raft_consensus.cc:2468] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
W20260709 13:20:19.268858  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4510) could send its response
W20260709 13:20:19.269353 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.269587  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4511) could send its response
W20260709 13:20:19.270242  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4512) could send its response
W20260709 13:20:19.270761  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4513) could send its response
W20260709 13:20:19.271353  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4514) could send its response
W20260709 13:20:19.271937  8925 connection.cc:466] server connection from 127.0.0.1:46802 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46802 (request call id 4515) could send its response
I20260709 13:20:19.274358 10095 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:19.278494 10095 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:19.266785 18315 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:19.283741 18314 raft_consensus.cc:2804] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:19.284360 18314 raft_consensus.cc:697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:19.285203 18314 consensus_queue.cc:237] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.277823 18292 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:19.289707 18288 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.291337 10131 raft_consensus.cc:2393] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 23: Already voted for candidate e9a849ac7eda426abfca97cf8b15da0f in this term.
I20260709 13:20:19.292004 18316 raft_consensus.cc:2804] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:19.292783 18316 raft_consensus.cc:697] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:19.293936 18316 consensus_queue.cc:237] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.295372 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:19.301546 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:19.302806 10200 raft_consensus.cc:3060] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:19.304379 10168 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.307673 18241 raft_consensus.cc:2804] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:19.308473 18241 raft_consensus.cc:697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:19.305076 18288 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.304978 18292 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:19.336295 18241 consensus_queue.cc:237] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.342144 10200 raft_consensus.cc:2468] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:19.350297 10095 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:19.351665 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:19.353104 10096 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.364449 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:19.365542 10096 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:19.377761 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.355058 18288 raft_consensus.cc:2804] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:19.396536 18288 raft_consensus.cc:697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:20:19.393838 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.358845 18305 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:19.398869 18288 consensus_queue.cc:237] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.402007 18314 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:19.401597 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.417286 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.417870 10096 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.423208 10169 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3664f114a52481c8f97e06e9b2d2068. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.426975 10123 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:19.433271 18292 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:19.480494 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:19.477298 18292 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.489264 18292 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.414436 18315 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:19.495733 10096 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.497498 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.502061 10201 raft_consensus.cc:1275] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:19.502858  9945 catalog_manager.cc:5697] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 20 to 21, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 21 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.505506 18245 consensus_queue.cc:1048] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:19.510124 18292 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:19.528834 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:19.529827 10127 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 18.
I20260709 13:20:19.541632 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:19.545780 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:19.546670 10196 raft_consensus.cc:3060] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:19.527156 18315 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.559902 18319 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:19.568989 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.576325 10196 raft_consensus.cc:2468] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
I20260709 13:20:19.579610 10095 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:19.580798 18315 raft_consensus.cc:2804] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:19.581429 18315 raft_consensus.cc:697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:19.573721 18319 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 01bfaf1ad56a4deea886bbc64dec466c)
I20260709 13:20:19.586756 18319 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:19.582978 18315 consensus_queue.cc:237] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:19.595428 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.597394 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.598448 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:19.614876 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.618431 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.620539 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59100.5us
I20260709 13:20:19.623205 18292 raft_consensus.cc:2955] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:20:19.637379 10170 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.640506 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.633157 10123 raft_consensus.cc:2955] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:20:19.654501 10096 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.673913 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:19.680971 18319 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.658474  9950 catalog_manager.cc:5697] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 23 to 24, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 24 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.687547  9950 catalog_manager.cc:5697] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 18 to 19, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 19 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.690346  9950 catalog_manager.cc:5697] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 22 to 23, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 23 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.752085 18319 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Requested vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:19.788290 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:19.816919  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 50159.5us
I20260709 13:20:19.831485 10202 raft_consensus.cc:1275] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:20:19.838439 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.839272 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.840106 10170 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.841074 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:19.846889 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 124859us
I20260709 13:20:19.846951 18320 consensus_queue.cc:1048] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
W20260709 13:20:19.876879  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:19.885181 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 165651us
I20260709 13:20:19.887833 18351 ts_tablet_manager.cc:933] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:19.899181 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:19.899971 10123 raft_consensus.cc:3060] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:19.901105 18351 tablet_copy_client.cc:323] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.67:40505
I20260709 13:20:19.909197 10123 raft_consensus.cc:2468] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 19.
I20260709 13:20:19.917016 10201 raft_consensus.cc:1275] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:20:19.923938  9950 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.955029 10168 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c
I20260709 13:20:19.957890 18343 raft_consensus.cc:2804] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:20:19.963003 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.962920 18343 raft_consensus.cc:697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:19.963994 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:19.965963 18343 consensus_queue.cc:237] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:19.969026  9950 catalog_manager.cc:5697] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 23 to 24, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 24 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:19.972625 10214 tablet_copy_service.cc:140] P d51ad94710c348889d417f7f00032948: Received BeginTabletCopySession request for tablet c3664f114a52481c8f97e06e9b2d2068 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:50726)
I20260709 13:20:19.983474 10214 tablet_copy_service.cc:161] P d51ad94710c348889d417f7f00032948: Beginning new tablet copy session on tablet c3664f114a52481c8f97e06e9b2d2068 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:50726: session id = db052bef7536453fa2d7f5a00e314a18-c3664f114a52481c8f97e06e9b2d2068
I20260709 13:20:19.973551 18320 consensus_queue.cc:1048] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:19.976094 10201 raft_consensus.cc:1275] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:19.991349 18315 consensus_queue.cc:1048] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:20.001430 10214 tablet_copy_source_session.cc:215] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:20:20.006661 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.010257 10123 raft_consensus.cc:1275] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:20.020557 10128 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:20:20.037190 10095 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2188d855354542c984f249b94175c660. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.052769 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.058298 18351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3664f114a52481c8f97e06e9b2d2068. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:20.083500 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.096558 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:20.107868 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 47533us
I20260709 13:20:20.109472 18343 consensus_queue.cc:1048] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:20.122602 10202 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:20:20.143208 18316 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:20.162640  9950 catalog_manager.cc:5697] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 18 to 19, leader changed from 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.195427 18351 tablet_copy_client.cc:806] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:20.196329 18351 tablet_copy_client.cc:670] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:20.214258 18288 raft_consensus.cc:2955] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:20:20.224625 10200 raft_consensus.cc:2955] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:20:20.226624 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.229997 18351 tablet_copy_client.cc:538] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:20.272403  9950 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index -1 to 13, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 22 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:20:20.289255 10200 raft_consensus.cc:1275] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:20:20.294822 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:20:20.295555 18351 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:20:20.300973 10128 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:20:20.308038  8925 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:20.295797 18316 consensus_queue.cc:1048] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 13:20:20.319572 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:20.350241 10095 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775): Couldn't send request to peer db052bef7536453fa2d7f5a00e314a18. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a169a099da7f404bb7804561c7aca3e6. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:20.373544 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.406468 10200 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:20.408939 18288 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:20.428901 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.432456 10128 raft_consensus.cc:1275] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:20.441520 18343 consensus_queue.cc:1048] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:20:20.489646 18286 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
I20260709 13:20:20.518821 18364 ts_tablet_manager.cc:933] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:20.517197 10200 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } } }
W20260709 13:20:20.544440 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.557181 18364 tablet_copy_client.cc:323] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.66:41681
I20260709 13:20:20.565454 10141 tablet_copy_service.cc:140] P e9a849ac7eda426abfca97cf8b15da0f: Received BeginTabletCopySession request for tablet 2188d855354542c984f249b94175c660 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:45470)
I20260709 13:20:20.566325 10141 tablet_copy_service.cc:161] P e9a849ac7eda426abfca97cf8b15da0f: Beginning new tablet copy session on tablet 2188d855354542c984f249b94175c660 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:45470: session id = db052bef7536453fa2d7f5a00e314a18-2188d855354542c984f249b94175c660
W20260709 13:20:20.589259 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:20.596371 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:20.600190 10141 tablet_copy_source_session.cc:215] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:20.607007  9950 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index -1 to 12, NON_VOTER db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) added. New cstate: current_term: 20 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:20.625999 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.662189 18364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2188d855354542c984f249b94175c660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:20.726147 18364 tablet_copy_client.cc:806] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:20.735900 18364 tablet_copy_client.cc:670] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
W20260709 13:20:20.738152 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.744313 18364 tablet_copy_client.cc:538] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:20.767019 18351 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:20:20.775298 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.776890 18351 tablet_bootstrap.cc:492] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:20:20.777963 18351 ts_tablet_manager.cc:1403] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.483s	user 0.040s	sys 0.020s
I20260709 13:20:20.779493 18364 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:20:20.785912 18351 raft_consensus.cc:359] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:20.786799 18351 raft_consensus.cc:740] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:20:20.787525 18351 consensus_queue.cc:260] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:20.790506 18351 ts_tablet_manager.cc:1434] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
W20260709 13:20:20.806354 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.846449 10214 tablet_copy_service.cc:342] P d51ad94710c348889d417f7f00032948: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-c3664f114a52481c8f97e06e9b2d2068 received from {username='slave'} at 127.0.0.1:50726
I20260709 13:20:20.847138 10214 tablet_copy_service.cc:434] P d51ad94710c348889d417f7f00032948: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-c3664f114a52481c8f97e06e9b2d2068 on tablet c3664f114a52481c8f97e06e9b2d2068 with peer db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:20.856211 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:20.895099 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.899561 10899 raft_consensus.cc:1217] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
I20260709 13:20:20.942018 18351 ts_tablet_manager.cc:933] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Initiating tablet copy from peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:20.966375 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:20.976025 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:20.993561 18351 tablet_copy_client.cc:323] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Beginning tablet copy session from remote peer at address 127.8.183.66:41681
W20260709 13:20:21.019946 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:21.027864 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:21.050001 10141 tablet_copy_service.cc:140] P e9a849ac7eda426abfca97cf8b15da0f: Received BeginTabletCopySession request for tablet a169a099da7f404bb7804561c7aca3e6 from peer db052bef7536453fa2d7f5a00e314a18 ({username='slave'} at 127.0.0.1:45470)
I20260709 13:20:21.050803 10141 tablet_copy_service.cc:161] P e9a849ac7eda426abfca97cf8b15da0f: Beginning new tablet copy session on tablet a169a099da7f404bb7804561c7aca3e6 from peer db052bef7536453fa2d7f5a00e314a18 at {username='slave'} at 127.0.0.1:45470: session id = db052bef7536453fa2d7f5a00e314a18-a169a099da7f404bb7804561c7aca3e6
I20260709 13:20:21.063791 10141 tablet_copy_source_session.cc:215] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:20:21.065019 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.083062 18351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169a099da7f404bb7804561c7aca3e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:21.156112 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:21.206274 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:21.206902 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.212649 18364 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:21.214066 18364 tablet_bootstrap.cc:492] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:20:21.227272 18351 tablet_copy_client.cc:806] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:21.228252 18351 tablet_copy_client.cc:670] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:21.229774 18364 ts_tablet_manager.cc:1403] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.451s	user 0.056s	sys 0.017s
I20260709 13:20:21.254931 18364 raft_consensus.cc:359] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:21.268534 18364 raft_consensus.cc:740] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
I20260709 13:20:21.269493 18364 consensus_queue.cc:260] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
W20260709 13:20:21.308688  8925 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:21.321261 18351 tablet_copy_client.cc:538] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:21.342422 18364 ts_tablet_manager.cc:1434] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.112s	user 0.008s	sys 0.000s
I20260709 13:20:21.357296 18351 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Bootstrap starting.
I20260709 13:20:21.376420 10141 tablet_copy_service.cc:342] P e9a849ac7eda426abfca97cf8b15da0f: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-2188d855354542c984f249b94175c660 received from {username='slave'} at 127.0.0.1:45470
I20260709 13:20:21.377077 10141 tablet_copy_service.cc:434] P e9a849ac7eda426abfca97cf8b15da0f: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-2188d855354542c984f249b94175c660 on tablet 2188d855354542c984f249b94175c660 with peer db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:21.415083 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.459432 18241 raft_consensus.cc:1064] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:20:21.462165 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:21.462966 18241 consensus_queue.cc:237] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:21.481736 10899 raft_consensus.cc:1217] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.12->[22.13-22.13]   Dedup: 22.13->[]
W20260709 13:20:21.507803 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.508219 10899 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 LEARNER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:21.517460 18277 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:21.530449 10130 raft_consensus.cc:1275] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:21.536972 18349 consensus_queue.cc:1048] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:21.549768 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:21.619036 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:21.622684 18343 raft_consensus.cc:2955] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:20:21.630723 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:21.633255 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.635982 10126 raft_consensus.cc:2955] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:21.641750 18351 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:21.642922 18351 tablet_bootstrap.cc:492] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Bootstrap complete.
I20260709 13:20:21.652868 10899 raft_consensus.cc:2955] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:21.672001 18351 ts_tablet_manager.cc:1403] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Time spent bootstrapping tablet: real 0.315s	user 0.046s	sys 0.012s
I20260709 13:20:21.672223  9947 catalog_manager.cc:5697] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 reported cstate change: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:21.692148 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.696115 18351 raft_consensus.cc:359] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:21.697428 18351 raft_consensus.cc:740] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Initialized, Role: LEARNER
W20260709 13:20:21.722651 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:21.698349 18351 consensus_queue.cc:260] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: NON_VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: true } }
I20260709 13:20:21.733876 18351 ts_tablet_manager.cc:1434] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:20:21.761330 10141 tablet_copy_service.cc:342] P e9a849ac7eda426abfca97cf8b15da0f: Request end of tablet copy session db052bef7536453fa2d7f5a00e314a18-a169a099da7f404bb7804561c7aca3e6 received from {username='slave'} at 127.0.0.1:45470
I20260709 13:20:21.762003 10141 tablet_copy_service.cc:434] P e9a849ac7eda426abfca97cf8b15da0f: ending tablet copy session db052bef7536453fa2d7f5a00e314a18-a169a099da7f404bb7804561c7aca3e6 on tablet a169a099da7f404bb7804561c7aca3e6 with peer db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:21.854265 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:21.883519 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:21.996680 10899 raft_consensus.cc:1217] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
W20260709 13:20:22.031432 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.048993 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:22.097970 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:22.104390 18320 raft_consensus.cc:1064] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
W20260709 13:20:22.113257 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:22.104748  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:22.117745 18320 consensus_queue.cc:237] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:22.192554 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.195478 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.197463 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:22.220052 10899 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:20:22.256130 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.256994 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.277529 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:22.294497 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.289572 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 67463.6us
I20260709 13:20:22.303501 10200 raft_consensus.cc:1275] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:20:22.309324  8925 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:22.289505 18320 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
W20260709 13:20:22.321128 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:22.320009 18384 consensus_queue.cc:1048] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:20:22.362908 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:22.384330 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:22.387988 18288 raft_consensus.cc:2955] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.396584 10896 raft_consensus.cc:2955] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.421896 10200 raft_consensus.cc:2955] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.442898  9945 catalog_manager.cc:5697] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 13 to 14, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:22.486276 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:22.498099 18288 raft_consensus.cc:1064] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: attempting to promote NON_VOTER db052bef7536453fa2d7f5a00e314a18 to VOTER
I20260709 13:20:22.501569 18288 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:22.548779 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:22.577319 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:22.597105 10896 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 LEARNER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:22.600122 10200 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:22.602520 18286 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:22.604701 18286 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:22.630357 10096 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:22.639806 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.666553 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:22.707231 18286 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.710455 10896 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.732723 10200 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:22.773478  9947 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: config changed from index 12 to 13, db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:22.851917 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.893112 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:22.893906 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:22.926863 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:20:22.939826 10128 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:22.957224 10896 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:22.961351 10200 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:22.963593 18286 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:22.970922 18288 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:20:22.980206 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:22.984758 18286 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:23.007617 10200 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:20:23.014974 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:20:23.021380 10896 raft_consensus.cc:2955] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
I20260709 13:20:23.056070  9938 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a169a099da7f404bb7804561c7aca3e6 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:23.064116  9950 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: config changed from index 13 to 14, VOTER 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65) evicted. New cstate: current_term: 20 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } } }
W20260709 13:20:23.117816  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:23.154011 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.212065 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:23.309966  8925 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:23.379856 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.405431 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.466454 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61095.5us
W20260709 13:20:23.570576 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.585090 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.585857 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.593513 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.594314 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.607832 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.741415 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:23.748817 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:23.755779 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.874306 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:23.953419 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.007684 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.021767 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.083695 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.116600 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:24.159577 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.210997 10865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 516 similar messages]
W20260709 13:20:24.215492 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:24.225867 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:24.310868  8925 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:24.451924 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.453140 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.519773 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:24.524688  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 92789.8us
W20260709 13:20:24.547237 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47434.7us
W20260709 13:20:24.551671  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:24.610208 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.626597 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:24.639670 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.640741 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.716799 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.718586 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.779356 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.819206 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:24.834781 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:24.882174 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:24.931170 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:24.941500 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:24.965788 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.047719 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.074185 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.231102 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:25.276006 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.317281  8925 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:25.338454 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:25.361701 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:25.374073 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.412870 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:25.473649 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.526816 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:25.567988 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:25.629758 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:25.658628 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.717420  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:25.786938 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.797686 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:25.942293 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:25.996526 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.033196 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.109244 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.241333 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.300012 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:26.301011 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.317903  8925 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:26.337828 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.405867 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.425837 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.563889 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.598832 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.613320 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.629526 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.678943 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.700462 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.716338 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:26.737664 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.772511 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:26.817471 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:26.864509 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:26.878861 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:26.917276 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:26.918718 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:26.999464 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.005191 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.053432 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.084234 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.116540 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.295662 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:27.310348 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.318573  8925 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:27.338088 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:27.471132 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.498600 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.537453 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:27.555365 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:27.568598 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:27.580090 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.690109 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:27.775131 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.858506 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:27.869935  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:27.925104 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:28.121152 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.121904 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:28.161265 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.268198 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:28.312984 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:28.319175  8925 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:28.333499 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:28.349458 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:28.382787 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.388973 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.522847 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.614253 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.633635 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.640413 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.763386 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.851166 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45769.5us
W20260709 13:20:28.867254 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:28.882776 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.927500 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:28.936554 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.006064 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.171993 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.217109 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.225795 10865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 492 similar messages]
W20260709 13:20:29.260687 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.288676 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.293020 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.296527 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.320397  8925 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:29.440220 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:29.476877 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.491425 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.517546 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.545058 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:29.624933 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.666268 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.776769 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:29.828975 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.836467 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.884318 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47967.2us
W20260709 13:20:29.885334 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:29.886025 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.886668 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.920699 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:29.933403 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:29.986335 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:29.994506 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.000129 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:20:30.043582 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:30.166589 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:30.218902 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.235522 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:30.321008  8925 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:30.347805 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.418716 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.613955 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:30.673413 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 59542us
W20260709 13:20:30.682200 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:30.830132  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 52912.5us
W20260709 13:20:30.834112  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:30.819239 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.909812 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 90657.6us
W20260709 13:20:30.914256 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.967962 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.969949 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:30.970734 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:30.978359 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:30.979151 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:31.013592 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:31.024129 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.059326 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.115825 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.321801  8925 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:31.322434 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.324540 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.411881 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.473711 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.544116 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.554561 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.579183 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:31.682060 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.836402 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:31.840696 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:31.844420 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.851866 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:31.860857 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:31.891860 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.920964 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.923560 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:31.960582 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:31.961835 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.005352 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.066857 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.078061 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.082551  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:32.127696 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.146569 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.209304 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.279572 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.303256 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:32.322686  8925 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:32.347929 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.379711 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.390743 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:32.410104 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.560238 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:32.584646 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.612689 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:20:32.658227 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:32.733973 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:32.734867 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:32.784652 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:33.054994 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.065451 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.090329 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.115648 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.165055 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:33.322170 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:33.324312  8925 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:33.326242 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:33.352639 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.386103 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.407603 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.473047 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:33.537225 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:33.619468 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.661652 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.829095 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.860569 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.966271 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:33.999997 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.043744 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.085637 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:34.215297 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:34.230113 10865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 494 similar messages]
W20260709 13:20:34.256886 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:34.324926  8925 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:34.381695 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:34.387670 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.395979 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.426038 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:34.427176 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.466537 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.599296 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.605895 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:34.621953 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:34.773846 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.777465 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.783185 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 59558.1us
W20260709 13:20:34.804839 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.835210 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:34.833271  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 310248us
W20260709 13:20:34.860690  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:34.872785 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:34.873634 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 48504.5us
W20260709 13:20:34.910712 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.001185 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.034233 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.124552 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:20:35.136013 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:35.142427 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55363.1us
W20260709 13:20:35.148994 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.199586 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.221644 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:35.241524 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.319414 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:35.326174  8925 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:35.344445 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.434531 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.455133 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.499055 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.503530 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.509331 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.597882 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.727730 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.833604 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:20:35.893031 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:35.945611 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:35.955992 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:36.040804 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:36.046195 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.165534 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.169304 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.235532 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.326952 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:36.328620  8925 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:36.415941 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.680449 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.702036 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 69513.8us
W20260709 13:20:36.760267 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.773030 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:36.776646 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:36.786595 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.789325 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.793536 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:36.824837 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:36.900440 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:36.903440 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:37.329450  8925 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:38.085095 18464 raft_consensus.cc:493] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.086688 18464 raft_consensus.cc:515] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.223031 18468 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.224076 18468 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.224866 18464 leader_election.cc:290] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.227569 18464 raft_consensus.cc:493] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.221086 18465 raft_consensus.cc:493] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.217921 18467 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.230739 18467 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.229725 18465 raft_consensus.cc:515] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.236694 18468 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.237131 18464 raft_consensus.cc:515] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.236750 18465 leader_election.cc:290] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.238545 18467 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.242882 18464 leader_election.cc:290] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.249279 18464 raft_consensus.cc:493] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.253690 18465 raft_consensus.cc:493] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.262141 18473 raft_consensus.cc:493] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.254323 18467 raft_consensus.cc:493] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.267212 18467 raft_consensus.cc:515] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.266721 18473 raft_consensus.cc:515] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.269882 18467 leader_election.cc:290] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.270529 18467 raft_consensus.cc:493] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.254825 18468 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.258021 18469 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.271160 18467 raft_consensus.cc:515] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.260852 18466 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.272322 18466 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.273620 18467 leader_election.cc:290] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.255488 18470 raft_consensus.cc:493] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.274550 18470 raft_consensus.cc:515] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.262126 18471 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.275643 18471 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.280512 18471 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.271680 18469 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.277040 18470 leader_election.cc:290] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.264079 18464 raft_consensus.cc:515] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.283089 18469 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.283968 18464 leader_election.cc:290] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.277359 18466 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.262725 18465 raft_consensus.cc:515] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.286589 18465 leader_election.cc:290] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.288631 18465 raft_consensus.cc:493] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.289412 18465 raft_consensus.cc:515] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.293483 18465 leader_election.cc:290] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.294529 18465 raft_consensus.cc:493] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.295229 18465 raft_consensus.cc:515] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.297705 18465 leader_election.cc:290] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.298709 18473 leader_election.cc:290] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.304651 18466 raft_consensus.cc:493] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.305552 18466 raft_consensus.cc:515] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.309316 18466 leader_election.cc:290] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.317891 18473 raft_consensus.cc:493] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.318673 18473 raft_consensus.cc:515] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.321308 18473 leader_election.cc:290] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.321969 18473 raft_consensus.cc:493] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.322564 18473 raft_consensus.cc:515] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.328655 18473 leader_election.cc:290] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.330350  8925 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:38.280601 18468 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.333810 18470 raft_consensus.cc:493] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.334749 18470 raft_consensus.cc:515] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.336454 18468 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.336928 18469 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.337162 18468 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.337654 18469 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.337885 18468 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.338183 18467 raft_consensus.cc:493] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.341214 18467 raft_consensus.cc:515] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.347327 18469 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.337208 18470 leader_election.cc:290] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.349112 18467 leader_election.cc:290] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.358530 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 78952.6us
I20260709 13:20:38.364616 18471 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.365392 18471 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.368778 18471 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.357412 18467 raft_consensus.cc:493] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.377041 18468 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.378675 18479 raft_consensus.cc:493] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.379528 18479 raft_consensus.cc:515] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.382174 18479 leader_election.cc:290] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.385520 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:38.389089 18473 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.389477 18468 raft_consensus.cc:493] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.376796 18467 raft_consensus.cc:515] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.392268 18473 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.390759 18471 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:38.391069 10096 leader_election.cc:336] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.396054 10096 leader_election.cc:336] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.393915 18468 raft_consensus.cc:515] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.395642 18466 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.413144 18467 leader_election.cc:290] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.397596 18464 raft_consensus.cc:493] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.425318 18465 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.426231 18470 raft_consensus.cc:493] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.425297 18474 raft_consensus.cc:493] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.427517 18470 raft_consensus.cc:515] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.427793 18474 raft_consensus.cc:515] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.430387 18470 leader_election.cc:290] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.426378 18467 raft_consensus.cc:493] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.431031 18470 raft_consensus.cc:493] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.432207 18474 leader_election.cc:290] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.434453 18467 raft_consensus.cc:515] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.434773 18474 raft_consensus.cc:493] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.436349 18474 raft_consensus.cc:515] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.430907 18465 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.440997 18466 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.443926 18466 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.437094 18467 leader_election.cc:290] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.440997 18464 raft_consensus.cc:515] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.433513 18470 raft_consensus.cc:515] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.400516 18471 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.443929 18465 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.497516 18464 leader_election.cc:290] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:38.499162 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:38.499835 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 103049us
W20260709 13:20:38.500341 10096 leader_election.cc:336] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.504825 18468 leader_election.cc:290] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.505129 18482 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.505952 18482 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.506287 18473 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.510553 18473 raft_consensus.cc:493] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.511586 18473 raft_consensus.cc:515] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.526428 18473 leader_election.cc:290] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.527367 18471 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.527545 18468 raft_consensus.cc:493] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.472500 18474 leader_election.cc:290] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.534257 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55225.8us
I20260709 13:20:38.542644 18483 raft_consensus.cc:493] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.543542 18483 raft_consensus.cc:515] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.546976 18483 leader_election.cc:290] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.548635 18464 raft_consensus.cc:493] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.541710 18468 raft_consensus.cc:515] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.569634 18468 leader_election.cc:290] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.570355 18481 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.571156 18481 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.574365 18481 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.577520 18482 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.579305 18470 leader_election.cc:290] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.591504 10096 leader_election.cc:336] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.593448 10096 leader_election.cc:336] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.594228 10096 leader_election.cc:336] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.595613 10096 leader_election.cc:336] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.596330 10096 leader_election.cc:336] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.590588 18474 raft_consensus.cc:493] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.597412 18474 raft_consensus.cc:515] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.600208 18474 leader_election.cc:290] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.605260 10096 leader_election.cc:336] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.606127 10096 leader_election.cc:336] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.560003 18473 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
W20260709 13:20:38.606943 10096 leader_election.cc:336] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.608253 10096 leader_election.cc:336] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.595106 18479 raft_consensus.cc:493] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.563766 18464 raft_consensus.cc:515] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:38.612898 10096 leader_election.cc:336] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.617347 18479 raft_consensus.cc:515] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.618417 18473 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:38.621981 10096 leader_election.cc:336] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.623333 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.625798 10096 leader_election.cc:336] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.615984 18474 raft_consensus.cc:493] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.625145 18473 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.624524 18484 raft_consensus.cc:493] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.627547 18464 leader_election.cc:290] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.641185 18484 raft_consensus.cc:515] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.623553 18479 leader_election.cc:290] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.636826 18494 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.637668 18477 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.647603 18477 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.638882 18492 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.650032 18492 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.652359 18494 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.639485 18471 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.639595 18493 raft_consensus.cc:493] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.640199 18474 raft_consensus.cc:515] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.655896 18493 raft_consensus.cc:515] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.624153 18489 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.666308 18489 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:38.674773 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44528.7us
I20260709 13:20:38.712281 18474 leader_election.cc:290] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.706704 18483 raft_consensus.cc:493] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.713665 18489 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.713846 18483 raft_consensus.cc:515] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.716480 18483 leader_election.cc:290] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:38.734136 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 58526.4us
I20260709 13:20:38.738226 18482 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.739017 18482 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.741539 18482 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.743289 18489 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.743906 18489 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.744376 18478 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.745023 18478 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.746292 18489 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.747488 18466 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.748133 18466 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.655071 18471 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.754835 18466 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.756995 18482 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.758049 18482 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.768623 18466 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.777305 18478 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.777376 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2219e1af873d4a00966a8c5975eb7987" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.771421 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.779897 10127 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:20:38.781260 18490 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.782092 18490 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.771677 18471 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.777529 18466 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:38.785389 18466 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.793133 18487 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.794113 18487 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.796814 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2869566cb59547689f49f6e04511ed8a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.799665 18482 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.800936 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.801857 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.802630 10126 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 20.
I20260709 13:20:38.784750 18490 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.804942 18488 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.805826 18488 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.808982 18488 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.809582 18488 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.810163 18488 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.811751 18493 leader_election.cc:290] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:38.813253 18477 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:38.803731 10096 leader_election.cc:336] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.821892 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:38.822492 10096 leader_election.cc:336] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.823380 10096 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.826444 18484 leader_election.cc:290] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.827530 18494 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.828567 18492 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:38.824259 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:38.837908 10096 leader_election.cc:336] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.840432 10096 leader_election.cc:336] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.841248 10096 leader_election.cc:336] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.841837 10096 leader_election.cc:336] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.796923 18487 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.839784 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.844038 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "161f2d2f2674494bbffbd5ee927cf5e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.846089 10865 leader_election.cc:336] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.847699 10096 leader_election.cc:336] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.844739 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.851406 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.852018 18498 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.852203 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.852761 18498 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.853228 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a353f267acea40eaaf03befcb32892f6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.856724 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c048cf8265e4264b35912a8caac9936" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.859263 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.859829 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.860467 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.861311 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fd90bb718ac49bf84e09fe79d462240" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.862205 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c7c855af77d40ab995a18b5609de8f5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.863541 10122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e14489a089594151b58748865ea36e81" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.864874 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3173f90d855c4371aeafe17b5f760205" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.865362 10127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "231b54e25cca40f6a3b0625d68e97cf7" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.865964 10130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d36b9784d64ff48d0d4f0224266506" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.864975 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d07e888fde29492b84e575218308b6ff" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.866751 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb08df714384523b94354944a523e54" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.868330 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.849838 10202 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:38.863662 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c502c18433fb46e2ac2a42d7d8746ce0" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.872798 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0c93dea5060480eb0618f168078f4e6" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.871547 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d6c92980e764b60a913b8c5533ed9ad" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.852293 10123 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
W20260709 13:20:38.880857 10096 leader_election.cc:336] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.885939 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87da05694091470284f66a8181207e2f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.887645 10096 leader_election.cc:336] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.896947 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.899971 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed07b7c3185a4d5d9fbcba31d80ed8f6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:38.900548 18488 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:38.905387 18481 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:38.906086 18481 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.908493 18481 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.910391 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.911021 18489 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.911355 10199 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 25.
I20260709 13:20:38.911780 18489 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.912421 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "371654d6e0a94cba825fa080a9998940" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.914732 18498 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:38.915426 10865 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.916409 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.912551 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fb38c54abe4cd4bb552eb544020f47" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.917761 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:38.918505 10865 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.919739 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d51d657bc4fac8f4199fbc38600c1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.920212 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79954d719dc94974b4864ff715e9ef07" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.923363 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3220b9037e44cae95e49af43de3523a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.923805 18489 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.927964 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c485d13f66d4ce8bdca72b60d242842" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.926139 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.925673 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.927284 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2169fe5ebfe5416c94b7d9095d9af6da" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.932487 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22230bc6afb42ceac52420e42db0292" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.934679 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
W20260709 13:20:38.937418 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.938771 18488 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:38.939438 18492 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:38.939579 10196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82960478409442549781547254b9d8ea" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.940304 18492 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.939509 18488 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:38.940842 18492 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:38.943289 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.943634 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee79852a0bc04642b58b690930f78267" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.943687 18488 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:38.944119 10204 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 20.
I20260709 13:20:38.946457 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aca8a04bf8df4212bfdf5fd990503c0a" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.904834 10096 leader_election.cc:336] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.948675 10197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b09e10904af344988a1a11ae9e576799" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.949153 10096 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44367.5us
W20260709 13:20:38.949728 10096 leader_election.cc:336] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.950253 10096 leader_election.cc:336] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.950759 10096 leader_election.cc:336] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.951293 10096 leader_election.cc:336] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.951726 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfacd39092d24ba5a33d0cd9451b91a1" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
W20260709 13:20:38.952132 10096 leader_election.cc:336] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:38.953137 10096 leader_election.cc:336] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.953521 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09ed53b9cff42a29c1d6c06e009ee94" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.955341 10198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5853a69cde724bc3a143613bf68a63fc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:38.962981 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.963352 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.963954 10896 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
W20260709 13:20:38.965588 10865 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:38.972107 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d97ed0ec5f214d95abb9e01d368a04b5" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.978286 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.979192 10899 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 27.
I20260709 13:20:38.980485 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:38.983090 10095 leader_election.cc:304] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:38.984754 18474 raft_consensus.cc:2749] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:38.986208 10095 leader_election.cc:304] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:38.953688 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:38.987635 18473 raft_consensus.cc:2749] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:38.988497 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b81db48ec24878966db719b2790127" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.991043 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:38.991680 10095 leader_election.cc:304] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:38.991816 10896 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 15.
I20260709 13:20:38.992595 18473 raft_consensus.cc:2749] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:38.995406 18499 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:38.996145 18499 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:38.996654 18499 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:38.996533 10095 leader_election.cc:304] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.004601 10095 leader_election.cc:304] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.004905 18473 raft_consensus.cc:2749] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.005985 18473 raft_consensus.cc:2749] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.009702 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a895dc8dee14dfba216f443121474f4" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.011698 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.006726 10095 leader_election.cc:304] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.014081 18474 raft_consensus.cc:2749] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.027789 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e46fa8cb2404116bf566b4ef9bbb2cf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.014659 10095 leader_election.cc:304] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.031908 18474 raft_consensus.cc:2749] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.035327 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.035619 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.036562 18474 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:39.037109 18474 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:39.037567 18474 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:39.037933 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.038769 10128 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:20:39.042870 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a7190358a9f475d82e5f9875677bccf" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.045156 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.045431 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.047398 10126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.049985 10128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5af05c8079540ec84b0cc3dc1a9e051" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.052615 10200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.053174 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.054445 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.057111 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.057493 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.059080 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a82842926e645c08dde847bfe182c31" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.061733 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f" is_pre_election: true
I20260709 13:20:39.062834 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.065088 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.065832 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ece395e49d7e474182be2d25642b90de" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.066066 10899 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 22.
I20260709 13:20:39.068141 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.068760 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75ddb7f5e3204a51b3dafe4903888dcc" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.068882 10896 raft_consensus.cc:2468] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 19.
I20260709 13:20:39.070257 10898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.071470 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.071924 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.071552 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a122c89ca68646b8bf88dbb1d5dcbfa8" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.073426 10892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf88b6d5c6504b3cb2cbd21cff68e57e" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.073678 18492 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.075099 10201 raft_consensus.cc:1240] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:39.075030 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82406372d02c4dd59d4ef5c32436e148" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.076905 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:39.079469 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:39.091948 18499 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.093616 10202 raft_consensus.cc:1240] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:39.094411 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.096146 18488 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:39.097764 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 62551.3us
I20260709 13:20:39.098913 10095 leader_election.cc:304] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.100152 18484 raft_consensus.cc:2749] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.102059 10095 leader_election.cc:304] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.102937 10863 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.123692 18488 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:20:39.103919 10921 debug-util.cc:398] Leaking SignalData structure 0x7b0800252e40 after lost signal to thread 8928
I20260709 13:20:39.124279 18488 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:39.124816 18488 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:39.126024 10921 debug-util.cc:398] Leaking SignalData structure 0x7b0800040880 after lost signal to thread 9988
W20260709 13:20:39.128501 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08002302e0 after lost signal to thread 10155
I20260709 13:20:39.110430 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.124310 10863 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.131793 18493 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:39.132254 18493 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:39.132607 18493 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:39.133065 10863 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.134339 18481 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.118345 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5391e406039d4af8bf2b17f1b2034ef5" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.119530 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.138157 10863 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.145910 18489 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.120965 18474 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.149498 18474 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.121865 18499 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:39.122525 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.135701 18481 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:39.119777 18492 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:39.130486 10921 debug-util.cc:398] Leaking SignalData structure 0x7b08001d2520 after lost signal to thread 10229
I20260709 13:20:39.140367 18484 raft_consensus.cc:2749] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:39.156343 10921 debug-util.cc:398] Leaking SignalData structure 0x7b0800307220 after lost signal to thread 10924
I20260709 13:20:39.160288 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:39.151202 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.162194 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
W20260709 13:20:39.162442 10170 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.141587 10095 leader_election.cc:304] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.163966 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.168058 18484 raft_consensus.cc:2749] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.171742 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66968e5dbd3040d290ce9716c146d46a" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:39.173794 10095 leader_election.cc:304] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.173978 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.174952 18484 raft_consensus.cc:2749] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:39.176013 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.176698 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.177596 18484 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:39.178681 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.178745 18488 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.181285 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.182703 10204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.184546 10129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:39.182987 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3664f114a52481c8f97e06e9b2d2068" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:39.163036 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:39.192569 10170 leader_election.cc:336] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.193284 18499 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:39.193693 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.194183 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.196708 18492 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.205921 10169 leader_election.cc:304] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.207162 18492 raft_consensus.cc:2749] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.207463 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:39.207960 10169 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.207717 18488 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:39.217350 18492 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.222500 10864 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:39.231618 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:20:39.235046 10169 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.235427 18488 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:20:39.238443 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 61850.3us
I20260709 13:20:39.239030 18494 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.240142 10169 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.240962 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.239517 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.248790 18488 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:39.247366 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 53695.5us
I20260709 13:20:39.250453 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.251173 18484 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:39.252727 18484 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:39.253280 18484 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:39.260247 18489 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.260385 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.261075 18494 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.261488 18489 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.198462 18493 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.262619 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.251941 10095 leader_election.cc:304] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.278965 18473 raft_consensus.cc:2749] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:39.246691 10096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 348 similar messages]
I20260709 13:20:39.288911 10095 leader_election.cc:304] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.289541 10168 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.291146 18479 raft_consensus.cc:2749] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:39.292315 18489 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:39.293247 18489 consensus_queue.cc:1059] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:39.294005 18482 consensus_queue.cc:1059] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:39.294529 18488 raft_consensus.cc:3055] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:39.295078 18488 raft_consensus.cc:740] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:39.295352 18481 raft_consensus.cc:3055] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:39.295872 18481 raft_consensus.cc:740] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:39.295941 10864 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.276952 18493 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.296255 18488 consensus_queue.cc:260] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.297349 18478 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:39.297083 18481 consensus_queue.cc:260] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.298216 18488 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:39.298843 18481 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:39.300231 10095 leader_election.cc:304] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.298874 10168 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.301788 18484 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.303188 18494 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.304168 18482 raft_consensus.cc:2749] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.304808 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.305754 18482 raft_consensus.cc:2749] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:39.306656 18484 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.307515 18505 raft_consensus.cc:2749] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:39.309298 10095 leader_election.cc:304] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.319665 18484 raft_consensus.cc:2749] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:39.320907 10095 leader_election.cc:304] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.314554 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.325089 18484 raft_consensus.cc:2749] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:39.326143 10095 leader_election.cc:304] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.327636 10095 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:39.330945  8925 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:39.330916 18505 raft_consensus.cc:2749] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:39.331355 18482 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.332156 18484 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.333649 10095 leader_election.cc:304] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.334568 18484 raft_consensus.cc:2749] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.334493 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.336321 10095 leader_election.cc:304] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.337384 18484 raft_consensus.cc:2749] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.338928 10095 leader_election.cc:304] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.340296 18484 raft_consensus.cc:2749] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:39.342330 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:39.356997 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:39.357617 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.358649 18484 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:39.359213 18484 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader db052bef7536453fa2d7f5a00e314a18)
I20260709 13:20:39.358974 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.359674 18484 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:39.363332 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:39.364516 10095 leader_election.cc:304] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.365432 10123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e9a849ac7eda426abfca97cf8b15da0f"
I20260709 13:20:39.368790 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.381743 18488 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.382863 18505 raft_consensus.cc:2749] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.381389 18484 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.430800 10896 raft_consensus.cc:1240] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:39.432782 18484 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.391880 10095 leader_election.cc:304] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.415143 10168 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:39.426767 10170 leader_election.cc:336] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.435365 18505 raft_consensus.cc:2749] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.435886 10170 leader_election.cc:304] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.436225 10095 leader_election.cc:304] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.436874 18477 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:20:39.437393 18474 raft_consensus.cc:2749] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:39.437403 18483 raft_consensus.cc:2749] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:39.438377 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.438885 10095 leader_election.cc:304] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:39.440582 10095 leader_election.cc:304] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:39.441169 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.442265 18505 raft_consensus.cc:2749] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:39.443327 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.444056 10095 leader_election.cc:304] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
W20260709 13:20:39.444480 10170 leader_election.cc:336] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.445030 10170 leader_election.cc:304] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.445178 18505 raft_consensus.cc:2749] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.441313 18484 raft_consensus.cc:2749] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.446226 18496 raft_consensus.cc:2749] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:39.446262 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.453130 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.458181 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.464071 10205 raft_consensus.cc:2393] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 21: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:39.464114 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.448621 10202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.466837 10202 raft_consensus.cc:2393] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e9a849ac7eda426abfca97cf8b15da0f in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
I20260709 13:20:39.468967 18496 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:39.470391 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:39.471030 10170 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.471149 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:39.476027 18477 consensus_queue.cc:1059] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 13:20:39.472486 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.487039 18496 raft_consensus.cc:3055] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:39.487327 18505 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:39.487982 18505 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:39.488452 18505 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:39.484902 10170 leader_election.cc:336] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.498070 10123 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
W20260709 13:20:39.498275 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:39.499243 18496 raft_consensus.cc:740] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:39.500105 18482 consensus_queue.cc:1059] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:20:39.501395 18482 raft_consensus.cc:3055] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:20:39.501776 18482 raft_consensus.cc:740] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:39.502671 18482 consensus_queue.cc:260] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.505316 18482 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:39.507401 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.509248 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.518324 10095 leader_election.cc:304] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.495472 10170 leader_election.cc:304] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.519627 18484 raft_consensus.cc:2749] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:39.520159 10170 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.520740 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.521381 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:39.521605 18494 raft_consensus.cc:2749] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:39.522126 18477 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:39.524184 18474 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:20:39.524767 18474 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:39.525246 18474 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:20:39.526371 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 21.
W20260709 13:20:39.531666 10170 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.500555 18496 consensus_queue.cc:260] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.540112 18496 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:39.538107 10095 leader_election.cc:304] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.547430 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:39.541316 10896 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 24.
I20260709 13:20:39.550017 18479 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:39.550575 18479 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:39.550987 18479 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:39.550802 10095 leader_election.cc:304] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.552305 18473 raft_consensus.cc:2749] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:39.537806 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.548297 18484 raft_consensus.cc:2749] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:39.543171 10201 raft_consensus.cc:1240] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Rejecting Update request from peer db052bef7536453fa2d7f5a00e314a18 for earlier term 25. Current term is 26. Ops: []
I20260709 13:20:39.555421 18477 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:39.557482 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.558423 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.559325 18477 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:39.561942 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:39.562875 10170 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:39.563663 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:39.541380 18474 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.568064 18505 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.569047 18474 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.570489 18505 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.572026 18479 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.574749 18479 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.585179 18477 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.593947 10893 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 26.
W20260709 13:20:39.627383 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 73710.2us
I20260709 13:20:39.628684 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: d51ad94710c348889d417f7f00032948
I20260709 13:20:39.637312 18473 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:39.638015 18473 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:39.638434 18473 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:39.638635 10095 leader_election.cc:304] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.644223 18479 raft_consensus.cc:2749] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:39.647600 10095 leader_election.cc:304] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.661475 10126 raft_consensus.cc:1240] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 27. Current term is 28. Ops: []
W20260709 13:20:39.668105 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:39.662853 18473 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.686532 18479 raft_consensus.cc:2749] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:39.687508 18473 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Requested vote from peers d51ad94710c348889d417f7f00032948 (127.8.183.67:40505), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:39.688210 18477 consensus_queue.cc:1059] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:20:39.689554 18477 raft_consensus.cc:3055] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:20:39.690012 18477 raft_consensus.cc:740] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:39.691713 10126 raft_consensus.cc:1240] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:39.691005 18477 consensus_queue.cc:260] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.693181 18477 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:20:39.696792 18496 consensus_queue.cc:1059] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:20:39.691365 10095 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.703830 18473 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:39.703986 10095 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.704656 18473 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:39.705502 10095 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f; no voters: 
I20260709 13:20:39.705829 18479 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:39.706524 18506 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:20:39.705997 18473 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.707235 18506 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:39.707499 10129 raft_consensus.cc:1240] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Rejecting Update request from peer d51ad94710c348889d417f7f00032948 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:39.708309 18506 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.706521 18479 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:39.712641 18483 raft_consensus.cc:3055] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:39.712448 18479 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.713243 18483 raft_consensus.cc:740] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:39.714450 18483 consensus_queue.cc:260] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.716645 18483 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:39.731853 18492 consensus_queue.cc:1059] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:39.733877 18490 raft_consensus.cc:3055] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:39.734422 18490 raft_consensus.cc:740] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:39.735589 18490 consensus_queue.cc:260] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.737108 18490 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:39.747256 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.748821 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.750085 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.753859 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:39.765827 10203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.765157 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.769358 10199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:39.772681 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
W20260709 13:20:39.791342 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:39.836557 10203 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:39.840535 10205 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 23.
I20260709 13:20:39.842653 10201 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 28.
I20260709 13:20:39.842547 10199 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9a849ac7eda426abfca97cf8b15da0f in term 16.
W20260709 13:20:39.923552 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 101330us
I20260709 13:20:39.922706  9947 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 23 to 24, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 24 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:39.931797  9947 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 20 to 21, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 21 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:39.936096 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:20:39.946947 10095 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.958838 18512 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:39.968060 18512 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:39.969429 18512 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:39.974959 10095 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:39.982372 18473 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:20:39.983193 18473 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:39.984157 18473 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:39.990420 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:39.991312 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:39.991899 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:39.995278 10095 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
W20260709 13:20:40.007222 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:40.008059 18479 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:40.008675 18479 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
W20260709 13:20:40.009832 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:40.009740 18479 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:40.010419 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:40.012380 10095 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f; no voters: db052bef7536453fa2d7f5a00e314a18
I20260709 13:20:40.013502 18511 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:40.014134 18511 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Becoming Leader. State: Replica: e9a849ac7eda426abfca97cf8b15da0f, State: Running, Role: LEADER
I20260709 13:20:40.015244 18511 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:40.055581 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:40.099959 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:20:40.104486 10199 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:40.117403 18510 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:20:40.141216 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:40.161890 10199 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:40.209827 18510 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:20:40.274216 10896 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:40.276413 18510 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:20:40.313566 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:40.331702  8925 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 10077)
I20260709 13:20:40.341316 10201 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:20:40.370096 18510 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:20:40.374836 10199 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:20:40.377012 18511 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:20:40.399658 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 96186.4us
I20260709 13:20:40.423883 10899 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:40.434455 10894 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 13:20:40.456197  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 44473us
I20260709 13:20:40.467273  9947 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 15 to 16, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 16 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:40.469650  9947 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 25 to 26, leader changed from db052bef7536453fa2d7f5a00e314a18 (127.8.183.68) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 26 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:40.471767  9947 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 22 to 23, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 23 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:40.473600  9947 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 27 to 28, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 28 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:40.483521 18479 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:20:40.447887 18512 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 13:20:40.502224 10894 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:20:40.507517 18512 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:20:40.481498  9947 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f reported cstate change: term changed from 22 to 23, leader changed from d51ad94710c348889d417f7f00032948 (127.8.183.67) to e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66). New cstate: current_term: 23 leader_uuid: "e9a849ac7eda426abfca97cf8b15da0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:40.520442 10203 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 13:20:40.526718 10892 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:20:40.528528 18473 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 13:20:40.537453 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:40.561741  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:40.561790 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:20:40.573300 10892 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 13:20:40.574764 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:40.583009 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:40.619553 10205 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:40.688284 10897 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:40.662304 18474 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 13:20:40.712415 10203 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:20:40.751827 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:20:40.780738 10897 raft_consensus.cc:1275] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:40.825387 18479 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:40.825946 18505 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:40.837023 18479 consensus_queue.cc:1048] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:40.841991 10894 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:40.869623 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:40.887722 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:40.905155 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:40.950163 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:40.952855 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:40.950913 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:40.962723 10894 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Refusing update from remote peer e9a849ac7eda426abfca97cf8b15da0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:40.973055 18473 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
W20260709 13:20:40.985226 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 227303us
W20260709 13:20:41.054365 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:41.064206 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 122336us
W20260709 13:20:41.085528 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.086530 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.087329 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:41.134543 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.167327 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.199051 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:41.230211 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:41.252807 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:41.253407 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.267436 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:41.332373  8925 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:41.376277 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.387879 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.532922 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.552659 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.698751 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:41.714486 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:41.731897 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 72826.4us
W20260709 13:20:41.732132 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:41.745513 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:41.807212 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:41.877192 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:42.041860 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:42.058362 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:42.103837 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:42.152210 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:42.191869 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:42.228404 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:42.333024  8925 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:42.351219 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:42.364045 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:42.419562 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 55602.5us
W20260709 13:20:42.819749 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:43.182991 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 77010.2us
W20260709 13:20:43.275940 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 64602.8us
W20260709 13:20:43.278632 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.334712  8925 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:43.338390 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:43.351815 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:43.352917 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:43.353572 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:20:43.354221 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:43.355091 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:43.356613 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.374670 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.390327 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:43.588743 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:43.615783 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:43.635237 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.648247 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.658507 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.663697 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.671619 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.710044 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:43.729713 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:43.754609 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.785254 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:43.803488 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:43.909178 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1f40 after lost signal to thread 8928
W20260709 13:20:43.910749 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7d00 after lost signal to thread 9988
W20260709 13:20:43.912612 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac960 after lost signal to thread 10155
W20260709 13:20:43.914140 10226 debug-util.cc:398] Leaking SignalData structure 0x7b08003ecca0 after lost signal to thread 10229
W20260709 13:20:43.922227 10226 debug-util.cc:398] Leaking SignalData structure 0x7b080016f020 after lost signal to thread 10924
W20260709 13:20:44.001824 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.082751 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:44.084092 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.176892 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:44.175647 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.215071 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:44.226985 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.258885 10170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 465 similar messages]
W20260709 13:20:44.325240 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:44.335387  8925 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:44.343283 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:44.362064 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:44.379221 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.418637 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.469336 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:44.498590 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:44.612346 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:44.686443 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.741185 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:44.741792 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:44.772042 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:44.846962 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 66154.1us
W20260709 13:20:44.931874 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 79151.4us
W20260709 13:20:44.935735 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 135376us
W20260709 13:20:44.989087 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:44.997207 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.018643 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 53129.4us
W20260709 13:20:45.029037 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.138795 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.187675 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 251826us
W20260709 13:20:45.337634  8925 thread.cc:527] Waited for 26000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:45.350303 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:45.350643 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.468014 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.489351 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.546377 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.564365 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.655282 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.680876 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:45.691781 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.723469 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:20:45.928121 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:45.951701 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:45.982766 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:46.021404 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:46.023694 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.061187 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:46.089860 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.184631 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.192420 10096 consensus_peers.cc:597] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.206161 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.226395 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.230168 10096 consensus_peers.cc:597] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.249946 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:46.254904 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:46.258069 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.267707 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.338331  8925 thread.cc:527] Waited for 27000ms trying to join with diag-logger (tid 10077)
W20260709 13:20:46.443738 10096 consensus_peers.cc:597] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.486467 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:46.544075 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:46.622546 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.685031 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.836856 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.857410 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.858038 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.858558 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.926577 10096 consensus_peers.cc:597] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:46.939359 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:46.975406 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:46.978749 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:47.032915 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:47.175442 10096 consensus_peers.cc:597] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:47.193037 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:47.223453 10096 consensus_peers.cc:597] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:47.244239 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 84918.3us
W20260709 13:20:47.295286  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:47.430910 10095 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 46501.4us
W20260709 13:20:47.448138 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 55795.2us
W20260709 13:20:47.519987 10096 consensus_peers.cc:597] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:47.504056 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:47.703974 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:47.705945 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 65767.2us
W20260709 13:20:47.725697 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:47.748674  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:20:47.838989 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 57028.6us
W20260709 13:20:47.908339 10096 consensus_peers.cc:597] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:47.911773 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 47057.6us
W20260709 13:20:47.916239 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:47.973958 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:47.982906 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:47.984522 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:48.062986 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.115752 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.116585 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.143239 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:48.151651 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.241199 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:48.254359 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:48.262125 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:48.538861  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
W20260709 13:20:48.661478 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:48.662130 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:48.662897 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:48.679080 10863 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.745302 10863 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.766523 10863 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.776530 10863 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.784477 10863 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.791859 10096 consensus_peers.cc:597] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:48.799546 10168 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.808095 10168 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.808910 10168 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.809746 10168 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.810526 10168 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.799615 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:48.812052 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.812951 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:48.813509 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:20:48.814127 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.815521 10168 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.818449 10168 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.824590 10168 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.860285 10168 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.862375 10168 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.863701 10168 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.865083 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:48.868151 10168 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.869028 10168 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.869755 10168 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.870420 10168 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.871089 10168 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.871738 10168 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.872370 10168 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.873024 10168 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.873667 10168 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.881882 10168 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.887014 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:48.887082  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:20:48.891057 10168 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.892083  8925 tablet_replica.cc:333] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
W20260709 13:20:48.892150 10168 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:48.893438 10168 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.893838  8925 raft_consensus.cc:2243] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:48.894212 10168 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.895256  8925 raft_consensus.cc:2272] T 87da05694091470284f66a8181207e2f P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:48.895289 10168 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.899669  8925 tablet_replica.cc:333] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.900856  8925 raft_consensus.cc:2243] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.902328  8925 raft_consensus.cc:2272] T cebbf225b866442ab895577b0496de1b P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.906054  8925 tablet_replica.cc:333] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.907181  8925 raft_consensus.cc:2243] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.908061  8925 raft_consensus.cc:2272] T 9a895dc8dee14dfba216f443121474f4 P e9a849ac7eda426abfca97cf8b15da0f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.911914  8925 tablet_replica.cc:333] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.913239  8925 raft_consensus.cc:2243] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.914902  8925 raft_consensus.cc:2272] T e14489a089594151b58748865ea36e81 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:48.917986 10168 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.919095  8925 tablet_replica.cc:333] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.920372  8925 raft_consensus.cc:2243] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.921905  8925 raft_consensus.cc:2272] T dc97aed5eb284c0e86c74b20d18b4399 P e9a849ac7eda426abfca97cf8b15da0f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.925585  8925 tablet_replica.cc:333] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.926781  8925 raft_consensus.cc:2243] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.928175  8925 raft_consensus.cc:2272] T 5853a69cde724bc3a143613bf68a63fc P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.931953  8925 tablet_replica.cc:333] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.933090  8925 raft_consensus.cc:2243] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.934819  8925 raft_consensus.cc:2272] T c5af05c8079540ec84b0cc3dc1a9e051 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.938511  8925 tablet_replica.cc:333] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.939749  8925 raft_consensus.cc:2243] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.941352  8925 raft_consensus.cc:2272] T a169a099da7f404bb7804561c7aca3e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.944818  8925 tablet_replica.cc:333] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.945860  8925 raft_consensus.cc:2243] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.947002  8925 raft_consensus.cc:2272] T f09ed53b9cff42a29c1d6c06e009ee94 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.950559  8925 tablet_replica.cc:333] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.951714  8925 raft_consensus.cc:2243] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.953307  8925 raft_consensus.cc:2272] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:48.957159 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:48.957113  8925 tablet_replica.cc:333] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.959162  8925 raft_consensus.cc:2243] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.961125  8925 raft_consensus.cc:2272] T 4ce48110a5fb4ffc97e81117165e3ffd P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.964811  8925 tablet_replica.cc:333] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.965929  8925 raft_consensus.cc:2243] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:20:48.967635  8925 raft_consensus.cc:2272] T 806f35e66c234d569cbafd8b3981518f P e9a849ac7eda426abfca97cf8b15da0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.972452  8925 tablet_replica.cc:333] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.973616  8925 raft_consensus.cc:2243] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.974581  8925 raft_consensus.cc:2272] T ee79852a0bc04642b58b690930f78267 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:48.975591 10168 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:48.978250  8925 tablet_replica.cc:333] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.979808  8925 raft_consensus.cc:2243] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.980870  8925 raft_consensus.cc:2272] T 82960478409442549781547254b9d8ea P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.984109  8925 tablet_replica.cc:333] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.985112  8925 raft_consensus.cc:2243] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.986002  8925 raft_consensus.cc:2272] T 161f2d2f2674494bbffbd5ee927cf5e6 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.989463  8925 tablet_replica.cc:333] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.990482  8925 raft_consensus.cc:2243] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.991745  8925 raft_consensus.cc:2272] T a0c93dea5060480eb0618f168078f4e6 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:48.996009  8925 tablet_replica.cc:333] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:48.997068  8925 raft_consensus.cc:2243] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:48.998222  8925 raft_consensus.cc:2272] T c502c18433fb46e2ac2a42d7d8746ce0 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.004491  8925 tablet_replica.cc:333] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.005643  8925 raft_consensus.cc:2243] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.007159  8925 raft_consensus.cc:2272] T 6278932947044bdbb7b0b1e14eea574b P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.010690  8925 tablet_replica.cc:333] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.011737  8925 raft_consensus.cc:2243] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.013281  8925 raft_consensus.cc:2272] T dbf6e6a0da3d4501acf0e5451cae4577 P e9a849ac7eda426abfca97cf8b15da0f [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.017102  8925 tablet_replica.cc:333] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.018177  8925 raft_consensus.cc:2243] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.019392  8925 raft_consensus.cc:2272] T 986d51d657bc4fac8f4199fbc38600c1 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.023205  8925 tablet_replica.cc:333] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.024252  8925 raft_consensus.cc:2243] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.025827  8925 raft_consensus.cc:2272] T 87d36b9784d64ff48d0d4f0224266506 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.029537  8925 tablet_replica.cc:333] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
W20260709 13:20:49.030428 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:49.030627  8925 raft_consensus.cc:2243] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.035100  8925 raft_consensus.cc:2272] T bfb08df714384523b94354944a523e54 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.039501  8925 tablet_replica.cc:333] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.040529  8925 raft_consensus.cc:2243] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.041464  8925 raft_consensus.cc:2272] T d97ed0ec5f214d95abb9e01d368a04b5 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.045821  8925 tablet_replica.cc:333] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.050593  8925 raft_consensus.cc:2243] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.051870  8925 raft_consensus.cc:2272] T 73fb38c54abe4cd4bb552eb544020f47 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.055369  8925 tablet_replica.cc:333] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.058346  8925 raft_consensus.cc:2243] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.059672  8925 raft_consensus.cc:2272] T aca8a04bf8df4212bfdf5fd990503c0a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.063843  8925 tablet_replica.cc:333] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.065416  8925 raft_consensus.cc:2243] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.066344  8925 raft_consensus.cc:2272] T 79954d719dc94974b4864ff715e9ef07 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.069823  8925 tablet_replica.cc:333] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.070904  8925 raft_consensus.cc:2243] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.071784  8925 raft_consensus.cc:2272] T f3220b9037e44cae95e49af43de3523a P e9a849ac7eda426abfca97cf8b15da0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.074910  8925 tablet_replica.cc:333] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.075925  8925 raft_consensus.cc:2243] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.077261  8925 raft_consensus.cc:2272] T 95b81db48ec24878966db719b2790127 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.080783  8925 tablet_replica.cc:333] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.081861  8925 raft_consensus.cc:2243] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.088083  8925 raft_consensus.cc:2272] T 2c048cf8265e4264b35912a8caac9936 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.095217  8925 tablet_replica.cc:333] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.096508  8925 raft_consensus.cc:2243] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.098225  8925 raft_consensus.cc:2272] T 5ecb570a023c42ada8943a1516f3ef4f P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.102300  8925 tablet_replica.cc:333] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.104552  8925 raft_consensus.cc:2243] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.106909  8925 raft_consensus.cc:2272] T 1c7c855af77d40ab995a18b5609de8f5 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.121649  8925 tablet_replica.cc:333] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.122953  8925 raft_consensus.cc:2243] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.125423  8925 raft_consensus.cc:2272] T 98f63a37309c453f9fd7bc3f75659536 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.132085 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.134573 10096 consensus_peers.cc:597] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:49.137420  8925 tablet_replica.cc:333] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.138514  8925 raft_consensus.cc:2243] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.139611  8925 raft_consensus.cc:2272] T b09e10904af344988a1a11ae9e576799 P e9a849ac7eda426abfca97cf8b15da0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.144018  8925 tablet_replica.cc:333] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.145069  8925 raft_consensus.cc:2243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.147938  8925 raft_consensus.cc:2272] T 5e46fa8cb2404116bf566b4ef9bbb2cf P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.173791  8925 tablet_replica.cc:333] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.175083  8925 raft_consensus.cc:2243] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.181121  8925 raft_consensus.cc:2272] T 2869566cb59547689f49f6e04511ed8a P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.200541  8925 tablet_replica.cc:333] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.201829  8925 raft_consensus.cc:2243] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.205514  8925 raft_consensus.cc:2272] T 231b54e25cca40f6a3b0625d68e97cf7 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.210472 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.215757 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.224167 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:49.228976  8925 tablet_replica.cc:333] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.230334  8925 raft_consensus.cc:2243] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.232059  8925 raft_consensus.cc:2272] T c3664f114a52481c8f97e06e9b2d2068 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.236091  8925 tablet_replica.cc:333] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.237140  8925 raft_consensus.cc:2243] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.238816  8925 raft_consensus.cc:2272] T 2219e1af873d4a00966a8c5975eb7987 P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.254503 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:20:49.263952 10168 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 531 similar messages]
I20260709 13:20:49.272234  8925 tablet_replica.cc:333] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.274185  8925 raft_consensus.cc:2243] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.275493  8925 raft_consensus.cc:2272] T 4d6c92980e764b60a913b8c5533ed9ad P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.297014 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:49.279321  8925 tablet_replica.cc:333] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.298784  8925 raft_consensus.cc:2243] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.299840  8925 raft_consensus.cc:2272] T 8fd90bb718ac49bf84e09fe79d462240 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.303213 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:49.304414  8925 tablet_replica.cc:333] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.305511  8925 raft_consensus.cc:2243] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.306331  8925 raft_consensus.cc:2272] T d22230bc6afb42ceac52420e42db0292 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.309477  8925 tablet_replica.cc:333] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.310437  8925 raft_consensus.cc:2243] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:49.328310 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:49.337878  8925 raft_consensus.cc:2272] T 75ddb7f5e3204a51b3dafe4903888dcc P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.342248 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08003000a0 after lost signal to thread 8928
W20260709 13:20:49.344010 10152 debug-util.cc:398] Leaking SignalData structure 0x7b080037a480 after lost signal to thread 9988
W20260709 13:20:49.345602 10152 debug-util.cc:398] Leaking SignalData structure 0x7b080009cf20 after lost signal to thread 10155
W20260709 13:20:49.346994 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08000620c0 after lost signal to thread 10229
I20260709 13:20:49.347432  8925 tablet_replica.cc:333] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
W20260709 13:20:49.348464 10152 debug-util.cc:398] Leaking SignalData structure 0x7b08001028c0 after lost signal to thread 10924
I20260709 13:20:49.348397  8925 raft_consensus.cc:2243] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.350538  8925 raft_consensus.cc:2272] T 2169fe5ebfe5416c94b7d9095d9af6da P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.360217  8925 tablet_replica.cc:333] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.361521  8925 raft_consensus.cc:2243] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.362792  8925 raft_consensus.cc:2272] T a353f267acea40eaaf03befcb32892f6 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.366521  8925 tablet_replica.cc:333] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.367604  8925 raft_consensus.cc:2243] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.369455  8925 raft_consensus.cc:2272] T d07e888fde29492b84e575218308b6ff P e9a849ac7eda426abfca97cf8b15da0f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.373994  8925 tablet_replica.cc:333] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.375275  8925 raft_consensus.cc:2243] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.379026  8925 raft_consensus.cc:2272] T 2c485d13f66d4ce8bdca72b60d242842 P e9a849ac7eda426abfca97cf8b15da0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.380753 18603 raft_consensus.cc:493] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:49.381467 18603 raft_consensus.cc:515] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:49.388561  8925 tablet_replica.cc:333] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
W20260709 13:20:49.390900 10096 leader_election.cc:336] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:49.392005 18603 leader_election.cc:290] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:49.394179 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "764b8122cc0a4ccc97816d729adafabe" candidate_uuid: "e9a849ac7eda426abfca97cf8b15da0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:49.396854 10095 leader_election.cc:304] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e9a849ac7eda426abfca97cf8b15da0f; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:49.392686  8925 raft_consensus.cc:2243] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:49.402408 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:49.402014  8925 raft_consensus.cc:2272] T a122c89ca68646b8bf88dbb1d5dcbfa8 P e9a849ac7eda426abfca97cf8b15da0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.404495 18603 raft_consensus.cc:2749] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:49.406489  8925 tablet_replica.cc:333] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.407749  8925 raft_consensus.cc:2243] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.408977  8925 raft_consensus.cc:2272] T dfacd39092d24ba5a33d0cd9451b91a1 P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.413261  8925 tablet_replica.cc:333] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.414336  8925 raft_consensus.cc:2243] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.415632  8925 raft_consensus.cc:2272] T 3173f90d855c4371aeafe17b5f760205 P e9a849ac7eda426abfca97cf8b15da0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.419153  8925 tablet_replica.cc:333] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.420066  8925 raft_consensus.cc:2243] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:49.423164  8925 raft_consensus.cc:2272] T 2188d855354542c984f249b94175c660 P e9a849ac7eda426abfca97cf8b15da0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.427155  8925 tablet_replica.cc:333] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.428175  8925 raft_consensus.cc:2243] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.429044  8925 raft_consensus.cc:2272] T 371654d6e0a94cba825fa080a9998940 P e9a849ac7eda426abfca97cf8b15da0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:49.437511  8925 tablet_replica.cc:333] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f: stopping tablet replica
I20260709 13:20:49.438422  8925 raft_consensus.cc:2243] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:49.441991  8925 raft_consensus.cc:2272] T 764b8122cc0a4ccc97816d729adafabe P e9a849ac7eda426abfca97cf8b15da0f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:49.648926 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.671254 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:49.692382 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:49.778352 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.809593 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:49.810207 10170 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:50.125167 18595 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.125833 18595 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.128607 18595 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.146718 18595 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.151145 18595 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.145437 10863 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.167542 18595 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.183594 10863 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.214087 18620 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.214736 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.215394 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.214839 18620 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.217674 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.219089 18620 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.221024 18619 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:50.221980 18620 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.222740 18620 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.224859 10863 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.261441 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:50.225030 18620 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.234424 18619 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.228585 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.263301 18619 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.264114 18620 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:50.250618 10170 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:50.266314 18619 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.255000 18595 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.267100 18595 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.269932 18595 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.271747 18595 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:50.271842 10863 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.273041 10863 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.273569 18595 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.276386 18595 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.277011 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.277562 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.278388 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.279886 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.280395 18620 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:50.281689 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.282383 18615 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:50.283560 10863 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.286639 18599 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.287381 18599 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.290509 18599 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.285413 18615 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:50.292117 10863 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.293198 18615 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.293332 18595 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.294018 18595 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.296381 18595 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.304387 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:50.305744 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:50.306787 10863 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.310413 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.310678 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.311260 10205 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 20.
I20260709 13:20:50.311566 10201 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 16.
I20260709 13:20:50.315310 10864 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.317090 18599 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.317728 18599 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.320086 18599 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.324934 18600 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.325636 18600 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.329514 18600 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.331135 18615 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.331753 18615 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.344197 18600 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.344971 18600 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.360318 18599 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.361002 18599 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.364588 18600 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.368140 18620 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:50.368306 18595 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.369071 18595 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.369639 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 54441.3us
I20260709 13:20:50.371654 18595 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.377195 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.370575 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.380179 18595 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.380977 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.385237 10864 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.380118 18615 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.383716 18595 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:20:50.388896 18599 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.408164 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.409456 18620 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:50.409901 18620 raft_consensus.cc:493] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.410367 18620 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:50.413587 18600 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.414284 18600 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.406942 18595 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:50.415830 18595 raft_consensus.cc:493] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.416389 18595 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:50.416980 18600 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:50.420636 10170 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.433557 18595 raft_consensus.cc:515] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.435745 10168 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.436558 10168 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.437073 10170 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.445756 18626 raft_consensus.cc:2749] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:50.448173 10168 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.448745 10168 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.449743 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.450594 10205 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 23.
I20260709 13:20:50.442286 18620 raft_consensus.cc:515] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.458763 18626 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:50.461091 10863 leader_election.cc:336] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.466840 18620 leader_election.cc:290] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.471221 10170 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:50.472420 10170 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:50.476542 18595 leader_election.cc:290] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.477438 18626 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.478062 18626 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.481005 18626 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.487515 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:50.488466 10897 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
W20260709 13:20:50.492043 10863 leader_election.cc:336] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.493155 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:50.496775 10897 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:20:50.494769 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:50.497980 10168 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.498051 10893 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 26.
I20260709 13:20:50.498678 10168 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.499569 18626 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.499876 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:50.500633 10168 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.500728 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 21.
W20260709 13:20:50.501416 10168 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.504287 10168 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.504920 10168 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.506872 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:50.506863 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
W20260709 13:20:50.507741 10170 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:50.508221 10899 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 28.
I20260709 13:20:50.510900 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806f35e66c234d569cbafd8b3981518f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:50.511627 10205 raft_consensus.cc:3060] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:50.511741 10170 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.522310 10205 raft_consensus.cc:2468] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 17.
I20260709 13:20:50.524421 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169a099da7f404bb7804561c7aca3e6" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:50.535678 10201 raft_consensus.cc:3060] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:50.524816 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.540378 18619 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.541209 18619 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.525876 18626 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:50.527274 10168 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.529354 18623 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.543617 18626 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.543838 18623 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.545367 18600 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:50.529323 18620 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.546440 18620 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.550076 18626 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.551492 18623 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.553184 18620 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:50.558750 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.561317 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.562577 18623 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:50.563069 18623 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.563576 18623 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:50.551492 18619 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.545867 18600 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.571482 18600 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:50.563436 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.567425 10170 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:50.573136 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.573300 18615 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:20:50.573843 18615 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.574298 18615 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:20:50.569511 10168 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.576763 10168 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.580206 18626 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.576215 10169 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.581789 18626 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:50.582340 18626 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.582749 18626 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:50.586858 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.588310 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.530622 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.594228 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 64014.1us
I20260709 13:20:50.594975 18619 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:50.600834 18619 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.601388 18619 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:50.604898 10168 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.605610 10168 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.606132 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:50.598237 18622 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:20:50.608213 18622 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.608700 18622 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:20:50.597465 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.610432 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.620546 18599 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.621417 18599 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.624027 18599 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.626361 18599 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.626962 18599 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.629420 18599 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:50.639797 18600 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.611507 18625 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:50.660279 18600 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.645473 18626 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.648370 10201 raft_consensus.cc:2468] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 21.
I20260709 13:20:50.611300 10205 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:20:50.649209 18620 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:50.652125 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.653071 18623 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.656941 18615 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.670940 18623 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.671824 18623 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:50.655535 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 58105.7us
I20260709 13:20:50.672394 18623 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.661309 10168 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.612401 10864 leader_election.cc:304] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.676626 10168 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.677796 18625 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.678645 18625 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:50.681309 18625 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:50.682339 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 70061.9us
I20260709 13:20:50.683017 18620 raft_consensus.cc:2804] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:50.684855 18620 raft_consensus.cc:697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:50.686092 18620 consensus_queue.cc:237] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.688182 18599 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.662262 18622 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.695703 18622 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.695549 10864 leader_election.cc:304] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.697414 18627 raft_consensus.cc:2804] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:50.698088 18627 raft_consensus.cc:697] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
W20260709 13:20:50.699189 10170 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:50.700167 18627 consensus_queue.cc:237] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:50.707813 18623 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.709599 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.710966 18595 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:50.711512 18595 raft_consensus.cc:493] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.711964 18595 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:20:50.713930 10168 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.716707 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:50.718339 10168 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.718936 10168 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.724498 18615 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
I20260709 13:20:50.726622 18600 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:50.729947 10170 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:50.739493 18626 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:50.755779 10168 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.756426 10168 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.756942 10168 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.757419 10168 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.757849 10168 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.758658 10168 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.760449 18626 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:50.761468 10170 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:50.769703 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:50.770670 10899 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 23.
I20260709 13:20:50.759088 18619 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.782362 10168 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.783049 10168 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:50.787284 18626 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.794157  9950 catalog_manager.cc:5697] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 16 to 17, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 17 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:50.800257 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:50.801385 10899 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate d51ad94710c348889d417f7f00032948 in current term 24: Already voted for candidate db052bef7536453fa2d7f5a00e314a18 in this term.
I20260709 13:20:50.803637 10899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cebbf225b866442ab895577b0496de1b" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:50.804513 10899 raft_consensus.cc:3060] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:50.809697 10168 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.776899 18595 raft_consensus.cc:515] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:50.786350 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:50.817592 10893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce48110a5fb4ffc97e81117165e3ffd" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:50.823796 10893 raft_consensus.cc:3060] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:20:50.831406 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 45099.7us
I20260709 13:20:50.842056 18619 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:50.843218 10897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf6e6a0da3d4501acf0e5451cae4577" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:50.845618 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc97aed5eb284c0e86c74b20d18b4399" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
W20260709 13:20:50.846172 10168 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:50.846378 10894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18" is_pre_election: true
I20260709 13:20:50.847934 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:50.796666 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:20:50.854943 10863 leader_election.cc:336] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:50.950321 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 104199us
W20260709 13:20:50.950883 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 95966.6us
W20260709 13:20:50.869578 10170 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:50.822819 18595 leader_election.cc:290] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:50.953361 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 83874us
W20260709 13:20:50.883186  8925 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10152)
I20260709 13:20:50.954780 18615 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:50.955379 18615 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:50.955899 18615 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:50.857296 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecb570a023c42ada8943a1516f3ef4f" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:50.918577 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:50.928961 18631 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
W20260709 13:20:50.972337 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 175712us
I20260709 13:20:50.855532 10893 raft_consensus.cc:2468] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 25.
I20260709 13:20:50.973685 10896 raft_consensus.cc:3060] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:20:50.975404 10201 raft_consensus.cc:3060] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:20:50.977370 10170 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:20:50.984396 10899 raft_consensus.cc:2468] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:50.984004 10897 raft_consensus.cc:3060] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:20:50.976940 18631 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.011394 10894 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 22.
I20260709 13:20:50.976222 10205 raft_consensus.cc:2393] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate db052bef7536453fa2d7f5a00e314a18 in current term 24: Already voted for candidate d51ad94710c348889d417f7f00032948 in this term.
W20260709 13:20:50.974624 10169 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:44775, user_credentials={real_user=slave}} blocked reactor thread for 126792us
W20260709 13:20:50.981952 10168 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.091979 10168 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 110094us
W20260709 13:20:51.057745 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 75159.4us
I20260709 13:20:51.120394 10169 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: db052bef7536453fa2d7f5a00e314a18, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.121678 18626 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:51.077564 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:51.127182 10864 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:40505, user_credentials={real_user=slave}} blocked reactor thread for 49747.3us
I20260709 13:20:51.128607 18619 raft_consensus.cc:2749] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:20:51.041177 10170 consensus_peers.cc:597] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:51.130360 10170 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 89266.7us
I20260709 13:20:51.091084 18631 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:51.139475 18615 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.141345 10896 raft_consensus.cc:2468] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 27.
I20260709 13:20:51.142201 18615 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), db052bef7536453fa2d7f5a00e314a18 (127.8.183.68:44775)
W20260709 13:20:51.073818 10863 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.143332 10863 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:41681, user_credentials={real_user=slave}} blocked reactor thread for 69617.6us
I20260709 13:20:51.150599 10169 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.151722 10205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2188d855354542c984f249b94175c660" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:51.152117 10169 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.152590 10205 raft_consensus.cc:2468] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 22.
I20260709 13:20:51.155463 18625 raft_consensus.cc:2804] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:51.155640 18626 raft_consensus.cc:2804] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:51.156214 18625 raft_consensus.cc:697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:51.157752 18625 consensus_queue.cc:237] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.156380 18626 raft_consensus.cc:697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:51.162761 10865 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 61668.7us
I20260709 13:20:51.161022 18626 consensus_queue.cc:237] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:51.177428 10865 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:51.179889 10897 raft_consensus.cc:2468] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 29.
I20260709 13:20:51.181077 10201 raft_consensus.cc:2468] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 25.
I20260709 13:20:51.188351  9950 catalog_manager.cc:5697] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 20 to 21, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 21 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.190421 10169 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.192646 10169 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.193387 18615 raft_consensus.cc:2804] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:20:51.194433 18599 raft_consensus.cc:2804] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:20:51.194836 10864 leader_election.cc:304] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.195132 18599 raft_consensus.cc:697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 29 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:51.195760 18631 raft_consensus.cc:2804] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:51.196262 18631 raft_consensus.cc:697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:51.196352 18599 consensus_queue.cc:237] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.197310 18631 consensus_queue.cc:237] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.198670 18615 raft_consensus.cc:697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 27 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
I20260709 13:20:51.200793 18615 consensus_queue.cc:237] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:51.204645 10896 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f63a37309c453f9fd7bc3f75659536" candidate_uuid: "d51ad94710c348889d417f7f00032948" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "db052bef7536453fa2d7f5a00e314a18"
I20260709 13:20:51.205498 10896 raft_consensus.cc:3060] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:51.223721 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:20:51.229264 10168 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.231611 10168 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.235810 10863 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:51.240922 10170 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:20:51.240531 10863 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:51.247861 10201 raft_consensus.cc:1275] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:20:51.253705 10168 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.254945 10168 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.257251 10168 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.258118 10168 consensus_peers.cc:597] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.258657 10168 consensus_peers.cc:597] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.259224 10168 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:51.260952 10201 raft_consensus.cc:1275] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:51.251296 18620 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:51.270977 18620 consensus_queue.cc:1048] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:51.276980 10168 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.279687 10168 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.281744 10896 raft_consensus.cc:2468] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d51ad94710c348889d417f7f00032948 in term 24.
I20260709 13:20:51.286916 10169 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
W20260709 13:20:51.287824 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:20:51.274660 18620 consensus_queue.cc:1048] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:20:51.290386 18600 raft_consensus.cc:2804] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:51.291188 18600 raft_consensus.cc:697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Becoming Leader. State: Replica: d51ad94710c348889d417f7f00032948, State: Running, Role: LEADER
W20260709 13:20:51.295156 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:51.296069 10863 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.298525 10863 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.305038  9950 catalog_manager.cc:5697] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 24 to 25, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 25 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:51.325217 10168 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.325945 10168 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.326498 10168 consensus_peers.cc:597] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.344594 10863 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.345551 10863 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.345031 18600 consensus_queue.cc:237] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:51.353214 10168 consensus_peers.cc:597] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.368021 10168 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.368736 10168 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.369382 10168 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.369985 10168 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.398131  9950 catalog_manager.cc:5697] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 24 to 25, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 25 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:51.408455  9950 catalog_manager.cc:5697] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 21 to 22, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 22 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:51.429566 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:20:51.440335 10168 consensus_peers.cc:597] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.479279 10168 consensus_peers.cc:597] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.480234 10168 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.484050 10863 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.492833  9940 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.183.65:43347, user_credentials={real_user=slave}} blocked reactor thread for 49063.7us
W20260709 13:20:51.504222 10170 consensus_peers.cc:597] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.511987 10170 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.512749 10170 consensus_peers.cc:597] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:20:51.513242 10170 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:51.519326 10863 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.520586 10168 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.508754  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:51.567546 10168 consensus_peers.cc:597] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.594815 10170 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.607683 10168 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.589156  9947 catalog_manager.cc:5697] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 28 to 29, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 29 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:51.609328  9947 catalog_manager.cc:5697] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 23 to 24, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 24 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:51.611454  9947 catalog_manager.cc:5697] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 reported cstate change: term changed from 26 to 27, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to d51ad94710c348889d417f7f00032948 (127.8.183.67). New cstate: current_term: 27 leader_uuid: "d51ad94710c348889d417f7f00032948" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:51.615744 10168 consensus_peers.cc:597] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:51.616449 10168 consensus_peers.cc:597] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.629257 10896 raft_consensus.cc:1275] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:51.631551 18645 consensus_queue.cc:1048] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:20:51.639237 10168 consensus_peers.cc:597] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.648854 10168 consensus_peers.cc:597] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:51.653833 10894 raft_consensus.cc:1275] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 13:20:51.655324 10168 consensus_peers.cc:597] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.656019 18626 consensus_queue.cc:1048] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:20:51.710585 10170 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:20:51.714460 10170 consensus_peers.cc:597] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:51.749158 10863 consensus_peers.cc:597] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.755456 10170 consensus_peers.cc:597] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:51.759852 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:20:51.763509 10168 consensus_peers.cc:597] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:51.770651 10894 raft_consensus.cc:1275] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:20:51.773255 18626 consensus_queue.cc:1048] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:20:51.805970 10896 raft_consensus.cc:1275] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 13:20:51.812156 18623 consensus_queue.cc:1048] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:20:51.829743 10201 raft_consensus.cc:1275] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Refusing update from remote peer db052bef7536453fa2d7f5a00e314a18: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:20:51.840122 18619 consensus_queue.cc:1048] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:20:51.845763 10168 consensus_peers.cc:597] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.850309 10170 consensus_peers.cc:597] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.874404 10168 consensus_peers.cc:597] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:51.941644 10170 consensus_peers.cc:597] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.942269 10170 consensus_peers.cc:597] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:51.958416  8925 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10152)
W20260709 13:20:51.985885 10170 consensus_peers.cc:597] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:51.986554 10170 consensus_peers.cc:597] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:52.009923 10896 raft_consensus.cc:1275] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Refusing update from remote peer d51ad94710c348889d417f7f00032948: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 13:20:52.012382 18615 consensus_queue.cc:1048] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:20:52.136134 10170 consensus_peers.cc:597] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:20:52.169835 18645 raft_consensus.cc:493] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.170684 18645 raft_consensus.cc:515] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.175284 18645 leader_election.cc:290] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.184170 10168 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:52.186022 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:20:52.267401 18645 raft_consensus.cc:493] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.268283 18645 raft_consensus.cc:515] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.273485 18645 leader_election.cc:290] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.280186 10168 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.297360 18645 raft_consensus.cc:493] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.298130 18645 raft_consensus.cc:515] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
W20260709 13:20:52.308408 10168 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.310737 18645 leader_election.cc:290] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.342891 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:20:52.345322 18645 raft_consensus.cc:493] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.346030 18645 raft_consensus.cc:515] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.348987 18645 leader_election.cc:290] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.381147 10168 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:52.391426 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:20:52.397342 18623 raft_consensus.cc:493] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.398051 18623 raft_consensus.cc:515] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.400882 18623 leader_election.cc:290] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.426079 10168 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.432683 18619 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.433449 18619 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:52.461828 10863 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:52.464141 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:52.464687 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:52.466250 18631 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:52.471308 18619 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:52.567608 18645 raft_consensus.cc:493] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.568253 18645 raft_consensus.cc:515] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.571192 18645 leader_election.cc:290] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.579392 10168 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.583220 18658 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.584124 18658 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:52.597577 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:52.599172 18658 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.606318 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.606895 10863 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:52.608376 18658 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:52.700472 18645 raft_consensus.cc:493] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:52.701228 18645 raft_consensus.cc:515] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } }
I20260709 13:20:52.706276 18645 leader_election.cc:290] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:52.768222 10168 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.893313  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
I20260709 13:20:52.945361 18658 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.946177 18658 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:52.951843 18658 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:52.955343 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:52.957432 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948" is_pre_election: true
I20260709 13:20:52.958328 10201 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 24.
I20260709 13:20:52.960176 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:52.961724 18658 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:52.962181 18658 raft_consensus.cc:493] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.962558 18658 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:52.978832 18658 raft_consensus.cc:515] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:52.999194 10863 leader_election.cc:336] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:53.005877 10201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6278932947044bdbb7b0b1e14eea574b" candidate_uuid: "db052bef7536453fa2d7f5a00e314a18" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "d51ad94710c348889d417f7f00032948"
I20260709 13:20:53.006902 10201 raft_consensus.cc:3060] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:53.023324 18658 leader_election.cc:290] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Requested vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:53.037840 10201 raft_consensus.cc:2468] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db052bef7536453fa2d7f5a00e314a18 in term 25.
I20260709 13:20:53.049793 10864 leader_election.cc:304] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d51ad94710c348889d417f7f00032948, db052bef7536453fa2d7f5a00e314a18; no voters: e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.053110 18658 raft_consensus.cc:2804] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:53.054636 18658 raft_consensus.cc:697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Becoming Leader. State: Replica: db052bef7536453fa2d7f5a00e314a18, State: Running, Role: LEADER
I20260709 13:20:53.056756 18658 consensus_queue.cc:237] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:53.091945  9947 catalog_manager.cc:5697] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 reported cstate change: term changed from 23 to 25, leader changed from e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66) to db052bef7536453fa2d7f5a00e314a18 (127.8.183.68). New cstate: current_term: 25 leader_uuid: "db052bef7536453fa2d7f5a00e314a18" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:53.157140  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
I20260709 13:20:53.161095 18658 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:53.162882 18658 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:53.171644 18658 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:53.174525 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:20:53.180320 10864 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:20:53.188242 10863 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:53.189103 10863 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.192232 18619 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:53.201668 10865 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:53.204088 10170 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:53.219429 10170 leader_election.cc:336] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.221958 10170 leader_election.cc:304] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.223043 18645 raft_consensus.cc:2749] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:53.224545 10170 leader_election.cc:336] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.230149 10170 leader_election.cc:304] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.231287 18625 raft_consensus.cc:2749] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:53.233009 10170 leader_election.cc:336] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.233588 10170 leader_election.cc:304] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.234783 18625 raft_consensus.cc:2749] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:53.235365 10170 leader_election.cc:336] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.235910 10170 leader_election.cc:304] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.237211 18645 raft_consensus.cc:2749] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:53.249558 10864 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:53.254756 10168 consensus_peers.cc:597] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:53.254900 10170 leader_election.cc:336] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.256083 10170 leader_election.cc:304] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.257128 18645 raft_consensus.cc:2749] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:53.258425 10170 consensus_peers.cc:597] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:53.259181 10170 consensus_peers.cc:597] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:53.266212 10170 leader_election.cc:336] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.266821 10170 leader_election.cc:304] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.267819 18645 raft_consensus.cc:2749] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:53.271589 10170 leader_election.cc:336] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:53.273268 10170 leader_election.cc:304] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d51ad94710c348889d417f7f00032948; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:53.279750 18645 raft_consensus.cc:2749] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:20:53.284080 10170 consensus_peers.cc:597] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:20:53.325470 10170 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:53.327296 10168 consensus_peers.cc:597] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.339709  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:53.342980  8925 tablet_replica.cc:333] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.345000  8925 raft_consensus.cc:2243] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.358417  8925 raft_consensus.cc:2272] T 87da05694091470284f66a8181207e2f P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.359287 10168 consensus_peers.cc:597] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.363102  8925 tablet_replica.cc:333] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.364305  8925 raft_consensus.cc:2243] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.365845  8925 raft_consensus.cc:2272] T ece395e49d7e474182be2d25642b90de P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.369096  8925 tablet_replica.cc:333] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.369948  8925 raft_consensus.cc:2243] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.371054  8925 raft_consensus.cc:2272] T 7a82842926e645c08dde847bfe182c31 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.374622  8925 tablet_replica.cc:333] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.375686  8925 raft_consensus.cc:2243] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.386811  8925 raft_consensus.cc:2272] T 87d36b9784d64ff48d0d4f0224266506 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.390561  8925 tablet_replica.cc:333] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.391680  8925 raft_consensus.cc:2243] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.393401  8925 raft_consensus.cc:2272] T cebbf225b866442ab895577b0496de1b P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.399930  8925 tablet_replica.cc:333] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.402522  8925 raft_consensus.cc:2243] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.403699  8925 raft_consensus.cc:2272] T 66968e5dbd3040d290ce9716c146d46a P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.408248  8925 tablet_replica.cc:333] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.409143  8925 raft_consensus.cc:2243] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.410272  8925 raft_consensus.cc:2272] T f09ed53b9cff42a29c1d6c06e009ee94 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.413184  8925 tablet_replica.cc:333] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.414985  8925 raft_consensus.cc:2243] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.417716  8925 raft_consensus.cc:2272] T dc97aed5eb284c0e86c74b20d18b4399 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.423614  8925 tablet_replica.cc:333] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.424645  8925 raft_consensus.cc:2243] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.425576  8925 raft_consensus.cc:2272] T e14489a089594151b58748865ea36e81 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.429469  8925 tablet_replica.cc:333] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.430539  8925 raft_consensus.cc:2243] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.432029  8925 raft_consensus.cc:2272] T 5853a69cde724bc3a143613bf68a63fc P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.440095 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:20:53.443791  8925 tablet_replica.cc:333] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.445078  8925 raft_consensus.cc:2243] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.446446  8925 raft_consensus.cc:2272] T a169a099da7f404bb7804561c7aca3e6 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.447708 10864 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.449811  8925 tablet_replica.cc:333] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.450866  8925 raft_consensus.cc:2243] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.451620  8925 raft_consensus.cc:2272] T ed07b7c3185a4d5d9fbcba31d80ed8f6 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.454952  8925 tablet_replica.cc:333] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.455955  8925 raft_consensus.cc:2243] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.457454  8925 raft_consensus.cc:2272] T a0c93dea5060480eb0618f168078f4e6 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.457679 10168 consensus_peers.cc:597] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.460837  8925 tablet_replica.cc:333] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.461838  8925 raft_consensus.cc:2243] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.463001  8925 raft_consensus.cc:2272] T 6a7190358a9f475d82e5f9875677bccf P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.465735  8925 tablet_replica.cc:333] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.466487  8925 raft_consensus.cc:2243] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.467720  8925 raft_consensus.cc:2272] T 4ce48110a5fb4ffc97e81117165e3ffd P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.470387  8925 tablet_replica.cc:333] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.471427  8925 raft_consensus.cc:2243] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.472157  8925 raft_consensus.cc:2272] T 806f35e66c234d569cbafd8b3981518f P d51ad94710c348889d417f7f00032948 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.478009 10864 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:53.480010 10168 consensus_peers.cc:597] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.480461  8925 tablet_replica.cc:333] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.481751  8925 raft_consensus.cc:2243] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.482587  8925 raft_consensus.cc:2272] T 5391e406039d4af8bf2b17f1b2034ef5 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.490505  8925 tablet_replica.cc:333] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.492151  8925 raft_consensus.cc:2243] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:20:53.493105 10863 consensus_peers.cc:597] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.493810  8925 raft_consensus.cc:2272] T 82960478409442549781547254b9d8ea P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.497975  8925 tablet_replica.cc:333] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.499835  8925 raft_consensus.cc:2243] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.501302  8925 raft_consensus.cc:2272] T ee79852a0bc04642b58b690930f78267 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.504917  8925 tablet_replica.cc:333] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.505991  8925 raft_consensus.cc:2243] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.507403  8925 raft_consensus.cc:2272] T 161f2d2f2674494bbffbd5ee927cf5e6 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.511008  8925 tablet_replica.cc:333] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.512079  8925 raft_consensus.cc:2243] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.514555  8925 raft_consensus.cc:2272] T dfacd39092d24ba5a33d0cd9451b91a1 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.518054  8925 tablet_replica.cc:333] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.519047  8925 raft_consensus.cc:2243] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.520503  8925 raft_consensus.cc:2272] T 986d51d657bc4fac8f4199fbc38600c1 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.524993  8925 tablet_replica.cc:333] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948: stopping tablet replica
W20260709 13:20:53.524789 10864 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.526495  8925 raft_consensus.cc:2243] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.528173  8925 raft_consensus.cc:2272] T bfb08df714384523b94354944a523e54 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.532775  8925 tablet_replica.cc:333] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.533841  8925 raft_consensus.cc:2243] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.534535  8925 raft_consensus.cc:2272] T 6278932947044bdbb7b0b1e14eea574b P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.538115  8925 tablet_replica.cc:333] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.539180  8925 raft_consensus.cc:2243] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.540656  8925 raft_consensus.cc:2272] T dbf6e6a0da3d4501acf0e5451cae4577 P d51ad94710c348889d417f7f00032948 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.544891  8925 tablet_replica.cc:333] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948: stopping tablet replica
W20260709 13:20:53.544891 10864 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.546350  8925 raft_consensus.cc:2243] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.547848  8925 raft_consensus.cc:2272] T c502c18433fb46e2ac2a42d7d8746ce0 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.551278  8925 tablet_replica.cc:333] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.552405  8925 raft_consensus.cc:2243] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.553838  8925 raft_consensus.cc:2272] T 73fb38c54abe4cd4bb552eb544020f47 P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.557147  8925 tablet_replica.cc:333] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.558140  8925 raft_consensus.cc:2243] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.559619  8925 raft_consensus.cc:2272] T aca8a04bf8df4212bfdf5fd990503c0a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.562721  8925 tablet_replica.cc:333] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.563645  8925 raft_consensus.cc:2243] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.565085  8925 raft_consensus.cc:2272] T 79954d719dc94974b4864ff715e9ef07 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.568452  8925 tablet_replica.cc:333] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.569442  8925 raft_consensus.cc:2243] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.570813  8925 raft_consensus.cc:2272] T f3220b9037e44cae95e49af43de3523a P d51ad94710c348889d417f7f00032948 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.580423  8925 tablet_replica.cc:333] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948: stopping tablet replica
W20260709 13:20:53.584095 10865 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:20:53.581668  8925 raft_consensus.cc:2243] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.585569  8925 raft_consensus.cc:2272] T 2869566cb59547689f49f6e04511ed8a P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.588634 10168 consensus_peers.cc:597] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.590895  8925 tablet_replica.cc:333] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.592129  8925 raft_consensus.cc:2243] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.593241  8925 raft_consensus.cc:2272] T 231b54e25cca40f6a3b0625d68e97cf7 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.601385 10864 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.603188  8925 tablet_replica.cc:333] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.604585  8925 raft_consensus.cc:2243] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.606791  8925 raft_consensus.cc:2272] T c3664f114a52481c8f97e06e9b2d2068 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.611444  8925 tablet_replica.cc:333] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948: stopping tablet replica
W20260709 13:20:53.611716 10170 consensus_peers.cc:597] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:20:53.612466  8925 raft_consensus.cc:2243] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.613914  8925 raft_consensus.cc:2272] T 98f63a37309c453f9fd7bc3f75659536 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.617316  8925 tablet_replica.cc:333] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.618271  8925 raft_consensus.cc:2243] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.618988  8925 raft_consensus.cc:2272] T cf88b6d5c6504b3cb2cbd21cff68e57e P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.622753  8925 tablet_replica.cc:333] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.623679  8925 raft_consensus.cc:2243] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.624506  8925 raft_consensus.cc:2272] T 82406372d02c4dd59d4ef5c32436e148 P d51ad94710c348889d417f7f00032948 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.627851  8925 tablet_replica.cc:333] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.628795  8925 raft_consensus.cc:2243] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.630205  8925 raft_consensus.cc:2272] T 2c485d13f66d4ce8bdca72b60d242842 P d51ad94710c348889d417f7f00032948 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.633550  8925 tablet_replica.cc:333] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.634554  8925 raft_consensus.cc:2243] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.635360  8925 raft_consensus.cc:2272] T d07e888fde29492b84e575218308b6ff P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.638674  8925 tablet_replica.cc:333] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.639889  8925 raft_consensus.cc:2243] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.641273  8925 raft_consensus.cc:2272] T 4d6c92980e764b60a913b8c5533ed9ad P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.644761  8925 tablet_replica.cc:333] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.645713  8925 raft_consensus.cc:2243] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.646580  8925 raft_consensus.cc:2272] T 2219e1af873d4a00966a8c5975eb7987 P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.646725 10863 consensus_peers.cc:597] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:53.650326  8925 tablet_replica.cc:333] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.651377  8925 raft_consensus.cc:2243] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.652711  8925 raft_consensus.cc:2272] T 2c048cf8265e4264b35912a8caac9936 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.655982  8925 tablet_replica.cc:333] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.656919  8925 raft_consensus.cc:2243] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:53.657631  8925 raft_consensus.cc:2272] T 5ecb570a023c42ada8943a1516f3ef4f P d51ad94710c348889d417f7f00032948 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.660953  8925 tablet_replica.cc:333] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.661921  8925 raft_consensus.cc:2243] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.663381  8925 raft_consensus.cc:2272] T 1c7c855af77d40ab995a18b5609de8f5 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.668344  8925 tablet_replica.cc:333] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.669356  8925 raft_consensus.cc:2243] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.670497  8925 raft_consensus.cc:2272] T b09e10904af344988a1a11ae9e576799 P d51ad94710c348889d417f7f00032948 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.675922  8925 tablet_replica.cc:333] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.676811  8925 raft_consensus.cc:2243] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.678093  8925 raft_consensus.cc:2272] T 2169fe5ebfe5416c94b7d9095d9af6da P d51ad94710c348889d417f7f00032948 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.681253  8925 tablet_replica.cc:333] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.682111  8925 raft_consensus.cc:2243] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.683526  8925 raft_consensus.cc:2272] T a353f267acea40eaaf03befcb32892f6 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.686638  8925 tablet_replica.cc:333] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.687567  8925 raft_consensus.cc:2243] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.688870  8925 raft_consensus.cc:2272] T d22230bc6afb42ceac52420e42db0292 P d51ad94710c348889d417f7f00032948 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.689173 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:20:53.692332 10863 consensus_peers.cc:597] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.691897  8925 tablet_replica.cc:333] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.693852  8925 raft_consensus.cc:2243] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.695314  8925 raft_consensus.cc:2272] T 8fd90bb718ac49bf84e09fe79d462240 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.698521  8925 tablet_replica.cc:333] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948: stopping tablet replica
W20260709 13:20:53.699437 10864 consensus_peers.cc:597] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:53.699477  8925 raft_consensus.cc:2243] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.701830  8925 raft_consensus.cc:2272] T 371654d6e0a94cba825fa080a9998940 P d51ad94710c348889d417f7f00032948 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.705628  8925 tablet_replica.cc:333] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.706660  8925 raft_consensus.cc:2243] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.708405  8925 raft_consensus.cc:2272] T 764b8122cc0a4ccc97816d729adafabe P d51ad94710c348889d417f7f00032948 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.718906  8925 tablet_replica.cc:333] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.720803  8925 raft_consensus.cc:2243] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:20:53.723172  8925 raft_consensus.cc:2272] T 3173f90d855c4371aeafe17b5f760205 P d51ad94710c348889d417f7f00032948 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:53.727584  8925 tablet_replica.cc:333] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948: stopping tablet replica
I20260709 13:20:53.728516  8925 raft_consensus.cc:2243] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:53.729092 10863 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:20:53.729835  8925 raft_consensus.cc:2272] T 2188d855354542c984f249b94175c660 P d51ad94710c348889d417f7f00032948 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:53.745787 10864 consensus_peers.cc:597] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:53.749013 10865 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:20:53.754364 10863 consensus_peers.cc:597] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:20:53.807534 10863 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:53.812546 10863 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:53.884980 10863 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:20:54.182240 10865 consensus_peers.cc:597] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:20:54.282101 10865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 726 similar messages]
W20260709 13:20:54.366585 10863 consensus_peers.cc:597] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 -> Peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Couldn't send request to peer e9a849ac7eda426abfca97cf8b15da0f. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:54.404747 18665 raft_consensus.cc:493] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.405565 18665 raft_consensus.cc:515] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:54.414623 10864 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.415402 18665 leader_election.cc:290] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.435963 10865 leader_election.cc:336] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:54.436888 10865 leader_election.cc:304] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:54.437883 18665 raft_consensus.cc:2749] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:54.597114 18665 raft_consensus.cc:493] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.597910 18665 raft_consensus.cc:515] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:54.606284 10863 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:54.613974 10864 leader_election.cc:336] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.614713 10864 leader_election.cc:304] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:54.615774 18658 raft_consensus.cc:2749] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:54.617228 18665 leader_election.cc:290] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
I20260709 13:20:54.627516 18665 raft_consensus.cc:493] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.628427 18665 raft_consensus.cc:515] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:54.645603 10863 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:54.665067 18665 leader_election.cc:290] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.689296 10864 leader_election.cc:336] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.693656 10864 leader_election.cc:304] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:54.704829 18658 raft_consensus.cc:2749] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:54.711519 10865 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:20:54.738632 10865 consensus_peers.cc:597] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:20:54.744086 18665 raft_consensus.cc:493] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.744935 18665 raft_consensus.cc:515] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:54.748077 18665 leader_election.cc:290] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.764230 10865 leader_election.cc:336] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:54.784799 18665 raft_consensus.cc:493] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.786019 18665 raft_consensus.cc:515] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:54.789098 18665 leader_election.cc:290] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.784299 10864 leader_election.cc:336] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.797019 10864 leader_election.cc:304] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:54.800105 18665 raft_consensus.cc:2749] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:54.805052 18658 raft_consensus.cc:493] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.805819 18658 raft_consensus.cc:515] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:54.808863 18658 leader_election.cc:290] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.798477 10863 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:54.820892 10865 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:54.849399 10864 leader_election.cc:336] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.852026 10864 leader_election.cc:304] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:54.856824 18665 raft_consensus.cc:2749] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:20:54.858093 10864 leader_election.cc:336] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.858697 10864 leader_election.cc:304] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, d51ad94710c348889d417f7f00032948
I20260709 13:20:54.862069 18665 raft_consensus.cc:2749] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:20:54.879408 18665 raft_consensus.cc:493] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.880110 18665 raft_consensus.cc:515] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:54.885963 18665 leader_election.cc:290] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.892318 10863 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:54.898677 18658 raft_consensus.cc:493] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:54.899533 18658 raft_consensus.cc:515] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } attrs { replace: true } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:54.903923 18658 leader_election.cc:290] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:54.913018 10865 consensus_peers.cc:597] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20260709 13:20:54.913906 10865 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:54.921505 10863 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:54.954418 10863 leader_election.cc:304] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:54.961632 18658 raft_consensus.cc:2749] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:54.969381 10864 leader_election.cc:336] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:54.970227 10864 leader_election.cc:304] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:54.972237 18665 raft_consensus.cc:2749] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:20:54.977471 10864 leader_election.cc:336] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:55.025247 18658 raft_consensus.cc:493] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:55.027325 18658 raft_consensus.cc:515] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:55.048897 10863 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:55.050951 10865 leader_election.cc:336] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:55.051653 10865 leader_election.cc:304] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:55.052729 18658 leader_election.cc:290] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
I20260709 13:20:55.054225 18665 raft_consensus.cc:2749] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:55.082433 18658 raft_consensus.cc:493] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader d51ad94710c348889d417f7f00032948)
I20260709 13:20:55.084120 18658 raft_consensus.cc:515] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
W20260709 13:20:55.094029 10863 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:55.102561 10864 leader_election.cc:336] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:55.103232 10864 leader_election.cc:304] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: d51ad94710c348889d417f7f00032948, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:55.104336 18665 raft_consensus.cc:2749] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:55.105636 18658 leader_election.cc:290] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:55.342293 10865 consensus_peers.cc:597] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:20:55.395262  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
W20260709 13:20:55.738749 10864 consensus_peers.cc:597] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:55.741154 18665 raft_consensus.cc:493] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:55.742355 18665 raft_consensus.cc:515] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "d51ad94710c348889d417f7f00032948" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40505 } attrs { replace: true } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:55.746722 18665 leader_election.cc:290] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681), d51ad94710c348889d417f7f00032948 (127.8.183.67:40505)
W20260709 13:20:55.755719 10865 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:20:55.760552 10863 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
I20260709 13:20:55.762571 10863 leader_election.cc:304] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:55.765405 18665 raft_consensus.cc:2749] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:20:55.766993 10864 leader_election.cc:336] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111)
I20260709 13:20:55.768314  8925 tablet_server.cc:179] TabletServer@127.8.183.68:0 shutting down...
W20260709 13:20:55.767351 10865 consensus_peers.cc:597] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:20:55.792943 10864 consensus_peers.cc:597] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:55.801930 18658 raft_consensus.cc:493] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e9a849ac7eda426abfca97cf8b15da0f)
I20260709 13:20:55.802960 18658 raft_consensus.cc:515] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "01bfaf1ad56a4deea886bbc64dec466c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43347 } } peers { permanent_uuid: "e9a849ac7eda426abfca97cf8b15da0f" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 41681 } } peers { permanent_uuid: "db052bef7536453fa2d7f5a00e314a18" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 44775 } attrs { promote: false } }
I20260709 13:20:55.806854 18658 leader_election.cc:290] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347), e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681)
W20260709 13:20:55.810928 10863 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer e9a849ac7eda426abfca97cf8b15da0f (127.8.183.66:41681): Network error: Client connection negotiation failed: client connection to 127.8.183.66:41681: connect: Connection refused (error 111)
W20260709 13:20:55.816181 10865 leader_election.cc:336] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:20:55.816875 10865 leader_election.cc:304] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db052bef7536453fa2d7f5a00e314a18; no voters: 01bfaf1ad56a4deea886bbc64dec466c, e9a849ac7eda426abfca97cf8b15da0f
I20260709 13:20:55.818887 18658 raft_consensus.cc:2749] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:55.851351  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:20:55.852682  8925 tablet_replica.cc:333] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.854418  8925 raft_consensus.cc:2243] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.857092  8925 raft_consensus.cc:2272] T a169a099da7f404bb7804561c7aca3e6 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.862372  8925 tablet_replica.cc:333] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.863727  8925 raft_consensus.cc:2243] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.864939  8925 raft_consensus.cc:2272] T 2188d855354542c984f249b94175c660 P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.868742  8925 tablet_replica.cc:333] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.869635  8925 raft_consensus.cc:2243] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.870507  8925 raft_consensus.cc:2272] T 7a82842926e645c08dde847bfe182c31 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.873541  8925 tablet_replica.cc:333] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.874375  8925 raft_consensus.cc:2243] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.875550  8925 raft_consensus.cc:2272] T c3664f114a52481c8f97e06e9b2d2068 P db052bef7536453fa2d7f5a00e314a18 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.878966  8925 tablet_replica.cc:333] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.879808  8925 raft_consensus.cc:2243] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.880695  8925 raft_consensus.cc:2272] T cebbf225b866442ab895577b0496de1b P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.885043  8925 tablet_replica.cc:333] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.886061  8925 raft_consensus.cc:2243] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.888309  8925 raft_consensus.cc:2272] T 66968e5dbd3040d290ce9716c146d46a P db052bef7536453fa2d7f5a00e314a18 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.892549  8925 tablet_replica.cc:333] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
W20260709 13:20:55.895027 10865 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): Couldn't send request to peer 01bfaf1ad56a4deea886bbc64dec466c. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20260709 13:20:55.895685  8925 raft_consensus.cc:2243] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.897104  8925 raft_consensus.cc:2272] T c5af05c8079540ec84b0cc3dc1a9e051 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.900789  8925 tablet_replica.cc:333] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.901535  8925 raft_consensus.cc:2243] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.902948  8925 raft_consensus.cc:2272] T 5ecb570a023c42ada8943a1516f3ef4f P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.906095  8925 tablet_replica.cc:333] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.906848  8925 raft_consensus.cc:2243] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.908188  8925 pending_rounds.cc:70] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Trying to abort 1 pending ops.
I20260709 13:20:55.908514  8925 pending_rounds.cc:77] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18: Aborting op as it isn't in flight: id { term: 25 index: 18 } timestamp: 7305638924559626240 op_type: NO_OP noop_request { }
I20260709 13:20:55.909058  8925 raft_consensus.cc:2889] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:55.909447  8925 raft_consensus.cc:2272] T 6278932947044bdbb7b0b1e14eea574b P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.912639  8925 tablet_replica.cc:333] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.913622  8925 raft_consensus.cc:2243] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.915199  8925 raft_consensus.cc:2272] T 5391e406039d4af8bf2b17f1b2034ef5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.918179  8925 tablet_replica.cc:333] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.918977  8925 raft_consensus.cc:2243] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.919852  8925 raft_consensus.cc:2272] T ece395e49d7e474182be2d25642b90de P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.922492  8925 tablet_replica.cc:333] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.923247  8925 raft_consensus.cc:2243] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.924646  8925 raft_consensus.cc:2272] T cf88b6d5c6504b3cb2cbd21cff68e57e P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.927593  8925 tablet_replica.cc:333] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.928279  8925 raft_consensus.cc:2243] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.929029  8925 raft_consensus.cc:2272] T dbf6e6a0da3d4501acf0e5451cae4577 P db052bef7536453fa2d7f5a00e314a18 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.931883  8925 tablet_replica.cc:333] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.932586  8925 raft_consensus.cc:2243] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.934111  8925 raft_consensus.cc:2272] T 9a895dc8dee14dfba216f443121474f4 P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.937794  8925 tablet_replica.cc:333] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.938613  8925 raft_consensus.cc:2243] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.939766  8925 raft_consensus.cc:2272] T 6a7190358a9f475d82e5f9875677bccf P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.943982  8925 tablet_replica.cc:333] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.944962  8925 raft_consensus.cc:2243] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.946063  8925 raft_consensus.cc:2272] T 4ce48110a5fb4ffc97e81117165e3ffd P db052bef7536453fa2d7f5a00e314a18 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:55.948478 10864 consensus_peers.cc:597] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 -> Peer d51ad94710c348889d417f7f00032948 (127.8.183.67:40505): Couldn't send request to peer d51ad94710c348889d417f7f00032948. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:40505: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:20:55.950045  8925 tablet_replica.cc:333] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.950838  8925 raft_consensus.cc:2243] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.951792  8925 raft_consensus.cc:2272] T 5e46fa8cb2404116bf566b4ef9bbb2cf P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.954751  8925 tablet_replica.cc:333] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.955446  8925 raft_consensus.cc:2243] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.956113  8925 raft_consensus.cc:2272] T dc97aed5eb284c0e86c74b20d18b4399 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.958616  8925 tablet_replica.cc:333] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.959364  8925 raft_consensus.cc:2243] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.960737  8925 raft_consensus.cc:2272] T 95b81db48ec24878966db719b2790127 P db052bef7536453fa2d7f5a00e314a18 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.964166  8925 tablet_replica.cc:333] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.965215  8925 raft_consensus.cc:2243] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.967073  8925 raft_consensus.cc:2272] T 75ddb7f5e3204a51b3dafe4903888dcc P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.970396  8925 tablet_replica.cc:333] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.971200  8925 raft_consensus.cc:2243] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.972599  8925 raft_consensus.cc:2272] T 82406372d02c4dd59d4ef5c32436e148 P db052bef7536453fa2d7f5a00e314a18 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.975728  8925 tablet_replica.cc:333] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.976390  8925 raft_consensus.cc:2243] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.978466  8925 raft_consensus.cc:2272] T 806f35e66c234d569cbafd8b3981518f P db052bef7536453fa2d7f5a00e314a18 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.982350  8925 tablet_replica.cc:333] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.982957  8925 raft_consensus.cc:2243] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.984545  8925 raft_consensus.cc:2272] T a122c89ca68646b8bf88dbb1d5dcbfa8 P db052bef7536453fa2d7f5a00e314a18 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.987702  8925 tablet_replica.cc:333] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.988338  8925 raft_consensus.cc:2243] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:20:55.989672  8925 raft_consensus.cc:2272] T d97ed0ec5f214d95abb9e01d368a04b5 P db052bef7536453fa2d7f5a00e314a18 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:55.992399  8925 tablet_replica.cc:333] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18: stopping tablet replica
I20260709 13:20:55.993005  8925 raft_consensus.cc:2243] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:55.993676  8925 raft_consensus.cc:2272] T 98f63a37309c453f9fd7bc3f75659536 P db052bef7536453fa2d7f5a00e314a18 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:56.366418  8925 tablet_server.cc:196] TabletServer@127.8.183.68:0 shutdown complete.
I20260709 13:20:56.437978  8925 master.cc:562] Master@127.8.183.126:40341 shutting down...
W20260709 13:20:56.486624 10002 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:20:56.487182 10002 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:20:56.487483 10002 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:20:56.501080  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800178600 after lost signal to thread 8928
W20260709 13:20:56.502413  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800377940 after lost signal to thread 9988
W20260709 13:20:56.503969  9985 debug-util.cc:398] Leaking SignalData structure 0x7b0800047940 after lost signal to thread 10924
W20260709 13:20:56.989545  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a169a099da7f404bb7804561c7aca3e6: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:21:20.086606  9940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111) [suppressed 112 similar messages]
W20260709 13:21:20.092171  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 806f35e66c234d569cbafd8b3981518f: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
W20260709 13:21:24.334345  9940 catalog_manager.cc:4755] TS 01bfaf1ad56a4deea886bbc64dec466c (127.8.183.65:43347): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cebbf225b866442ab895577b0496de1b: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43347: connect: Connection refused (error 111)
I20260709 13:21:29.671489  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:29.672850  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:29.673353  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P db0c98fae3aa4b45a39c8914c5e641cc: stopping tablet replica
I20260709 13:21:29.927922  8925 master.cc:584] Master@127.8.183.126:40341 shutdown complete.
I20260709 13:21:29.991011  8925 test_util.cc:182] -----------------------------------------------
I20260709 13:21:29.991364  8925 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602929618813-8925-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (335445 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:21:29.995831  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:45441
I20260709 13:21:29.997184  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:30.005270 18679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.005329 18678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.005780 18681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.008414  8925 server_base.cc:1061] running on GCE node
I20260709 13:21:30.013640  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:30.013994  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:30.014199  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603290014172 us; error 0 us; skew 500 ppm
I20260709 13:21:30.014848  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:30.018039  8925 webserver.cc:533] Webserver started at http://127.8.183.126:43775/ using document root <none> and password file <none>
I20260709 13:21:30.018733  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:30.019008  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:30.019416  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:30.020885  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "e2ca2de18d2e4c83a44246d79f233970"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.026921  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.006s
I20260709 13:21:30.031348 18686 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.032419  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:21:30.032866  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "e2ca2de18d2e4c83a44246d79f233970"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.033315  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:30.056154  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:30.057562  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:30.105890  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:45441
I20260709 13:21:30.106003 18737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:45441 every 8 connection(s)
I20260709 13:21:30.111420 18738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:30.125771 18738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970: Bootstrap starting.
I20260709 13:21:30.131870 18738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:30.138195 18738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970: No bootstrap required, opened a new log
I20260709 13:21:30.141098 18738 raft_consensus.cc:359] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER }
I20260709 13:21:30.141678 18738 raft_consensus.cc:385] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:30.142030 18738 raft_consensus.cc:740] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2ca2de18d2e4c83a44246d79f233970, State: Initialized, Role: FOLLOWER
I20260709 13:21:30.142906 18738 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER }
I20260709 13:21:30.143564 18738 raft_consensus.cc:399] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:21:30.143848 18738 raft_consensus.cc:493] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:21:30.144145 18738 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:30.151921 18738 raft_consensus.cc:515] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER }
I20260709 13:21:30.152660 18738 leader_election.cc:304] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2ca2de18d2e4c83a44246d79f233970; no voters: 
I20260709 13:21:30.154141 18738 leader_election.cc:290] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:21:30.154565 18741 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:30.156339 18741 raft_consensus.cc:697] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 1 LEADER]: Becoming Leader. State: Replica: e2ca2de18d2e4c83a44246d79f233970, State: Running, Role: LEADER
I20260709 13:21:30.157311 18741 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER }
I20260709 13:21:30.157796 18738 sys_catalog.cc:565] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:21:30.167042 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2ca2de18d2e4c83a44246d79f233970. Latest consensus state: current_term: 1 leader_uuid: "e2ca2de18d2e4c83a44246d79f233970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER } }
I20260709 13:21:30.168110 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:30.170318 18751 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:21:30.169726 18742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2ca2de18d2e4c83a44246d79f233970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2ca2de18d2e4c83a44246d79f233970" member_type: VOTER } }
I20260709 13:21:30.170791 18742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [sys.catalog]: This master's current role is: LEADER
I20260709 13:21:30.177536 18751 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:21:30.180202  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:21:30.189766 18751 catalog_manager.cc:1383] Generated new cluster ID: 5b5c6a9275a741928c3eee8e832c0754
I20260709 13:21:30.190164 18751 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:21:30.226161 18751 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:21:30.227766 18751 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:21:30.252547 18751 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970: Generated new TSK 0
I20260709 13:21:30.253304 18751 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:21:30.313050  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:30.321885 18759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.323491 18760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.326737 18762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.326951  8925 server_base.cc:1061] running on GCE node
I20260709 13:21:30.330224  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:30.330526  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:30.330816  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603290330790 us; error 0 us; skew 500 ppm
I20260709 13:21:30.331564  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:30.334854  8925 webserver.cc:533] Webserver started at http://127.8.183.65:40061/ using document root <none> and password file <none>
I20260709 13:21:30.335645  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:30.335872  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:30.336141  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:30.337626  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "878e795a8488495f978538952e7b2324"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.343446  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:21:30.347929 18767 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.349202  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:21:30.349783  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "878e795a8488495f978538952e7b2324"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.350178  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:30.409577  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:30.411373  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:30.413395  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:30.416805  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:30.417129  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.417491  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:30.417723  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.472972  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:40287
I20260709 13:21:30.473388 18829 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:40287 every 8 connection(s)
I20260709 13:21:30.479852  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:30.491506 18834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.494211 18835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.498574  8925 server_base.cc:1061] running on GCE node
W20260709 13:21:30.498713 18837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.504580  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:30.505069  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:30.505251  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603290505235 us; error 0 us; skew 500 ppm
I20260709 13:21:30.506006  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:30.506070 18830 heartbeater.cc:344] Connected to a master server at 127.8.183.126:45441
I20260709 13:21:30.506691 18830 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:30.507972 18830 heartbeater.cc:507] Master 127.8.183.126:45441 requested a full tablet report, sending...
I20260709 13:21:30.509820  8925 webserver.cc:533] Webserver started at http://127.8.183.66:37567/ using document root <none> and password file <none>
I20260709 13:21:30.510622  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:30.510834  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:30.510905 18703 ts_manager.cc:194] Registered new tserver with Master: 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:30.511301  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:30.513227  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.513386 18703 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48472
I20260709 13:21:30.520143  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:21:30.525830 18842 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.527099  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:21:30.527639  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.527999  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:30.566859  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:30.568540  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:30.570454  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:30.573633  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:30.573966  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.574281  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:30.574479  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.627187  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:37707
I20260709 13:21:30.627301 18904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:37707 every 8 connection(s)
I20260709 13:21:30.633505  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:30.643082 18908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.645514 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.647359  8925 server_base.cc:1061] running on GCE node
W20260709 13:21:30.649567 18911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.650403 18905 heartbeater.cc:344] Connected to a master server at 127.8.183.126:45441
I20260709 13:21:30.650902 18905 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:30.651002  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:30.651409  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:30.651643  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603290651620 us; error 0 us; skew 500 ppm
I20260709 13:21:30.652107 18905 heartbeater.cc:507] Master 127.8.183.126:45441 requested a full tablet report, sending...
I20260709 13:21:30.652539  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:30.655040 18703 ts_manager.cc:194] Registered new tserver with Master: 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:30.656279  8925 webserver.cc:533] Webserver started at http://127.8.183.67:36655/ using document root <none> and password file <none>
I20260709 13:21:30.657214  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:30.657531  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:30.657471 18703 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48480
I20260709 13:21:30.658134  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:30.660193  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.666335  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:21:30.671779 18916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.672996  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:21:30.673405  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.673732  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:30.703934  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:30.705857  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:30.707733  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:30.710786  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:30.711125  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.711485  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:30.711694  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.767755  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:37419
I20260709 13:21:30.767884 18978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:37419 every 8 connection(s)
I20260709 13:21:30.775278  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:30.785322 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:30.787226 18983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.790004 18979 heartbeater.cc:344] Connected to a master server at 127.8.183.126:45441
I20260709 13:21:30.790788 18979 heartbeater.cc:461] Registering TS with master...
W20260709 13:21:30.791356 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:30.792114  8925 server_base.cc:1061] running on GCE node
I20260709 13:21:30.792246 18979 heartbeater.cc:507] Master 127.8.183.126:45441 requested a full tablet report, sending...
I20260709 13:21:30.793623  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:30.793944  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:30.794262  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603290794241 us; error 0 us; skew 500 ppm
I20260709 13:21:30.795197  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:30.795841 18703 ts_manager.cc:194] Registered new tserver with Master: de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:30.798029 18703 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48488
I20260709 13:21:30.798666  8925 webserver.cc:533] Webserver started at http://127.8.183.68:41293/ using document root <none> and password file <none>
I20260709 13:21:30.799588  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:30.799919  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:21:30.800247  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:21:30.801715  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-3-root/instance:
uuid: "f3e2fa7cf3974c27a19386f362688d29"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.807261  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 13:21:30.811575 18990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.812605  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:21:30.813056  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
uuid: "f3e2fa7cf3974c27a19386f362688d29"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:30.813452  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:30.840759  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:30.842519  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:30.844543  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:30.847666  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:21:30.847963  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.848253  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:21:30.848453  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:30.906551  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.68:46235
I20260709 13:21:30.906677 19052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.68:46235 every 8 connection(s)
I20260709 13:21:30.926499 19053 heartbeater.cc:344] Connected to a master server at 127.8.183.126:45441
I20260709 13:21:30.927039 19053 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:30.928123 19053 heartbeater.cc:507] Master 127.8.183.126:45441 requested a full tablet report, sending...
I20260709 13:21:30.930830 18703 ts_manager.cc:194] Registered new tserver with Master: f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:30.931288  8925 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.020511721s
I20260709 13:21:30.933152 18703 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48490
I20260709 13:21:30.971055 18703 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48502:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:21:31.175429 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:31.296615 18870 tablet_service.cc:1511] Processing CreateTablet for tablet a3b2d906586545bda1c94132ed20782a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:31.298606 18870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b2d906586545bda1c94132ed20782a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.300602 18869 tablet_service.cc:1511] Processing CreateTablet for tablet 09abb2d24f6748bf938539c210dc01fc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:31.301291 18944 tablet_service.cc:1511] Processing CreateTablet for tablet 09abb2d24f6748bf938539c210dc01fc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:31.302520 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09abb2d24f6748bf938539c210dc01fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.303057 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09abb2d24f6748bf938539c210dc01fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.309973 18940 tablet_service.cc:1511] Processing CreateTablet for tablet e3e819f8fc7440af86dd0b3190f1edf3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:31.304646 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 0346a3d5c16b4965a7492826c0da6872 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:31.307538 19018 tablet_service.cc:1511] Processing CreateTablet for tablet a3b2d906586545bda1c94132ed20782a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:31.312028 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0346a3d5c16b4965a7492826c0da6872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.312249 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b2d906586545bda1c94132ed20782a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.305816 18867 tablet_service.cc:1511] Processing CreateTablet for tablet d217d9f5218640ed95be5641ff1fb098 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:31.316324 18867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217d9f5218640ed95be5641ff1fb098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.306043 18942 tablet_service.cc:1511] Processing CreateTablet for tablet 0346a3d5c16b4965a7492826c0da6872 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:31.307233 18941 tablet_service.cc:1511] Processing CreateTablet for tablet d217d9f5218640ed95be5641ff1fb098 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:31.320600 18940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3e819f8fc7440af86dd0b3190f1edf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.320779 18939 tablet_service.cc:1511] Processing CreateTablet for tablet f54c60d833534725beae61cec6e75c4b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:31.322666 18939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c60d833534725beae61cec6e75c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.329016 18793 tablet_service.cc:1511] Processing CreateTablet for tablet ab7b798c63c94bce8e69fefd480e295b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:31.330077 18941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217d9f5218640ed95be5641ff1fb098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.332196 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0346a3d5c16b4965a7492826c0da6872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.332836 18792 tablet_service.cc:1511] Processing CreateTablet for tablet d217d9f5218640ed95be5641ff1fb098 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:21:31.334856 19017 tablet_service.cc:1511] Processing CreateTablet for tablet ab7b798c63c94bce8e69fefd480e295b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:31.336097 18791 tablet_service.cc:1511] Processing CreateTablet for tablet e076455b2f73491282b009a90112d07d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:31.328835 18794 tablet_service.cc:1511] Processing CreateTablet for tablet 09abb2d24f6748bf938539c210dc01fc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:21:31.336915 19017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7b798c63c94bce8e69fefd480e295b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.338971 19015 tablet_service.cc:1511] Processing CreateTablet for tablet e076455b2f73491282b009a90112d07d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:31.340340 18795 tablet_service.cc:1511] Processing CreateTablet for tablet a3b2d906586545bda1c94132ed20782a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:21:31.340819 19015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e076455b2f73491282b009a90112d07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.342990 18790 tablet_service.cc:1511] Processing CreateTablet for tablet e3e819f8fc7440af86dd0b3190f1edf3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:31.338971 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 0346a3d5c16b4965a7492826c0da6872 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:21:31.346526 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0346a3d5c16b4965a7492826c0da6872. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.348963 18789 tablet_service.cc:1511] Processing CreateTablet for tablet 13175817e62e49c38eafca15e3877bc6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:31.304658 18943 tablet_service.cc:1511] Processing CreateTablet for tablet ab7b798c63c94bce8e69fefd480e295b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:21:31.351576 19014 tablet_service.cc:1511] Processing CreateTablet for tablet e3e819f8fc7440af86dd0b3190f1edf3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:21:31.356858 18943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7b798c63c94bce8e69fefd480e295b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.357560 19014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3e819f8fc7440af86dd0b3190f1edf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.353993 18788 tablet_service.cc:1511] Processing CreateTablet for tablet 03fbc43806c04306990613e14c199ff1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:31.355324 18787 tablet_service.cc:1511] Processing CreateTablet for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:31.364056 19013 tablet_service.cc:1511] Processing CreateTablet for tablet f54c60d833534725beae61cec6e75c4b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:31.365744 18866 tablet_service.cc:1511] Processing CreateTablet for tablet e076455b2f73491282b009a90112d07d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:21:31.365798 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c60d833534725beae61cec6e75c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.367347 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e076455b2f73491282b009a90112d07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.369843 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7b798c63c94bce8e69fefd480e295b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.371019 18795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3b2d906586545bda1c94132ed20782a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.372645 18787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.372860 18865 tablet_service.cc:1511] Processing CreateTablet for tablet f54c60d833534725beae61cec6e75c4b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:21:31.373669 18792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d217d9f5218640ed95be5641ff1fb098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.374565 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54c60d833534725beae61cec6e75c4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.374565 18794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09abb2d24f6748bf938539c210dc01fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.376178 18789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13175817e62e49c38eafca15e3877bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.377331 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03fbc43806c04306990613e14c199ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.378965 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e076455b2f73491282b009a90112d07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.383450 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3e819f8fc7440af86dd0b3190f1edf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.389827 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 13175817e62e49c38eafca15e3877bc6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:31.391919 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13175817e62e49c38eafca15e3877bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.397131 18937 tablet_service.cc:1511] Processing CreateTablet for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:31.397426 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3966983799476b9516092542b22438 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:31.400710 18935 tablet_service.cc:1511] Processing CreateTablet for tablet 45bd3ea063de41839fe0187978e00474 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:31.402511 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.404969 18786 tablet_service.cc:1511] Processing CreateTablet for tablet 45bd3ea063de41839fe0187978e00474 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:31.406461 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3966983799476b9516092542b22438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.407454 18935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45bd3ea063de41839fe0187978e00474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.406898 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45bd3ea063de41839fe0187978e00474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.432251 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 13175817e62e49c38eafca15e3877bc6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:21:31.434190 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13175817e62e49c38eafca15e3877bc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.433705 19012 tablet_service.cc:1511] Processing CreateTablet for tablet 03fbc43806c04306990613e14c199ff1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:31.436283 19012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03fbc43806c04306990613e14c199ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.436650 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:21:31.438369 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33445ebf2e3d4cb2bfea774ce4be1ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.441411 19010 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3966983799476b9516092542b22438 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:31.445427 19009 tablet_service.cc:1511] Processing CreateTablet for tablet da6dc24978cc43a79db989f59bfed38a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:31.447471 19010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3966983799476b9516092542b22438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.451653 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6dc24978cc43a79db989f59bfed38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.474802 18863 tablet_service.cc:1511] Processing CreateTablet for tablet 03fbc43806c04306990613e14c199ff1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:21:31.493947 18863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03fbc43806c04306990613e14c199ff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.495501 18862 tablet_service.cc:1511] Processing CreateTablet for tablet 3e3966983799476b9516092542b22438 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:21:31.497231 18862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e3966983799476b9516092542b22438. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.501742 18861 tablet_service.cc:1511] Processing CreateTablet for tablet 45bd3ea063de41839fe0187978e00474 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:21:31.503652 18861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45bd3ea063de41839fe0187978e00474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.517441 18830 heartbeater.cc:499] Master 127.8.183.126:45441 was elected leader, sending a full tablet report...
I20260709 13:21:31.616793 19073 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:31.619243 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 14fe966c22b14da9985eba8dd9ed1f2c (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:31.621133 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fe966c22b14da9985eba8dd9ed1f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.652884 19075 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:31.654615 18786 tablet_service.cc:1511] Processing CreateTablet for tablet da6dc24978cc43a79db989f59bfed38a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:31.656160 19074 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:31.656555 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6dc24978cc43a79db989f59bfed38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.661885 18905 heartbeater.cc:499] Master 127.8.183.126:45441 was elected leader, sending a full tablet report...
I20260709 13:21:31.666186 18936 tablet_service.cc:1511] Processing CreateTablet for tablet d350b38546104d82881fdcd2f9c251d3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:31.668041 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d350b38546104d82881fdcd2f9c251d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.674233 18943 tablet_service.cc:1511] Processing CreateTablet for tablet 14fe966c22b14da9985eba8dd9ed1f2c (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:31.683777 18943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fe966c22b14da9985eba8dd9ed1f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.683404 18786 tablet_service.cc:1511] Processing CreateTablet for tablet d350b38546104d82881fdcd2f9c251d3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:31.712576 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d350b38546104d82881fdcd2f9c251d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.687026 18787 tablet_service.cc:1511] Processing CreateTablet for tablet c67b792f4ba14e55abb2a3a92d4d0b2b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:31.716367 18787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67b792f4ba14e55abb2a3a92d4d0b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.718452 19075 tablet_bootstrap.cc:654] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.695006 18865 tablet_service.cc:1511] Processing CreateTablet for tablet 14fe966c22b14da9985eba8dd9ed1f2c (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:21:31.727406 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14fe966c22b14da9985eba8dd9ed1f2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.734872 18940 tablet_service.cc:1511] Processing CreateTablet for tablet 4832e091fcef44ab877283bf9a254752 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:31.744479 19016 tablet_service.cc:1511] Processing CreateTablet for tablet d350b38546104d82881fdcd2f9c251d3 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:21:31.746120 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d350b38546104d82881fdcd2f9c251d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.752372 19073 tablet_bootstrap.cc:654] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.754345 19076 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:31.705165 18793 tablet_service.cc:1511] Processing CreateTablet for tablet 04089f925f3e44ab917020605bd8a75e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:31.761005 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04089f925f3e44ab917020605bd8a75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.700634 18942 tablet_service.cc:1511] Processing CreateTablet for tablet c67b792f4ba14e55abb2a3a92d4d0b2b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:31.756582 18870 tablet_service.cc:1511] Processing CreateTablet for tablet da6dc24978cc43a79db989f59bfed38a (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:21:31.706733 18790 tablet_service.cc:1511] Processing CreateTablet for tablet 4832e091fcef44ab877283bf9a254752 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:31.759332 18940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4832e091fcef44ab877283bf9a254752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.763001 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 04089f925f3e44ab917020605bd8a75e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:31.781400 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04089f925f3e44ab917020605bd8a75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.765529 18870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6dc24978cc43a79db989f59bfed38a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.766634 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4832e091fcef44ab877283bf9a254752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.765048 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 04089f925f3e44ab917020605bd8a75e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:21:31.774173 19074 tablet_bootstrap.cc:654] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.813315 18979 heartbeater.cc:499] Master 127.8.183.126:45441 was elected leader, sending a full tablet report...
I20260709 13:21:31.746120 18862 tablet_service.cc:1511] Processing CreateTablet for tablet c67b792f4ba14e55abb2a3a92d4d0b2b (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:21:31.819712 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67b792f4ba14e55abb2a3a92d4d0b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.820520 18869 tablet_service.cc:1511] Processing CreateTablet for tablet a4a64164017441fd8ce325d46dcbbd07 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:31.777594 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 4832e091fcef44ab877283bf9a254752 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:21:31.824719 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4832e091fcef44ab877283bf9a254752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.832721 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04089f925f3e44ab917020605bd8a75e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.857852 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a64164017441fd8ce325d46dcbbd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.871598 18862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c67b792f4ba14e55abb2a3a92d4d0b2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.878118 19073 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:31.878978 19073 ts_tablet_manager.cc:1403] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.263s	user 0.016s	sys 0.016s
I20260709 13:21:31.882800 19073 raft_consensus.cc:359] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:31.889889 19073 raft_consensus.cc:385] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.890312 19073 raft_consensus.cc:740] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.891194 18791 tablet_service.cc:1511] Processing CreateTablet for tablet bcc4d045faa6417e894366a8fb7b9f52 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:31.891438 19073 consensus_queue.cc:260] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:31.893020 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc4d045faa6417e894366a8fb7b9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.897755 19009 tablet_service.cc:1511] Processing CreateTablet for tablet a4a64164017441fd8ce325d46dcbbd07 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:31.899505 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a64164017441fd8ce325d46dcbbd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.924341 18866 tablet_service.cc:1511] Processing CreateTablet for tablet bcc4d045faa6417e894366a8fb7b9f52 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:31.926216 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc4d045faa6417e894366a8fb7b9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.929698 18938 tablet_service.cc:1511] Processing CreateTablet for tablet a4a64164017441fd8ce325d46dcbbd07 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:21:31.933156 18944 tablet_service.cc:1511] Processing CreateTablet for tablet bcc4d045faa6417e894366a8fb7b9f52 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:21:31.936556 19075 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:31.937300 19075 ts_tablet_manager.cc:1403] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.285s	user 0.030s	sys 0.029s
I20260709 13:21:31.938073 19053 heartbeater.cc:499] Master 127.8.183.126:45441 was elected leader, sending a full tablet report...
I20260709 13:21:31.941951 19075 raft_consensus.cc:359] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:31.943511 19075 raft_consensus.cc:385] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:31.944945 19076 tablet_bootstrap.cc:654] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:31.944525 19075 raft_consensus.cc:740] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:31.943454 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4a64164017441fd8ce325d46dcbbd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.986755 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcc4d045faa6417e894366a8fb7b9f52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.990899 19009 tablet_service.cc:1511] Processing CreateTablet for tablet e3bb98bec02448d6990ba27836e9e7a9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:31.992820 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3bb98bec02448d6990ba27836e9e7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:31.998993 18786 tablet_service.cc:1511] Processing CreateTablet for tablet e3bb98bec02448d6990ba27836e9e7a9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:32.000840 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3bb98bec02448d6990ba27836e9e7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.000379 18861 tablet_service.cc:1511] Processing CreateTablet for tablet e3bb98bec02448d6990ba27836e9e7a9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:21:32.004315 18861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3bb98bec02448d6990ba27836e9e7a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.008569 19073 ts_tablet_manager.cc:1434] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.129s	user 0.033s	sys 0.040s
I20260709 13:21:32.010089 19073 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:31.962510 19075 consensus_queue.cc:260] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.019909 18935 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac0dee3476f4361a8531c64161fee77 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:32.021694 18935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac0dee3476f4361a8531c64161fee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.021994 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac0dee3476f4361a8531c64161fee77 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:32.023710 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac0dee3476f4361a8531c64161fee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.025928 18788 tablet_service.cc:1511] Processing CreateTablet for tablet 8ac0dee3476f4361a8531c64161fee77 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:21:32.030084 18864 tablet_service.cc:1511] Processing CreateTablet for tablet c692689ede48466a882155f06b0763ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:32.031981 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c692689ede48466a882155f06b0763ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.040213 19074 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:32.049968 18937 tablet_service.cc:1511] Processing CreateTablet for tablet c692689ede48466a882155f06b0763ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:32.051786 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c692689ede48466a882155f06b0763ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.060025 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ac0dee3476f4361a8531c64161fee77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.067771 19079 raft_consensus.cc:493] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.068506 19079 raft_consensus.cc:515] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.062779 19073 tablet_bootstrap.cc:654] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.074074 19076 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:32.074822 19076 ts_tablet_manager.cc:1403] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.321s	user 0.011s	sys 0.012s
I20260709 13:21:32.086862 19074 ts_tablet_manager.cc:1403] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.431s	user 0.018s	sys 0.030s
I20260709 13:21:32.096299 19075 ts_tablet_manager.cc:1434] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.159s	user 0.019s	sys 0.018s
I20260709 13:21:32.097851 19075 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:32.098219 19076 raft_consensus.cc:359] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:32.121115 19079 leader_election.cc:290] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:32.103320 19076 raft_consensus.cc:385] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.122680 19076 raft_consensus.cc:740] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.123724 19076 consensus_queue.cc:260] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:32.131071 19074 raft_consensus.cc:359] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:32.136080 19074 raft_consensus.cc:385] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.136421 19074 raft_consensus.cc:740] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.137440 19074 consensus_queue.cc:260] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:32.161123 19082 raft_consensus.cc:493] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.162135 19082 raft_consensus.cc:515] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:32.174300 19054 tablet.cc:2406] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:21:32.176049 19073 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:32.176846 19073 ts_tablet_manager.cc:1403] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.167s	user 0.016s	sys 0.003s
I20260709 13:21:32.177989 19075 tablet_bootstrap.cc:654] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:32.183808 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:32.214689 19073 raft_consensus.cc:359] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.215781 19073 raft_consensus.cc:385] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.216205 19073 raft_consensus.cc:740] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.217438 19073 consensus_queue.cc:260] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.185755 19082 leader_election.cc:290] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:32.221079 19076 ts_tablet_manager.cc:1434] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.146s	user 0.012s	sys 0.008s
I20260709 13:21:32.228523 19009 tablet_service.cc:1511] Processing CreateTablet for tablet ccb9f9f9dbb04b459b94b15ddb6864d5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:32.230571 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb9f9f9dbb04b459b94b15ddb6864d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.222971 19076 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:32.241814 19074 ts_tablet_manager.cc:1434] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.152s	user 0.020s	sys 0.002s
I20260709 13:21:32.182844 18786 tablet_service.cc:1511] Processing CreateTablet for tablet b57e1d1c76904a8492670c30af464d8d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:32.253177 19076 tablet_bootstrap.cc:654] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.258559 18861 tablet_service.cc:1511] Processing CreateTablet for tablet ccb9f9f9dbb04b459b94b15ddb6864d5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:32.271627 19075 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:32.272253 18861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb9f9f9dbb04b459b94b15ddb6864d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.272501 19075 ts_tablet_manager.cc:1403] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.175s	user 0.023s	sys 0.036s
I20260709 13:21:32.277015 19013 tablet_service.cc:1511] Processing CreateTablet for tablet c692689ede48466a882155f06b0763ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:21:32.277266 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 4f2be91edb9e4188a248db3e309f0e44 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:32.278228 19075 raft_consensus.cc:359] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.278895 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c692689ede48466a882155f06b0763ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.272336 18788 tablet_service.cc:1511] Processing CreateTablet for tablet ccb9f9f9dbb04b459b94b15ddb6864d5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:21:32.279716 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2be91edb9e4188a248db3e309f0e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.285168 19075 raft_consensus.cc:385] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.290814 19075 raft_consensus.cc:740] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.292760 18791 tablet_service.cc:1511] Processing CreateTablet for tablet 4f2be91edb9e4188a248db3e309f0e44 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:32.292936 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:32.295270 18991 leader_election.cc:343] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.296940 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57e1d1c76904a8492670c30af464d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.291270 18866 tablet_service.cc:1511] Processing CreateTablet for tablet b57e1d1c76904a8492670c30af464d8d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:32.291865 19075 consensus_queue.cc:260] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.298993 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57e1d1c76904a8492670c30af464d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.305706 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2be91edb9e4188a248db3e309f0e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.306120 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 211a9a1cbd3f4dca903a20b4ecb32209 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:32.307868 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211a9a1cbd3f4dca903a20b4ecb32209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.311740 19074 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:32.312307 19073 ts_tablet_manager.cc:1434] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.135s	user 0.008s	sys 0.003s
I20260709 13:21:32.312700 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccb9f9f9dbb04b459b94b15ddb6864d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.313701 19073 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:32.316326 19075 ts_tablet_manager.cc:1434] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 13:21:32.317749 19075 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:32.333055 19018 tablet_service.cc:1511] Processing CreateTablet for tablet 211a9a1cbd3f4dca903a20b4ecb32209 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:32.334760 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211a9a1cbd3f4dca903a20b4ecb32209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.337644 18937 tablet_service.cc:1511] Processing CreateTablet for tablet b57e1d1c76904a8492670c30af464d8d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:21:32.340039 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b57e1d1c76904a8492670c30af464d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.348775 18790 tablet_service.cc:1511] Processing CreateTablet for tablet 6870bd6f453d45a19a939d17ee0f92ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:32.349519 19075 tablet_bootstrap.cc:654] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.350682 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6870bd6f453d45a19a939d17ee0f92ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.338830 18935 tablet_service.cc:1511] Processing CreateTablet for tablet 211a9a1cbd3f4dca903a20b4ecb32209 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:21:32.352210 18935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 211a9a1cbd3f4dca903a20b4ecb32209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.356323 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 6870bd6f453d45a19a939d17ee0f92ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:32.356989 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.358131 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6870bd6f453d45a19a939d17ee0f92ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:32.359088 18994 leader_election.cc:343] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.359768 18994 leader_election.cc:304] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:32.360358 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 6870bd6f453d45a19a939d17ee0f92ad (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:21:32.361459 19083 raft_consensus.cc:2749] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.362061 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6870bd6f453d45a19a939d17ee0f92ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.363437 18865 tablet_service.cc:1511] Processing CreateTablet for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:32.364270 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:32.365007 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.365715 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.341471 18942 tablet_service.cc:1511] Processing CreateTablet for tablet 4f2be91edb9e4188a248db3e309f0e44 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:21:32.342301 19074 tablet_bootstrap.cc:654] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.372020 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f2be91edb9e4188a248db3e309f0e44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.373095 18787 tablet_service.cc:1511] Processing CreateTablet for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:21:32.375056 18787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c3bd79ae04b4dfe8ebf5c82eb93cac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.397256 19073 tablet_bootstrap.cc:654] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.398252 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:32.402864 18917 leader_election.cc:343] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.409907 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:32.428709 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.434026 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:32.435875 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.468807 19083 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.469666 19083 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.472357 19076 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:32.473086 19076 ts_tablet_manager.cc:1403] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.250s	user 0.018s	sys 0.007s
I20260709 13:21:32.476984 19088 raft_consensus.cc:493] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.477180 19076 raft_consensus.cc:359] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.478156 19076 raft_consensus.cc:385] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.477782 19088 raft_consensus.cc:515] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:32.478547 19076 raft_consensus.cc:740] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.481537 19088 leader_election.cc:290] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:32.497495 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:32.500020 18918 leader_election.cc:343] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.500689 18918 leader_election.cc:304] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:32.502686 19082 raft_consensus.cc:2749] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.502779 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.510552 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:32.511754 18994 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.511430 19076 consensus_queue.cc:260] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:32.512532 18991 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.513093 18991 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:32.515933 19079 raft_consensus.cc:2749] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.519065 19076 ts_tablet_manager.cc:1434] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.000s
I20260709 13:21:32.521081 19076 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:32.522899 19087 raft_consensus.cc:493] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.528626 19087 raft_consensus.cc:515] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.536085 18869 tablet_service.cc:1511] Processing CreateTablet for tablet d3f20d931bf640d09b913bc7a1e8e3a8 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:32.549499 19087 leader_election.cc:290] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:32.500142 19083 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:32.550829 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f20d931bf640d09b913bc7a1e8e3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.569506 18944 tablet_service.cc:1511] Processing CreateTablet for tablet d3f20d931bf640d09b913bc7a1e8e3a8 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:32.570534 19074 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:32.571324 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f20d931bf640d09b913bc7a1e8e3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.571473 19074 ts_tablet_manager.cc:1403] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.281s	user 0.041s	sys 0.021s
I20260709 13:21:32.583467 19074 raft_consensus.cc:359] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.584939 19074 raft_consensus.cc:385] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.585448 19074 raft_consensus.cc:740] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.586995 19074 consensus_queue.cc:260] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.596243 19088 raft_consensus.cc:493] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.597193 19088 raft_consensus.cc:515] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.592161 19074 ts_tablet_manager.cc:1434] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.003s
I20260709 13:21:32.615089 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:32.616616 19075 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:32.616616 19073 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:32.617540 19073 ts_tablet_manager.cc:1403] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.304s	user 0.028s	sys 0.020s
I20260709 13:21:32.617606 19074 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:32.617859 18768 leader_election.cc:343] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.624707 19073 raft_consensus.cc:359] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.617540 19075 ts_tablet_manager.cc:1403] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.300s	user 0.028s	sys 0.031s
I20260709 13:21:32.631002 19073 raft_consensus.cc:385] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.631669 19073 raft_consensus.cc:740] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.629835 19074 tablet_bootstrap.cc:654] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.620424 19076 tablet_bootstrap.cc:654] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.636101 19075 raft_consensus.cc:359] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.608421 19088 leader_election.cc:290] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:32.642407 19087 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.654390 18791 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:21:32.655385 18787 tablet_service.cc:1511] Processing CreateTablet for tablet f302bf297c554158b7f4ebcd5c7760bc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:32.678560 19087 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:32.682348 19073 consensus_queue.cc:260] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.688223 18787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f302bf297c554158b7f4ebcd5c7760bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.691603 19073 ts_tablet_manager.cc:1434] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.001s
I20260709 13:21:32.693048 19073 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:32.695088 18865 tablet_service.cc:1511] Processing CreateTablet for tablet f302bf297c554158b7f4ebcd5c7760bc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:32.697005 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f302bf297c554158b7f4ebcd5c7760bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.677062 19075 raft_consensus.cc:385] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.708447 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac00a69e4894ac79bdbbb74eaa3ad51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.703265 19083 raft_consensus.cc:493] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.709286 19083 raft_consensus.cc:515] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.700726 19075 raft_consensus.cc:740] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.727368 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.717404 19083 leader_election.cc:290] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:32.729703 18994 leader_election.cc:343] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.736572 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 7175a5fa52a94e2d8504a19eb7aa1421 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:32.728861 19075 consensus_queue.cc:260] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.743824 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7175a5fa52a94e2d8504a19eb7aa1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.745317 19075 ts_tablet_manager.cc:1434] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.000s
I20260709 13:21:32.746582 19075 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:32.747005 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:32.749801 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.720647 19087 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:32.785981 19073 tablet_bootstrap.cc:654] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.787722 18866 tablet_service.cc:1511] Processing CreateTablet for tablet bdd0d96cdc814e8fa997150f58dda773 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:32.791760 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd0d96cdc814e8fa997150f58dda773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.791951 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.788614 19082 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.793591 19082 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.795945 19018 tablet_service.cc:1511] Processing CreateTablet for tablet d3f20d931bf640d09b913bc7a1e8e3a8 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:21:32.798031 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3f20d931bf640d09b913bc7a1e8e3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.799737 18937 tablet_service.cc:1511] Processing CreateTablet for tablet bdd0d96cdc814e8fa997150f58dda773 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:32.800442 19082 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:32.801568 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd0d96cdc814e8fa997150f58dda773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.803103 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:32.805025 18846 leader_election.cc:343] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.805683 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 577d52a3cf7346349e47f24b3e1286e1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 13:21:32.806591 18918 leader_election.cc:343] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.788966 19076 tablet_bootstrap.cc:492] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:32.807590 18936 tablet_service.cc:1511] Processing CreateTablet for tablet f302bf297c554158b7f4ebcd5c7760bc (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:21:32.816800 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577d52a3cf7346349e47f24b3e1286e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.817931 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 7175a5fa52a94e2d8504a19eb7aa1421 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:32.819439 18790 tablet_service.cc:1511] Processing CreateTablet for tablet 7175a5fa52a94e2d8504a19eb7aa1421 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:21:32.820724 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7175a5fa52a94e2d8504a19eb7aa1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.821185 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7175a5fa52a94e2d8504a19eb7aa1421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.823553 18788 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:32.823746 18786 tablet_service.cc:1511] Processing CreateTablet for tablet 577d52a3cf7346349e47f24b3e1286e1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:32.812932 19076 ts_tablet_manager.cc:1403] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.292s	user 0.011s	sys 0.013s
I20260709 13:21:32.826382 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:21:32.828387 19016 tablet_service.cc:1511] Processing CreateTablet for tablet bdd0d96cdc814e8fa997150f58dda773 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:21:32.829806 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.830847 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:32.836782 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f302bf297c554158b7f4ebcd5c7760bc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:32.850521 18771 leader_election.cc:343] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.851389 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:32.851688 18771 leader_election.cc:343] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.852211 18771 leader_election.cc:304] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:32.860577 19088 raft_consensus.cc:2749] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.862599 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd0d96cdc814e8fa997150f58dda773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.860975 18944 tablet_service.cc:1511] Processing CreateTablet for tablet 577d52a3cf7346349e47f24b3e1286e1 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:21:32.867904 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577d52a3cf7346349e47f24b3e1286e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.868575 19009 tablet_service.cc:1511] Processing CreateTablet for tablet afdc899ecc694b09991abe77a2195448 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:32.868636 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb1cf5992cd4659b7ab7f2ff1307fd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.872396 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afdc899ecc694b09991abe77a2195448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.872390 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577d52a3cf7346349e47f24b3e1286e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.866647 19076 raft_consensus.cc:359] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.888605 19076 raft_consensus.cc:385] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.889003 19076 raft_consensus.cc:740] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.858177 19075 tablet_bootstrap.cc:654] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:32.907995 19073 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:32.909425 19073 ts_tablet_manager.cc:1403] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.217s	user 0.027s	sys 0.004s
I20260709 13:21:32.910620 18793 tablet_service.cc:1511] Processing CreateTablet for tablet afdc899ecc694b09991abe77a2195448 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:32.911983 19074 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:32.912334 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afdc899ecc694b09991abe77a2195448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.912599 19074 ts_tablet_manager.cc:1403] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.295s	user 0.010s	sys 0.033s
I20260709 13:21:32.915675 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:32.917487 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:32.919878 18844 leader_election.cc:343] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.920449 18844 leader_election.cc:304] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:32.921214 19074 raft_consensus.cc:359] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:32.921866 18844 leader_election.cc:343] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.922188 19074 raft_consensus.cc:385] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.922571 19074 raft_consensus.cc:740] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:32.922753 19087 raft_consensus.cc:2749] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.928068 19076 consensus_queue.cc:260] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:32.929950 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:32.931344 19076 ts_tablet_manager.cc:1434] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.003s
I20260709 13:21:32.935907 18869 tablet_service.cc:1511] Processing CreateTablet for tablet afdc899ecc694b09991abe77a2195448 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:21:32.937645 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afdc899ecc694b09991abe77a2195448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:32.934651 19076 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:32.932400 19074 consensus_queue.cc:260] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:32.977833 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:32.978371 19074 ts_tablet_manager.cc:1434] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.065s	user 0.005s	sys 0.005s
W20260709 13:21:32.975647 18769 leader_election.cc:343] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.979475 18769 leader_election.cc:304] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:32.982301 19088 raft_consensus.cc:2749] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:32.984647 18993 leader_election.cc:343] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.985316 18993 leader_election.cc:304] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:32.986922 19083 raft_consensus.cc:2749] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.987103 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:32.989081 18845 leader_election.cc:343] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:32.989737 18845 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:32.991272 19087 raft_consensus.cc:2749] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:32.998314 19073 raft_consensus.cc:359] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:32.999471 19073 raft_consensus.cc:385] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:32.999812 19073 raft_consensus.cc:740] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.003646 19073 consensus_queue.cc:260] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.012426 19074 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:33.021279 19087 raft_consensus.cc:493] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.027858 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:33.027745 19087 raft_consensus.cc:515] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:33.029922 18919 leader_election.cc:343] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.030561 18919 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:33.032111 19082 raft_consensus.cc:2749] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.032652 19087 leader_election.cc:290] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.032419 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.033332 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:33.033975 18954 raft_consensus.cc:2468] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:33.036235 18846 leader_election.cc:304] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:33.033915 19027 raft_consensus.cc:2468] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:33.038072 18944 tablet_service.cc:1511] Processing CreateTablet for tablet b4c3f02bc00e460dba727510708feee6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:33.040704 19087 raft_consensus.cc:2804] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.048475 19087 raft_consensus.cc:493] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.049249 19087 raft_consensus.cc:3060] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.043614 19073 ts_tablet_manager.cc:1434] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.134s	user 0.010s	sys 0.001s
I20260709 13:21:33.060931 19073 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:33.065654 18868 tablet_service.cc:1511] Processing CreateTablet for tablet c4490b378f034620bcb7df67470bdab9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:33.047910 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c3f02bc00e460dba727510708feee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.067418 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4490b378f034620bcb7df67470bdab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.069438 18793 tablet_service.cc:1511] Processing CreateTablet for tablet b4c3f02bc00e460dba727510708feee6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:33.081207 19009 tablet_service.cc:1511] Processing CreateTablet for tablet b4c3f02bc00e460dba727510708feee6 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:21:33.088037 18936 tablet_service.cc:1511] Processing CreateTablet for tablet c4490b378f034620bcb7df67470bdab9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:33.091933 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4490b378f034620bcb7df67470bdab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.092089 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c3f02bc00e460dba727510708feee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.089742 19087 raft_consensus.cc:515] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.093262 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4c3f02bc00e460dba727510708feee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.106297 19013 tablet_service.cc:1511] Processing CreateTablet for tablet c4490b378f034620bcb7df67470bdab9 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:21:33.107935 19075 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:33.108434 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4490b378f034620bcb7df67470bdab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.109622 19075 ts_tablet_manager.cc:1403] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.363s	user 0.064s	sys 0.038s
I20260709 13:21:33.095700 19076 tablet_bootstrap.cc:654] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.115069 19073 tablet_bootstrap.cc:654] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.127826 19074 tablet_bootstrap.cc:654] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.127367 19075 raft_consensus.cc:359] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.128463 19075 raft_consensus.cc:385] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.128826 19075 raft_consensus.cc:740] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.128976 19088 raft_consensus.cc:493] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.129575 19088 raft_consensus.cc:515] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:33.129972 19075 consensus_queue.cc:260] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.132872 19088 leader_election.cc:290] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:33.133258 19075 ts_tablet_manager.cc:1434] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.004s
I20260709 13:21:33.137033 18790 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed73ebff034411ab7b405ac83d2e34e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:33.138837 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed73ebff034411ab7b405ac83d2e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.138435 18869 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed73ebff034411ab7b405ac83d2e34e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:21:33.139963 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed73ebff034411ab7b405ac83d2e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.142074 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:33.142580 19075 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
W20260709 13:21:33.144984 18768 leader_election.cc:343] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.146796 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.147248 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:33.148033 18954 raft_consensus.cc:3060] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:33.148882 18769 leader_election.cc:343] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.149547 18769 leader_election.cc:304] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:33.151751 19088 raft_consensus.cc:2749] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.163715 19075 tablet_bootstrap.cc:654] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.172888 19009 tablet_service.cc:1511] Processing CreateTablet for tablet d850a82d849e47ef9576b6d0b73700c0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:33.174633 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d850a82d849e47ef9576b6d0b73700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.175559 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0346a3d5c16b4965a7492826c0da6872" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:33.175751 19073 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:33.176347 19027 raft_consensus.cc:3060] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.177150 19073 ts_tablet_manager.cc:1403] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.117s	user 0.009s	sys 0.009s
I20260709 13:21:33.173454 18954 raft_consensus.cc:2468] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:33.185874 19087 leader_election.cc:290] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.184502 18786 tablet_service.cc:1511] Processing CreateTablet for tablet d850a82d849e47ef9576b6d0b73700c0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:33.191392 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d850a82d849e47ef9576b6d0b73700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.191901 19027 raft_consensus.cc:2468] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:33.195637 18844 leader_election.cc:304] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:33.201325 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed73ebff034411ab7b405ac83d2e34e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:21:33.202544 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:33.202970 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed73ebff034411ab7b405ac83d2e34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.204618 19075 tablet_bootstrap.cc:492] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:33.198698 19087 raft_consensus.cc:2804] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.205422 19075 ts_tablet_manager.cc:1403] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.002s
I20260709 13:21:33.205636 19087 raft_consensus.cc:697] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:33.207309 19087 consensus_queue.cc:237] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.219074 19073 raft_consensus.cc:359] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.219599 18944 tablet_service.cc:1511] Processing CreateTablet for tablet 6fde9dbeb78f450581397317487e31f0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:33.223875 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 6fde9dbeb78f450581397317487e31f0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:33.225390 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fde9dbeb78f450581397317487e31f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.225523 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fde9dbeb78f450581397317487e31f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.220568 19076 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:33.226497 19076 ts_tablet_manager.cc:1403] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.294s	user 0.015s	sys 0.009s
I20260709 13:21:33.234098 18790 tablet_service.cc:1511] Processing CreateTablet for tablet 6fde9dbeb78f450581397317487e31f0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:21:33.225616 19074 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:33.235286 19073 raft_consensus.cc:385] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.236698 19074 ts_tablet_manager.cc:1403] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.225s	user 0.027s	sys 0.004s
I20260709 13:21:33.236783 19073 raft_consensus.cc:740] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.241170 19083 raft_consensus.cc:493] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.241170 19074 raft_consensus.cc:359] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.242235 19074 raft_consensus.cc:385] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.242094 19083 raft_consensus.cc:515] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.242615 19074 raft_consensus.cc:740] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.243958 18790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fde9dbeb78f450581397317487e31f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.245883 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:33.251796 18994 leader_election.cc:343] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.241195 19075 raft_consensus.cc:359] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.254468 19075 raft_consensus.cc:385] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.256197 19083 leader_election.cc:290] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:33.251641 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 56139.7us
I20260709 13:21:33.257645 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:33.259876 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:33.260948 19074 consensus_queue.cc:260] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.271844 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.273214 19009 tablet_service.cc:1511] Processing CreateTablet for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:33.274922 19009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.279147 18869 tablet_service.cc:1511] Processing CreateTablet for tablet d850a82d849e47ef9576b6d0b73700c0 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:21:33.279454 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:21:33.265887 18805 raft_consensus.cc:2468] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:33.281034 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d850a82d849e47ef9576b6d0b73700c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.281903 18993 leader_election.cc:304] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:33.268733 19073 consensus_queue.cc:260] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.282404 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f9127f17e646d7a2d0f25d4a4b0d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.263072 19075 raft_consensus.cc:740] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.293756 19083 raft_consensus.cc:2804] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.294483 19083 raft_consensus.cc:493] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.294305 18866 tablet_service.cc:1511] Processing CreateTablet for tablet 706f5712f7ac487b97054415f672886f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:33.295251 19083 raft_consensus.cc:3060] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.300981 18937 tablet_service.cc:1511] Processing CreateTablet for tablet 706f5712f7ac487b97054415f672886f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:33.302774 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706f5712f7ac487b97054415f672886f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.303407 18701 catalog_manager.cc:5697] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.308832 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706f5712f7ac487b97054415f672886f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.324194 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 5977247dfbeb4ce29c4baf035adc2321 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:33.323999 18788 tablet_service.cc:1511] Processing CreateTablet for tablet 5977247dfbeb4ce29c4baf035adc2321 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:33.325971 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5977247dfbeb4ce29c4baf035adc2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.325856 19083 raft_consensus.cc:515] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.328104 19074 ts_tablet_manager.cc:1434] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.091s	user 0.009s	sys 0.001s
I20260709 13:21:33.325971 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5977247dfbeb4ce29c4baf035adc2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.319857 19076 raft_consensus.cc:359] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.335235 19076 raft_consensus.cc:385] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.336295 18793 tablet_service.cc:1511] Processing CreateTablet for tablet 706f5712f7ac487b97054415f672886f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:21:33.345680 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 706f5712f7ac487b97054415f672886f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.333139 19075 consensus_queue.cc:260] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.353605 18944 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad65f24c73445fca035eefe94eb8d15 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:33.362727 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad65f24c73445fca035eefe94eb8d15 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:33.364289 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad65f24c73445fca035eefe94eb8d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.336959 19076 raft_consensus.cc:740] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.347888 18865 tablet_service.cc:1511] Processing CreateTablet for tablet 5977247dfbeb4ce29c4baf035adc2321 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:21:33.365830 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5977247dfbeb4ce29c4baf035adc2321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.348780 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:33.349962 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33445ebf2e3d4cb2bfea774ce4be1ee4" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:33.367854 18805 raft_consensus.cc:3060] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.369051 19076 consensus_queue.cc:260] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.377650 19076 ts_tablet_manager.cc:1434] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.151s	user 0.005s	sys 0.004s
I20260709 13:21:33.342567 19074 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:33.364419 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad65f24c73445fca035eefe94eb8d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.366569 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 6808727e98af41939b051f079357a49f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:33.381430 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6808727e98af41939b051f079357a49f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:33.372025 18994 leader_election.cc:343] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.377357 18942 tablet_service.cc:1511] Processing CreateTablet for tablet 6808727e98af41939b051f079357a49f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:33.388206 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6808727e98af41939b051f079357a49f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.388304 19076 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:33.343672 19073 ts_tablet_manager.cc:1434] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.166s	user 0.014s	sys 0.000s
I20260709 13:21:33.365872 18786 tablet_service.cc:1511] Processing CreateTablet for tablet 4ad65f24c73445fca035eefe94eb8d15 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:21:33.360879 19075 ts_tablet_manager.cc:1434] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.145s	user 0.008s	sys 0.001s
I20260709 13:21:33.394879 19075 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:33.350642 19083 leader_election.cc:290] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:33.392482 19073 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:33.403601 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ad65f24c73445fca035eefe94eb8d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.409683 19018 tablet_service.cc:1511] Processing CreateTablet for tablet 6808727e98af41939b051f079357a49f (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:21:33.411599 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6808727e98af41939b051f079357a49f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.411743 19075 tablet_bootstrap.cc:654] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.412726 18791 tablet_service.cc:1511] Processing CreateTablet for tablet e83e89f3973f4275a44d76a152e43126 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:33.414552 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e89f3973f4275a44d76a152e43126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.416831 19076 tablet_bootstrap.cc:654] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.418313 19073 tablet_bootstrap.cc:654] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.435217 18936 tablet_service.cc:1511] Processing CreateTablet for tablet e83e89f3973f4275a44d76a152e43126 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:33.437017 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e89f3973f4275a44d76a152e43126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.437456 18805 raft_consensus.cc:2468] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:33.438984 18993 leader_election.cc:304] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:33.440415 19082 raft_consensus.cc:493] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.441126 19082 raft_consensus.cc:515] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.444554 19083 raft_consensus.cc:2804] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.445246 19083 raft_consensus.cc:697] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:33.453008 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:33.453956 18880 raft_consensus.cc:2468] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:33.446317 19083 consensus_queue.cc:237] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.471590 19121 raft_consensus.cc:493] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.472628 19121 raft_consensus.cc:515] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.475595 18917 leader_election.cc:304] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:33.482474 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:33.483675 19082 leader_election.cc:290] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.484402 18880 raft_consensus.cc:2468] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:33.484609 19082 raft_consensus.cc:2804] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.485087 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.486001 19027 raft_consensus.cc:2468] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:33.485996 19121 leader_election.cc:290] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:33.485338 19082 raft_consensus.cc:493] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.490110 19082 raft_consensus.cc:3060] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.490851 18991 leader_election.cc:304] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:33.492405 18869 tablet_service.cc:1511] Processing CreateTablet for tablet e83e89f3973f4275a44d76a152e43126 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:21:33.492887 19121 raft_consensus.cc:2804] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.493446 19121 raft_consensus.cc:493] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.493870 19121 raft_consensus.cc:3060] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.494500 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e83e89f3973f4275a44d76a152e43126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.498078 19013 tablet_service.cc:1511] Processing CreateTablet for tablet a01b903fffc64c949e13c3c74abe8489 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:33.499722 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01b903fffc64c949e13c3c74abe8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.514539 19088 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.522434 19088 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.530525 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.530958 18786 tablet_service.cc:1511] Processing CreateTablet for tablet a01b903fffc64c949e13c3c74abe8489 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:33.532615 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01b903fffc64c949e13c3c74abe8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.533129 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:33.533712 18769 leader_election.cc:343] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.533910 18954 raft_consensus.cc:2468] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:33.535904 18771 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:33.533950 18703 catalog_manager.cc:5697] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.537421 19088 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.538283 19088 raft_consensus.cc:2804] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.538725 19088 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.539150 19088 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.564410 19076 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:33.565419 19076 ts_tablet_manager.cc:1403] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.177s	user 0.013s	sys 0.002s
I20260709 13:21:33.567545 19088 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:33.587086 18771 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 51312.4us
I20260709 13:21:33.594538 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:33.595425 18954 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.596607 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 35fbf5e9b0e446b39953dd28e5aa71be (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:33.598390 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35fbf5e9b0e446b39953dd28e5aa71be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.600500 19121 raft_consensus.cc:515] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.601804 19127 raft_consensus.cc:493] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.602519 19127 raft_consensus.cc:515] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.612213 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.613145 19028 raft_consensus.cc:2468] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:33.614409 19082 raft_consensus.cc:515] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.567792 18865 tablet_service.cc:1511] Processing CreateTablet for tablet a01b903fffc64c949e13c3c74abe8489 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:21:33.620452 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a01b903fffc64c949e13c3c74abe8489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.621362 19082 leader_election.cc:290] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.590978 19076 raft_consensus.cc:359] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:33.591886 19075 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:33.624840 19076 raft_consensus.cc:385] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.625200 19075 ts_tablet_manager.cc:1403] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.231s	user 0.019s	sys 0.009s
I20260709 13:21:33.625303 19076 raft_consensus.cc:740] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.591568 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:33.626556 19076 consensus_queue.cc:260] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:33.561300 19074 tablet_bootstrap.cc:700] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Time spent opening tablet: real 0.102s	user 0.002s	sys 0.003s
I20260709 13:21:33.629235 19074 tablet_bootstrap.cc:654] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.630347 19076 ts_tablet_manager.cc:1434] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.005s
I20260709 13:21:33.604544 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:33.631476 18880 raft_consensus.cc:3060] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.632153 18791 tablet_service.cc:1511] Processing CreateTablet for tablet fcb4a0d3f9924af3881fae78b771264e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:33.633749 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb4a0d3f9924af3881fae78b771264e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:33.571336 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 101735us
I20260709 13:21:33.636464 18864 tablet_service.cc:1511] Processing CreateTablet for tablet fcb4a0d3f9924af3881fae78b771264e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:33.638240 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb4a0d3f9924af3881fae78b771264e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.639571 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 0f439429f2a945c5b906f0a9e2d57e41 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:33.640093 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 35fbf5e9b0e446b39953dd28e5aa71be (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:33.640473 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:33.641408 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f439429f2a945c5b906f0a9e2d57e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.641773 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35fbf5e9b0e446b39953dd28e5aa71be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:33.639461 18769 leader_election.cc:343] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.643427 18938 tablet_service.cc:1511] Processing CreateTablet for tablet fee6b2223720435195a84c3570506ea5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:33.645078 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee6b2223720435195a84c3570506ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.607276 19121 leader_election.cc:290] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:33.640393 19016 tablet_service.cc:1511] Processing CreateTablet for tablet fee6b2223720435195a84c3570506ea5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:33.650048 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee6b2223720435195a84c3570506ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.649626 19018 tablet_service.cc:1511] Processing CreateTablet for tablet 0f439429f2a945c5b906f0a9e2d57e41 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:33.650447 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:33.651314 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f439429f2a945c5b906f0a9e2d57e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.651796 18952 raft_consensus.cc:2393] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:33.614933 18918 leader_election.cc:304] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:33.615056 18869 tablet_service.cc:1511] Processing CreateTablet for tablet fee6b2223720435195a84c3570506ea5 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:21:33.652513 19075 raft_consensus.cc:359] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:33.653426 19075 raft_consensus.cc:385] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.653887 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee6b2223720435195a84c3570506ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.589973 19073 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:33.589768 19088 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:33.655155 19073 ts_tablet_manager.cc:1403] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.263s	user 0.008s	sys 0.022s
I20260709 13:21:33.655678 19082 raft_consensus.cc:2804] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.656188 19082 raft_consensus.cc:493] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.656581 19082 raft_consensus.cc:3060] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.617609 18786 tablet_service.cc:1511] Processing CreateTablet for tablet 35fbf5e9b0e446b39953dd28e5aa71be (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:21:33.654846 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:33.659879 19027 raft_consensus.cc:2393] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de62ff9486f7483e8f0e0b93dcca24bb in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:33.660286 18786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35fbf5e9b0e446b39953dd28e5aa71be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.650508 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f54c60d833534725beae61cec6e75c4b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:33.661456 18953 raft_consensus.cc:2393] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:33.661763 18918 leader_election.cc:304] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:33.650074 18944 tablet_service.cc:1511] Processing CreateTablet for tablet fcb4a0d3f9924af3881fae78b771264e (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:21:33.608492 18954 raft_consensus.cc:2468] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:33.653810 19075 raft_consensus.cc:740] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.678829 19134 raft_consensus.cc:2749] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.684971 19076 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:33.686574 18771 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:33.688172 19088 raft_consensus.cc:2804] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.688783 19088 raft_consensus.cc:697] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:33.689795 19088 consensus_queue.cc:237] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.691715 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcb4a0d3f9924af3881fae78b771264e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.693003 18793 tablet_service.cc:1511] Processing CreateTablet for tablet 0f439429f2a945c5b906f0a9e2d57e41 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:21:33.695582 19127 leader_election.cc:290] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:33.704283 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f439429f2a945c5b906f0a9e2d57e41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.705601 18942 tablet_service.cc:1511] Processing CreateTablet for tablet b7cb8b0da1be426bad1c6d04041a06db (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:33.683027 19073 raft_consensus.cc:359] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.707721 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:33.708293 19075 consensus_queue.cc:260] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:33.716517 18919 leader_election.cc:343] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.707831 19073 raft_consensus.cc:385] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.729460 19073 raft_consensus.cc:740] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.736603 18788 tablet_service.cc:1511] Processing CreateTablet for tablet b7cb8b0da1be426bad1c6d04041a06db (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:33.738431 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cb8b0da1be426bad1c6d04041a06db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.738685 19075 ts_tablet_manager.cc:1434] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.113s	user 0.010s	sys 0.001s
I20260709 13:21:33.741626 19127 raft_consensus.cc:493] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.742520 19127 raft_consensus.cc:515] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:33.747668 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:33.748824 18880 raft_consensus.cc:2468] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:33.749459 19087 consensus_queue.cc:1048] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.750679 18991 leader_election.cc:304] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:33.753539 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cb8b0da1be426bad1c6d04041a06db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.754815 19121 raft_consensus.cc:2804] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.755589 19121 raft_consensus.cc:697] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:33.755686 19013 tablet_service.cc:1511] Processing CreateTablet for tablet b7cb8b0da1be426bad1c6d04041a06db (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:21:33.757325 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cb8b0da1be426bad1c6d04041a06db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.756707 19121 consensus_queue.cc:237] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:33.759471 18919 leader_election.cc:343] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.731870 19073 consensus_queue.cc:260] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.764274 19075 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:33.773087 19073 ts_tablet_manager.cc:1434] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.117s	user 0.006s	sys 0.007s
I20260709 13:21:33.775337 19073 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:33.781569 19127 leader_election.cc:290] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:33.782541 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:33.784789 18917 leader_election.cc:343] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.785236 19074 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:33.785362 18917 leader_election.cc:304] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:33.786372 19074 ts_tablet_manager.cc:1403] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.444s	user 0.014s	sys 0.017s
I20260709 13:21:33.763550 19082 raft_consensus.cc:515] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.758543 19076 tablet_bootstrap.cc:654] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.787573 19087 consensus_queue.cc:1048] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:33.799885 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:33.807448 19127 raft_consensus.cc:2749] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:33.810447 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:33.811321 19082 leader_election.cc:290] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:33.811539 19087 raft_consensus.cc:493] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.823024 19027 raft_consensus.cc:3060] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.822813 19087 raft_consensus.cc:515] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:33.819510 19075 tablet_bootstrap.cc:654] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.825600 19087 leader_election.cc:290] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:33.834760 18919 leader_election.cc:343] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.836287 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:33.837214 18953 raft_consensus.cc:2468] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:33.837821 19073 tablet_bootstrap.cc:654] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:33.848737 18846 leader_election.cc:304] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:33.849871 19087 raft_consensus.cc:2804] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:33.850332 19087 raft_consensus.cc:493] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.850689 19087 raft_consensus.cc:3060] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:33.850898 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 0240263315db416dbece9ba6b467818d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:33.837865 19074 raft_consensus.cc:359] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.857982 19074 raft_consensus.cc:385] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:33.858385 19074 raft_consensus.cc:740] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:33.882069 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:33.885061 18791 tablet_service.cc:1511] Processing CreateTablet for tablet 232db8a92f07473e91159b6e4182f3ef (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:33.886838 18791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 232db8a92f07473e91159b6e4182f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.859337 19074 consensus_queue.cc:260] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.893049 19074 ts_tablet_manager.cc:1434] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.004s
I20260709 13:21:33.894508 19074 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:33.896557 18844 leader_election.cc:343] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:33.896849 18869 tablet_service.cc:1511] Processing CreateTablet for tablet 0240263315db416dbece9ba6b467818d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:33.899315 18703 catalog_manager.cc:5697] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:33.901546 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 232db8a92f07473e91159b6e4182f3ef (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:33.903616 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0240263315db416dbece9ba6b467818d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.905411 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 232db8a92f07473e91159b6e4182f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.898802 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 0240263315db416dbece9ba6b467818d (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:21:33.909757 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0240263315db416dbece9ba6b467818d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.910117 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0240263315db416dbece9ba6b467818d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.912051 19076 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:33.912734 19076 ts_tablet_manager.cc:1403] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.231s	user 0.020s	sys 0.006s
I20260709 13:21:33.920400 19027 raft_consensus.cc:2468] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:33.925582 18944 tablet_service.cc:1511] Processing CreateTablet for tablet 232db8a92f07473e91159b6e4182f3ef (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:21:33.927366 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 232db8a92f07473e91159b6e4182f3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.932624 18703 catalog_manager.cc:5697] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:33.949716 18918 leader_election.cc:304] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:33.950647 19018 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:33.951267 19127 raft_consensus.cc:2804] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:33.951817 19127 raft_consensus.cc:697] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:33.952266 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.952800 19127 consensus_queue.cc:237] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:33.922912 19087 raft_consensus.cc:515] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:33.987483 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 51362.7us
I20260709 13:21:33.993943 19073 tablet_bootstrap.cc:492] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:33.998462 19016 tablet_service.cc:1511] Processing CreateTablet for tablet dd6bf8d58b494607937dbeb92cdb2015 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:34.000056 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd6bf8d58b494607937dbeb92cdb2015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:33.999181 19073 ts_tablet_manager.cc:1403] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.224s	user 0.014s	sys 0.011s
I20260709 13:21:33.995721 19087 leader_election.cc:290] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:33.996016 18994 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.002868 19076 raft_consensus.cc:359] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:33.998417 19074 tablet_bootstrap.cc:654] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.006917 19076 raft_consensus.cc:385] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.007529 19076 raft_consensus.cc:740] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.006052 19073 raft_consensus.cc:359] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.008993 19073 raft_consensus.cc:385] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.012441 18944 tablet_service.cc:1511] Processing CreateTablet for tablet dd6bf8d58b494607937dbeb92cdb2015 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:34.014815 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd6bf8d58b494607937dbeb92cdb2015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:34.018774 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:34.021992 18793 tablet_service.cc:1511] Processing CreateTablet for tablet dd6bf8d58b494607937dbeb92cdb2015 (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:21:34.023653 18793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd6bf8d58b494607937dbeb92cdb2015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:34.023375 19076 consensus_queue.cc:260] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.016057 19073 raft_consensus.cc:740] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.027019 19075 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.027881 19075 ts_tablet_manager.cc:1403] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.264s	user 0.020s	sys 0.015s
I20260709 13:21:34.008378 18788 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:34.026522 19076 ts_tablet_manager.cc:1434] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.113s	user 0.008s	sys 0.000s
I20260709 13:21:34.032604 18788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:34.029985 18869 tablet_service.cc:1511] Processing CreateTablet for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf (DEFAULT_TABLE table=test-workload [id=f1b48fb17000443e9376aa106904bf29]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:21:34.042138 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b993ab3a9c34cada69f06e7bfc4f7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:34.032624 19076 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.053475 19075 raft_consensus.cc:359] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.053725 19073 consensus_queue.cc:260] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.064512 19073 ts_tablet_manager.cc:1434] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.064s	user 0.005s	sys 0.003s
I20260709 13:21:34.067657 19075 raft_consensus.cc:385] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.068212 19075 raft_consensus.cc:740] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.055874 19083 consensus_queue.cc:1048] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.044031 19145 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.069262 19027 raft_consensus.cc:3060] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.073967 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e3966983799476b9516092542b22438" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:34.074783 18953 raft_consensus.cc:3060] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.075368 19075 consensus_queue.cc:260] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.077504 19076 tablet_bootstrap.cc:654] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.078799 19075 ts_tablet_manager.cc:1434] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.004s
I20260709 13:21:34.087803 19075 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.092360 18703 catalog_manager.cc:5697] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.097083 19073 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.117113 19074 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:34.117928 19074 ts_tablet_manager.cc:1403] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.224s	user 0.012s	sys 0.011s
I20260709 13:21:34.089884 19145 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.121716 19074 raft_consensus.cc:359] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.123106 19074 raft_consensus.cc:385] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.124755 19027 raft_consensus.cc:2468] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:34.137411 19073 tablet_bootstrap.cc:654] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.138374 19145 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:34.138824 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:34.140892 18845 leader_election.cc:343] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.138553 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:34.144446 18846 leader_election.cc:343] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.145337 18846 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:34.126431 19074 raft_consensus.cc:740] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.148995 19075 tablet_bootstrap.cc:654] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.150451 19136 raft_consensus.cc:2749] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.157506 18844 leader_election.cc:304] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:34.157755 19028 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.159022 19136 raft_consensus.cc:2804] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.160045 19136 raft_consensus.cc:697] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:34.161171 19136 consensus_queue.cc:237] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.162463 19088 raft_consensus.cc:493] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.163070 19088 raft_consensus.cc:515] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.148253 19074 consensus_queue.cc:260] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.166627 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:34.167340 19088 leader_election.cc:290] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:34.173187 19074 ts_tablet_manager.cc:1434] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.053s	user 0.011s	sys 0.000s
I20260709 13:21:34.177476 19074 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:34.181974 19076 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:34.182718 19076 ts_tablet_manager.cc:1403] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.022s
I20260709 13:21:34.194336 19073 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.195016 19073 ts_tablet_manager.cc:1403] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.098s	user 0.026s	sys 0.020s
I20260709 13:21:34.195101 18953 raft_consensus.cc:2468] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:34.197697 19145 raft_consensus.cc:493] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.198328 19145 raft_consensus.cc:515] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.201309 19145 leader_election.cc:290] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:34.196903 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3e819f8fc7440af86dd0b3190f1edf3" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:34.202699 19027 raft_consensus.cc:2393] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 878e795a8488495f978538952e7b2324 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:34.203586 19088 consensus_queue.cc:1048] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:34.205785 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:34.206804 19073 raft_consensus.cc:359] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.206593 18769 leader_election.cc:304] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:34.207692 19073 raft_consensus.cc:385] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.208089 19073 raft_consensus.cc:740] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.209221 19130 raft_consensus.cc:3060] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.208937 19073 consensus_queue.cc:260] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.202584 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.214109 19074 tablet_bootstrap.cc:654] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.217643 19076 raft_consensus.cc:359] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.218669 19076 raft_consensus.cc:385] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.219048 19073 ts_tablet_manager.cc:1434] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
W20260709 13:21:34.219214 18845 leader_election.cc:343] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.219075 19076 raft_consensus.cc:740] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.221858 19149 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.222486 19149 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.224543 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:34.225834 19028 raft_consensus.cc:2468] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:34.227886 18844 leader_election.cc:304] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:34.229192 19145 raft_consensus.cc:2804] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.229604 19145 raft_consensus.cc:493] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.230031 19145 raft_consensus.cc:3060] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.230244 19073 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.230724 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.233194 19088 consensus_queue.cc:1048] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.240621 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:34.245311 18994 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:34.246521 19130 raft_consensus.cc:2749] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.238956 19149 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:34.247617 19121 consensus_queue.cc:1048] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.252127 19076 consensus_queue.cc:260] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.263736 19073 tablet_bootstrap.cc:654] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.269780 19076 ts_tablet_manager.cc:1434] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.002s
I20260709 13:21:34.276785 19076 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.284634 19075 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.285681 19075 ts_tablet_manager.cc:1403] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.206s	user 0.010s	sys 0.011s
I20260709 13:21:34.289261 18700 catalog_manager.cc:5697] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.302557 19083 raft_consensus.cc:2749] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:34.307205 19130 raft_consensus.cc:493] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.310146 19075 raft_consensus.cc:359] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.310842 19075 raft_consensus.cc:385] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.311154 19075 raft_consensus.cc:740] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.312319 19075 consensus_queue.cc:260] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.314253 19075 ts_tablet_manager.cc:1434] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:21:34.315883 19075 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.315730 19145 raft_consensus.cc:515] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.308130 19130 raft_consensus.cc:515] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.318812 19145 leader_election.cc:290] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:34.320442 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
W20260709 13:21:34.322229 18845 leader_election.cc:343] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.323342 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.324503 19130 leader_election.cc:290] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:34.325489 19075 tablet_bootstrap.cc:654] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:34.326148 18768 leader_election.cc:343] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.324920 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:34.328595 18771 leader_election.cc:343] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.329178 18771 leader_election.cc:304] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:34.330695 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:34.332018 19130 raft_consensus.cc:2749] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.335269 19074 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:34.332773 19027 raft_consensus.cc:3060] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.347000 19149 consensus_queue.cc:1048] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.355988 19074 ts_tablet_manager.cc:1403] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.179s	user 0.025s	sys 0.017s
I20260709 13:21:34.367585 18952 raft_consensus.cc:3060] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.378759 19076 tablet_bootstrap.cc:654] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.379846 19073 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.382347 19073 ts_tablet_manager.cc:1403] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.155s	user 0.027s	sys 0.002s
I20260709 13:21:34.382998 19027 raft_consensus.cc:2468] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:34.385492 18844 leader_election.cc:304] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:34.389747 19073 raft_consensus.cc:359] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.390722 19073 raft_consensus.cc:385] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.390766 19145 raft_consensus.cc:2804] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.392593 19073 raft_consensus.cc:740] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.392949 19145 raft_consensus.cc:697] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:34.393720 19073 consensus_queue.cc:260] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.392285 19074 raft_consensus.cc:359] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.395627 19074 raft_consensus.cc:385] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.396040 19074 raft_consensus.cc:740] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.397195 19074 consensus_queue.cc:260] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.394078 19145 consensus_queue.cc:237] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.404099 19073 ts_tablet_manager.cc:1434] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:21:34.410226 19075 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.402845 19074 ts_tablet_manager.cc:1434] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.002s
I20260709 13:21:34.411432 19075 ts_tablet_manager.cc:1403] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.096s	user 0.035s	sys 0.021s
I20260709 13:21:34.412163 19074 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:34.415725 19073 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.416947 19075 raft_consensus.cc:359] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.418007 19075 raft_consensus.cc:385] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.418396 19075 raft_consensus.cc:740] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.419361 19075 consensus_queue.cc:260] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.423962 19073 tablet_bootstrap.cc:654] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.430364 19075 ts_tablet_manager.cc:1434] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:21:34.431761 19075 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.434506 19127 consensus_queue.cc:1048] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.438365 19074 tablet_bootstrap.cc:654] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.440604 18703 catalog_manager.cc:5697] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.457345 19149 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.458041 19149 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.461773 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:34.464039 18991 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.465543 19149 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:34.467468 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:34.470034 18994 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.470646 18994 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:34.472543 19075 tablet_bootstrap.cc:654] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.473796 19076 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:34.474434 19076 ts_tablet_manager.cc:1403] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.198s	user 0.023s	sys 0.006s
I20260709 13:21:34.475709 19149 raft_consensus.cc:2749] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.478770 19076 raft_consensus.cc:359] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.479094 19082 consensus_queue.cc:1048] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:34.482774 19076 raft_consensus.cc:385] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.483310 19076 raft_consensus.cc:740] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.485572 19076 consensus_queue.cc:260] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.500221 19076 ts_tablet_manager.cc:1434] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:21:34.501811 19076 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.515641 19074 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:34.516317 19074 ts_tablet_manager.cc:1403] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.001s
I20260709 13:21:34.519479 19074 raft_consensus.cc:359] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.520295 19074 raft_consensus.cc:385] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.520624 19074 raft_consensus.cc:740] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.521376 19074 consensus_queue.cc:260] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.525336 19073 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.526090 19073 ts_tablet_manager.cc:1403] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.111s	user 0.021s	sys 0.047s
I20260709 13:21:34.529971 19073 raft_consensus.cc:359] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.530344 19074 ts_tablet_manager.cc:1434] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:21:34.530767 19073 raft_consensus.cc:385] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.532341 19075 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.535363 19075 ts_tablet_manager.cc:1403] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.104s	user 0.021s	sys 0.003s
I20260709 13:21:34.534519 19074 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:34.533865 19073 raft_consensus.cc:740] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.538388 19073 consensus_queue.cc:260] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.539747 19075 raft_consensus.cc:359] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.540584 19073 ts_tablet_manager.cc:1434] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:21:34.540802 19075 raft_consensus.cc:385] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.541380 19149 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.541589 19075 raft_consensus.cc:740] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.542042 19149 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.542780 19075 consensus_queue.cc:260] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.543591 19082 raft_consensus.cc:493] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.544189 19082 raft_consensus.cc:515] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.545480 19075 ts_tablet_manager.cc:1434] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:21:34.546700 19075 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.535297 19076 tablet_bootstrap.cc:654] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.548010 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.550398 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.548067 19073 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.551858 19082 leader_election.cc:290] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.552953 19127 raft_consensus.cc:493] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.561388 19127 raft_consensus.cc:515] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.553910 19075 tablet_bootstrap.cc:654] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.564303 19073 tablet_bootstrap.cc:654] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.565544 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.566425 18805 raft_consensus.cc:2468] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:34.568044 18919 leader_election.cc:304] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
W20260709 13:21:34.554530 18917 leader_election.cc:343] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.571547 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.554951 19149 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:34.555678 18993 leader_election.cc:343] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.555753 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.575241 19076 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:34.576000 19076 ts_tablet_manager.cc:1403] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.004s
W20260709 13:21:34.576542 18991 leader_election.cc:343] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.577454 18991 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:34.579509 19149 raft_consensus.cc:2749] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.579752 19076 raft_consensus.cc:359] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.581565 19076 raft_consensus.cc:385] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:21:34.581219 18917 leader_election.cc:343] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.582108 19076 raft_consensus.cc:740] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.582827 19088 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.560154 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.583686 19088 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.584626 18804 raft_consensus.cc:2468] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:34.563792 19127 leader_election.cc:290] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.582957 19082 raft_consensus.cc:2804] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.586663 19082 raft_consensus.cc:493] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.587145 19082 raft_consensus.cc:3060] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.587312 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:34.588130 18953 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.588933 18919 leader_election.cc:304] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:34.590713 19127 raft_consensus.cc:2804] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.591212 19073 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.546963 19074 tablet_bootstrap.cc:654] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.591406 18771 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:34.591984 19073 ts_tablet_manager.cc:1403] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.004s
I20260709 13:21:34.592803 19163 raft_consensus.cc:2804] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.593279 19163 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.593631 19144 raft_consensus.cc:493] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.594180 19144 raft_consensus.cc:515] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.591208 19127 raft_consensus.cc:493] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.595348 19127 raft_consensus.cc:3060] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.595803 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.595857 19073 raft_consensus.cc:359] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.597030 19073 raft_consensus.cc:385] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.597563 19073 raft_consensus.cc:740] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.596926 18880 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.593667 19163 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.598860 19073 consensus_queue.cc:260] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.599956 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:34.600515 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.601815 19073 ts_tablet_manager.cc:1434] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
W20260709 13:21:34.602039 18846 leader_election.cc:343] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.603225 19088 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.603433 19073 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.607139 19149 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.607589 19074 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:34.607705 19149 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.608367 19074 ts_tablet_manager.cc:1403] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.006s
I20260709 13:21:34.610549 19127 raft_consensus.cc:515] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.611856 19074 raft_consensus.cc:359] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.612568 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.607501 19075 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.613608 19075 ts_tablet_manager.cc:1403] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.008s
I20260709 13:21:34.614169 19149 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.615477 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:34.615942 18991 leader_election.cc:343] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.617959 19144 leader_election.cc:290] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:34.622126 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.623340 19082 raft_consensus.cc:515] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:34.624544 18993 leader_election.cc:343] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.625126 18993 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:34.625998 19082 leader_election.cc:290] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.583534 19076 consensus_queue.cc:260] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.627480 18880 raft_consensus.cc:3060] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.628579 19076 ts_tablet_manager.cc:1434] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:21:34.612884 19074 raft_consensus.cc:385] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:21:34.616957 18845 leader_election.cc:343] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.629453 19074 raft_consensus.cc:740] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.629734 18845 leader_election.cc:304] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:34.616403 19073 tablet_bootstrap.cc:654] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.617944 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45bd3ea063de41839fe0187978e00474" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:34.618086 19075 raft_consensus.cc:359] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.631556 19075 raft_consensus.cc:385] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.630578 19074 consensus_queue.cc:260] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.631601 18805 raft_consensus.cc:3060] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.627668 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:34.633766 18917 leader_election.cc:343] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.634572 19144 raft_consensus.cc:2749] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.627164 19149 raft_consensus.cc:2749] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:34.635695 19076 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.637898 19074 ts_tablet_manager.cc:1434] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:21:34.639212 19074 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:34.644924 19076 tablet_bootstrap.cc:654] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.646497 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:34.648221 18804 raft_consensus.cc:3060] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.647950 19145 consensus_queue.cc:1048] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.631927 19075 raft_consensus.cc:740] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.616135 19127 leader_election.cc:290] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.651957 19075 consensus_queue.cc:260] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.652483 19074 tablet_bootstrap.cc:654] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.653748 19145 raft_consensus.cc:493] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.654634 19075 ts_tablet_manager.cc:1434] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:21:34.654351 18880 raft_consensus.cc:2468] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:34.654374 19145 raft_consensus.cc:515] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.659349 19075 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.661608 19136 consensus_queue.cc:1048] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:34.666610 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.667788 18803 raft_consensus.cc:2468] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:34.669732 18845 leader_election.cc:304] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:34.673261 19145 leader_election.cc:290] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:34.673579 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:34.675132 18952 raft_consensus.cc:2468] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:34.628784 19163 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.674000 19136 raft_consensus.cc:2804] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.678937 19136 raft_consensus.cc:493] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.666517 18804 raft_consensus.cc:2468] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:34.679850 19136 raft_consensus.cc:3060] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.680951 18919 leader_election.cc:304] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:34.681649 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:34.682428 18952 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.683138 19075 tablet_bootstrap.cc:654] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.687498 19127 raft_consensus.cc:2804] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.691486 19127 raft_consensus.cc:697] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:34.693254 19127 consensus_queue.cc:237] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.701246 18805 raft_consensus.cc:2468] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:34.704420 19163 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.706300 18917 leader_election.cc:304] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:34.720510 19082 raft_consensus.cc:2804] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.723924 19075 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.724664 19075 ts_tablet_manager.cc:1403] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.003s
I20260709 13:21:34.709357 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:34.726266 18952 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:34.726800 18880 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.727622 19073 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.752385 19073 ts_tablet_manager.cc:1403] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.022s
I20260709 13:21:34.756223 19073 raft_consensus.cc:359] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.757144 19073 raft_consensus.cc:385] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.757503 19073 raft_consensus.cc:740] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.730892 19076 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:34.733314 19088 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.758621 19088 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.760708 18880 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:34.762995 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.763283 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:34.737840 19082 raft_consensus.cc:697] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:34.733090 19075 raft_consensus.cc:359] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.764387 19076 ts_tablet_manager.cc:1403] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.129s	user 0.017s	sys 0.005s
I20260709 13:21:34.764664 19075 raft_consensus.cc:385] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.764992 19075 raft_consensus.cc:740] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.764876 19082 consensus_queue.cc:237] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.766008 19075 consensus_queue.cc:260] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:34.761497 19088 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.762403 18771 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:34.767855 19076 raft_consensus.cc:359] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.768854 19076 raft_consensus.cc:385] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.769173 19088 raft_consensus.cc:2804] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.769282 19076 raft_consensus.cc:740] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.769902 19088 raft_consensus.cc:697] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:34.770598 19076 consensus_queue.cc:260] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.763880 18879 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.771096 19075 ts_tablet_manager.cc:1434] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:21:34.770915 19088 consensus_queue.cc:237] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.772997 19073 consensus_queue.cc:260] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.773941 18768 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:34.774370 19076 ts_tablet_manager.cc:1434] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:21:34.776448 19027 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.776984 19076 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.774869 19075 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.780514 19163 raft_consensus.cc:2804] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.781145 19163 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.760462 19136 raft_consensus.cc:515] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.781560 19163 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.786574 18702 catalog_manager.cc:5697] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.797465 19136 leader_election.cc:290] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:34.798255 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:34.798987 18952 raft_consensus.cc:3060] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.804632 19163 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.805840 19076 tablet_bootstrap.cc:654] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.805009 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d217d9f5218640ed95be5641ff1fb098" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:34.806646 18805 raft_consensus.cc:3060] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.809446 19075 tablet_bootstrap.cc:654] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.810458 19074 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:34.811189 19074 ts_tablet_manager.cc:1403] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.172s	user 0.065s	sys 0.029s
I20260709 13:21:34.812529 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:34.813380 19027 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.812831 19073 ts_tablet_manager.cc:1434] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.002s
I20260709 13:21:34.816375 19073 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:34.817762 19163 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.824414 18703 catalog_manager.cc:5697] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.820160 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:34.827582 18879 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.844892 18879 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:34.847656 18768 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:34.848085 18952 raft_consensus.cc:2468] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:34.849026 19163 raft_consensus.cc:2804] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.849548 18846 leader_election.cc:304] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:34.849970 19163 raft_consensus.cc:697] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:34.850608 19136 raft_consensus.cc:2804] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:34.851220 19136 raft_consensus.cc:697] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:34.851200 19163 consensus_queue.cc:237] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.851905 19027 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:34.857610 18700 catalog_manager.cc:5697] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.862654 19082 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.862850 19074 raft_consensus.cc:359] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.863860 19074 raft_consensus.cc:385] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.863519 19082 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.861215 19136 consensus_queue.cc:237] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.864245 19074 raft_consensus.cc:740] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.871452 19144 consensus_queue.cc:1048] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:21:34.875723 19076 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:34.876494 19076 ts_tablet_manager.cc:1403] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.100s	user 0.017s	sys 0.002s
I20260709 13:21:34.880415 19073 tablet_bootstrap.cc:654] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.880290 19076 raft_consensus.cc:359] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.883700 19076 raft_consensus.cc:385] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.885578 19076 raft_consensus.cc:740] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
W20260709 13:21:34.886413 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.886644 19076 consensus_queue.cc:260] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.888029 19074 consensus_queue.cc:260] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.893364 18805 raft_consensus.cc:2468] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:34.893937 18702 catalog_manager.cc:5697] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.901247 19082 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:34.908526 19076 ts_tablet_manager.cc:1434] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:21:34.910528 19076 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:34.908033 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:34.914746 18917 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.915668 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:34.916565 19028 raft_consensus.cc:2468] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:34.917354 19075 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:34.918056 19075 ts_tablet_manager.cc:1403] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.146s	user 0.013s	sys 0.008s
I20260709 13:21:34.918429 18918 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:34.920004 19082 raft_consensus.cc:2804] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.920498 19082 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.920869 19082 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.928040 19074 ts_tablet_manager.cc:1434] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.117s	user 0.009s	sys 0.005s
I20260709 13:21:34.929498 19163 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.932749 19074 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:34.932128 19075 raft_consensus.cc:359] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.933887 19075 raft_consensus.cc:385] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:34.934306 19075 raft_consensus.cc:740] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:34.935316 19075 consensus_queue.cc:260] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:34.934547 19163 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.939817 19163 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.941144 19083 raft_consensus.cc:493] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.941614 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:34.941802 19083 raft_consensus.cc:515] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.942092 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.942376 19028 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.942986 18879 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:34.943826 18769 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:34.944288 19083 leader_election.cc:290] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:34.949510 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:34.949918 19075 ts_tablet_manager.cc:1434] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.031s	user 0.012s	sys 0.007s
I20260709 13:21:34.950479 18805 raft_consensus.cc:2468] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:34.951354 19163 raft_consensus.cc:2804] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.951886 19163 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.952652 18993 leader_election.cc:304] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:34.952666 19163 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.953832 19083 raft_consensus.cc:2804] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:34.954270 19083 raft_consensus.cc:493] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.954457 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:34.954751 19083 raft_consensus.cc:3060] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:34.956444 18991 leader_election.cc:343] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.954839 19082 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:34.951862 19075 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:34.963788 19074 tablet_bootstrap.cc:654] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.965318 19163 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:34.969452 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:34.970296 19028 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.973647 19075 tablet_bootstrap.cc:654] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.973488 19076 tablet_bootstrap.cc:654] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:34.979759 18703 catalog_manager.cc:5697] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.982729 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:34.983875 19082 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:34.984503 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
W20260709 13:21:34.984822 18917 leader_election.cc:343] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:34.985224 19026 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:34.997659 19073 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:34.998458 19073 ts_tablet_manager.cc:1403] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.005s
I20260709 13:21:35.002264 19073 raft_consensus.cc:359] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.003371 19073 raft_consensus.cc:385] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.003594 19073 raft_consensus.cc:740] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.006635 19083 raft_consensus.cc:515] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.009353 19083 leader_election.cc:290] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:35.010906 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:35.015048 19074 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:35.016834 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:35.022825 19074 ts_tablet_manager.cc:1403] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.010s
I20260709 13:21:35.004346 19163 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:35.026415 19073 consensus_queue.cc:260] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:35.029767 18991 leader_election.cc:343] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.012562 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:35.032979 18879 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.026110 18805 raft_consensus.cc:3060] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.037472 19026 raft_consensus.cc:2468] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:35.039208 18918 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:35.040699 19082 raft_consensus.cc:2804] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.040846 18879 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:35.041432 19082 raft_consensus.cc:697] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:35.042622 18768 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:35.042529 19082 consensus_queue.cc:237] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.048231 19073 ts_tablet_manager.cc:1434] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.003s
I20260709 13:21:35.011569 19028 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:35.048681 19163 raft_consensus.cc:2804] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.051281 19073 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:35.051489 19163 raft_consensus.cc:697] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:35.054091 19163 consensus_queue.cc:237] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.058415 19075 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:35.044556 18805 raft_consensus.cc:2468] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:35.059087 19075 ts_tablet_manager.cc:1403] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.108s	user 0.014s	sys 0.010s
I20260709 13:21:35.065217 18993 leader_election.cc:304] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:35.069047 19074 raft_consensus.cc:359] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.069998 19074 raft_consensus.cc:385] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.070375 19074 raft_consensus.cc:740] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.071501 19083 raft_consensus.cc:2804] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.072173 19075 raft_consensus.cc:359] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.073274 19075 raft_consensus.cc:385] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.073675 19075 raft_consensus.cc:740] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.074585 19075 consensus_queue.cc:260] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.072716 19083 raft_consensus.cc:697] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:35.074965 19073 tablet_bootstrap.cc:654] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.076488 19083 consensus_queue.cc:237] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.081238 19074 consensus_queue.cc:260] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.084108 19074 ts_tablet_manager.cc:1434] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:21:35.085582 19074 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:35.092015 19076 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:35.092795 19076 ts_tablet_manager.cc:1403] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.183s	user 0.015s	sys 0.000s
I20260709 13:21:35.101070 19074 tablet_bootstrap.cc:654] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.105438 19073 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:35.106348 19073 ts_tablet_manager.cc:1403] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.008s
I20260709 13:21:35.110054 19073 raft_consensus.cc:359] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.113696 18702 catalog_manager.cc:5697] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.117635 19075 ts_tablet_manager.cc:1434] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.001s
I20260709 13:21:35.118927 19075 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:35.110850 19073 raft_consensus.cc:385] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.126515 19073 raft_consensus.cc:740] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.120777 19076 raft_consensus.cc:359] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.127744 19076 raft_consensus.cc:385] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.128098 19076 raft_consensus.cc:740] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.134423 19073 consensus_queue.cc:260] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.137396 19073 ts_tablet_manager.cc:1434] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:21:35.139015 18700 catalog_manager.cc:5697] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.141113 18702 catalog_manager.cc:5697] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.128979 19076 consensus_queue.cc:260] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.150070 19076 ts_tablet_manager.cc:1434] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.003s
I20260709 13:21:35.151829 19073 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:35.157522 19076 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:35.159937 19163 raft_consensus.cc:493] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.161098 19075 tablet_bootstrap.cc:654] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.160676 19163 raft_consensus.cc:515] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.163189 19082 raft_consensus.cc:493] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.163887 19082 raft_consensus.cc:515] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.164515 19088 consensus_queue.cc:1048] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.166747 19074 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:35.167529 19074 ts_tablet_manager.cc:1403] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.082s	user 0.030s	sys 0.030s
I20260709 13:21:35.167927 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4832e091fcef44ab877283bf9a254752" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:35.168798 19028 raft_consensus.cc:2468] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:35.171165 19074 raft_consensus.cc:359] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.172003 19074 raft_consensus.cc:385] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.172294 19074 raft_consensus.cc:740] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.173237 19074 consensus_queue.cc:260] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.175426 19083 raft_consensus.cc:493] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.176157 19083 raft_consensus.cc:515] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.176558 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4832e091fcef44ab877283bf9a254752" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:35.178653 19083 leader_election.cc:290] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:35.166735 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:35.180794 19082 leader_election.cc:290] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.189774 19074 ts_tablet_manager.cc:1434] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.022s	user 0.013s	sys 0.005s
I20260709 13:21:35.191608 19074 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:35.196476 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:35.197559 18953 raft_consensus.cc:2468] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:35.200435 19076 tablet_bootstrap.cc:654] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.201169 19074 tablet_bootstrap.cc:654] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.204819 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:35.208365 18991 leader_election.cc:343] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.187235 18918 leader_election.cc:304] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:35.211169 18994 leader_election.cc:304] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:35.210863 19082 raft_consensus.cc:2804] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.212641 19163 leader_election.cc:290] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:35.214111 19083 raft_consensus.cc:2804] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.215780 19083 raft_consensus.cc:493] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.216181 19083 raft_consensus.cc:3060] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:35.216367 18771 leader_election.cc:343] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.213960 19082 raft_consensus.cc:493] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:35.213587 18919 leader_election.cc:343] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.218698 19082 raft_consensus.cc:3060] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.232435 19075 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:35.230640 19145 raft_consensus.cc:493] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.233451 19075 ts_tablet_manager.cc:1403] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.004s
I20260709 13:21:35.234040 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:35.237226 18769 leader_election.cc:343] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.237861 18769 leader_election.cc:304] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:35.237905 19145 raft_consensus.cc:515] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:35.238332 19075 raft_consensus.cc:359] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.239537 19075 raft_consensus.cc:385] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.239857 19075 raft_consensus.cc:740] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.240731 19130 raft_consensus.cc:2749] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:35.240643 19075 consensus_queue.cc:260] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.246644 19075 ts_tablet_manager.cc:1434] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:21:35.242673 19088 consensus_queue.cc:1048] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:35.244355 19145 leader_election.cc:290] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:35.252548 19074 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:35.253202 19074 ts_tablet_manager.cc:1403] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.062s	user 0.018s	sys 0.000s
I20260709 13:21:35.253870 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:35.262425 19083 raft_consensus.cc:515] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.265683 18879 raft_consensus.cc:3060] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:35.265689 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 79920.8us
I20260709 13:21:35.267385 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:35.267663 19184 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.264601 19082 raft_consensus.cc:515] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:35.269405 18991 leader_election.cc:343] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.272348 19083 leader_election.cc:290] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:35.272352 19082 leader_election.cc:290] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.268340 19184 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:35.276494 19184 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:35.277899 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4832e091fcef44ab877283bf9a254752" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:35.276656 19073 tablet_bootstrap.cc:654] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.278455 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:35.278679 19028 raft_consensus.cc:3060] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:35.280408 18917 leader_election.cc:343] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.282954 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:35.283895 18953 raft_consensus.cc:3060] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.283982 19075 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:35.278497 19076 tablet_bootstrap.cc:492] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:35.287927 19076 ts_tablet_manager.cc:1403] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.132s	user 0.024s	sys 0.003s
I20260709 13:21:35.289569 19028 raft_consensus.cc:2468] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:35.290838 18918 leader_election.cc:304] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:35.291987 19184 raft_consensus.cc:2804] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.292567 19184 raft_consensus.cc:697] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:35.292471 19076 raft_consensus.cc:359] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.293815 19076 raft_consensus.cc:385] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.294376 19076 raft_consensus.cc:740] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.293906 19184 consensus_queue.cc:237] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.298322 18953 raft_consensus.cc:2468] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:35.302181 18994 leader_election.cc:304] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:35.302816 19073 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:35.303370 19083 raft_consensus.cc:2804] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:35.304013 19083 raft_consensus.cc:697] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:35.303721 19073 ts_tablet_manager.cc:1403] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.152s	user 0.017s	sys 0.000s
I20260709 13:21:35.304862 19083 consensus_queue.cc:237] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.311301 19075 tablet_bootstrap.cc:654] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.315724 19076 consensus_queue.cc:260] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.326191 19073 raft_consensus.cc:359] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.327143 19073 raft_consensus.cc:385] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.327466 19073 raft_consensus.cc:740] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.328365 19073 consensus_queue.cc:260] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.247988 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13175817e62e49c38eafca15e3877bc6" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:35.392201 18805 raft_consensus.cc:2393] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:35.398916 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4832e091fcef44ab877283bf9a254752" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:35.404613 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:35.408506 19073 ts_tablet_manager.cc:1434] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.104s	user 0.011s	sys 0.000s
I20260709 13:21:35.411384 19127 raft_consensus.cc:493] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.438084 19127 raft_consensus.cc:515] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.449077 18845 leader_election.cc:304] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:35.445693 19076 ts_tablet_manager.cc:1434] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.157s	user 0.013s	sys 0.001s
I20260709 13:21:35.457620 19073 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:35.464205 19074 raft_consensus.cc:359] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.465572 19074 raft_consensus.cc:385] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.466157 19145 raft_consensus.cc:2749] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:35.467387 18919 leader_election.cc:343] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.468302 19144 consensus_queue.cc:1048] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:35.474912 19127 leader_election.cc:290] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.468384 19136 raft_consensus.cc:493] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.476263 19076 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:35.476987 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:35.471053 19074 raft_consensus.cc:740] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.493994 19184 consensus_queue.cc:1048] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.505911 19083 raft_consensus.cc:493] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.492412 19074 consensus_queue.cc:260] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.531461 19082 consensus_queue.cc:1048] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.531361 19083 raft_consensus.cc:515] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.476392 19136 raft_consensus.cc:515] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.547499 19130 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.548324 19130 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.535312 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:35.551738 18803 raft_consensus.cc:2468] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:35.551937 19130 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:35.472010 18768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 45759.1us
I20260709 13:21:35.539690 19075 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:35.554235 19075 ts_tablet_manager.cc:1403] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.271s	user 0.025s	sys 0.021s
I20260709 13:21:35.557641 19212 raft_consensus.cc:493] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.541929 19127 consensus_queue.cc:1048] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.558650 19212 raft_consensus.cc:515] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.557335 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:35.561715 19205 consensus_queue.cc:1048] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.563941 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:35.565532 19026 raft_consensus.cc:2468] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:35.546510 19074 ts_tablet_manager.cc:1434] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.293s	user 0.002s	sys 0.013s
I20260709 13:21:35.569644 19074 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:35.570695 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:35.610558 19076 tablet_bootstrap.cc:654] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.623101 19130 consensus_queue.cc:1048] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.627347 19212 leader_election.cc:290] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:35.631645 19027 raft_consensus.cc:2468] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:35.643878 19073 tablet_bootstrap.cc:654] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:35.617723 18919 leader_election.cc:343] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.645323 18919 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:35.645478 18769 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:35.646469 19127 raft_consensus.cc:2749] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:35.646548 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 45845.5us
W20260709 13:21:35.647315 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 122958us
I20260709 13:21:35.648515 19205 raft_consensus.cc:2804] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.646237 19217 raft_consensus.cc:493] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:35.650457 18768 leader_election.cc:343] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.655676 19127 consensus_queue.cc:1048] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:35.657476 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:35.650806 19205 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.664417 19205 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.667888 19217 raft_consensus.cc:515] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.678133 19082 consensus_queue.cc:1048] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:35.678851 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:35.679874 18919 leader_election.cc:304] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:35.681267 19214 raft_consensus.cc:2804] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.681838 19214 raft_consensus.cc:493] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.682274 19214 raft_consensus.cc:3060] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.626534 19075 raft_consensus.cc:359] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.695937 19075 raft_consensus.cc:385] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.696558 19075 raft_consensus.cc:740] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.652452 19212 consensus_queue.cc:1048] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:21:35.707427 19075 consensus_queue.cc:260] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.716001 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:35.716398 19149 consensus_queue.cc:1048] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:35.717855 18917 consensus_peers.cc:597] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:35.713382 18803 raft_consensus.cc:3060] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.707369 19083 leader_election.cc:290] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.712019 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6dc24978cc43a79db989f59bfed38a" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:35.728710 18768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 105430us
I20260709 13:21:35.773129 19217 leader_election.cc:290] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:35.774046 19217 consensus_queue.cc:1048] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:35.776082 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03fbc43806c04306990613e14c199ff1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:35.778160 18805 raft_consensus.cc:2393] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:35.780675 19136 leader_election.cc:290] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.764009 19212 consensus_queue.cc:1048] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
W20260709 13:21:35.776284 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 93251.2us
W20260709 13:21:35.788553 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 47220.3us
I20260709 13:21:35.791690 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:35.792140 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:35.792740 18805 raft_consensus.cc:2468] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:35.793135 18952 raft_consensus.cc:2393] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:35.795941 18768 leader_election.cc:304] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:35.798211 19074 tablet_bootstrap.cc:654] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:35.799904 18993 leader_election.cc:304] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:35.800954 19083 raft_consensus.cc:2804] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:35.801501 19083 raft_consensus.cc:493] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.801868 19083 raft_consensus.cc:3060] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.805394 19214 raft_consensus.cc:515] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:35.839469 18844 leader_election.cc:304] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:35.840981 19076 tablet_bootstrap.cc:492] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:35.841383 18878 raft_consensus.cc:3060] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.841763 19076 ts_tablet_manager.cc:1403] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.366s	user 0.031s	sys 0.046s
I20260709 13:21:35.848224 18876 raft_consensus.cc:3060] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:35.851867 18768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 56047.7us
I20260709 13:21:35.804665 19205 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:35.857197 19205 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:35.881570 19214 leader_election.cc:290] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:35.881747 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:35.884394 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:35.885260 19027 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.893518 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4a64164017441fd8ce325d46dcbbd07" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:35.893930 18768 leader_election.cc:343] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:35.896929 19075 ts_tablet_manager.cc:1434] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.342s	user 0.020s	sys 0.009s
I20260709 13:21:35.891698 18954 raft_consensus.cc:2393] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:35.913538 19076 raft_consensus.cc:359] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.914515 19076 raft_consensus.cc:385] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.914844 19076 raft_consensus.cc:740] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.922755 19075 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:35.936209 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:35.937395 18800 raft_consensus.cc:3060] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:35.949425 19076 consensus_queue.cc:260] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.956600 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:35.960657 19127 raft_consensus.cc:493] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.961942 19073 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:35.961524 19127 raft_consensus.cc:515] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:35.963008 19073 ts_tablet_manager.cc:1403] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.506s	user 0.027s	sys 0.017s
I20260709 13:21:35.974992 19074 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:35.975854 19074 ts_tablet_manager.cc:1403] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.407s	user 0.020s	sys 0.015s
I20260709 13:21:35.971441 18703 catalog_manager.cc:5697] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:35.979400 19228 log.cc:927] Time spent T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:21:35.981067 19076 ts_tablet_manager.cc:1434] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.139s	user 0.008s	sys 0.004s
I20260709 13:21:35.982698 19076 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:35.950935 18844 leader_election.cc:304] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:35.988476 19223 raft_consensus.cc:2749] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:35.988566 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 47137.9us
I20260709 13:21:35.993966 19074 raft_consensus.cc:359] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:35.994937 19074 raft_consensus.cc:385] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:35.995291 19074 raft_consensus.cc:740] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:35.996096 19074 consensus_queue.cc:260] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:35.996378 19127 leader_election.cc:290] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:35.996980 19027 raft_consensus.cc:2468] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:36.007575 19224 consensus_queue.cc:1048] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.009367 19073 raft_consensus.cc:359] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.010646 19073 raft_consensus.cc:385] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.011086 19073 raft_consensus.cc:740] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.012277 19073 consensus_queue.cc:260] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.014659 19075 tablet_bootstrap.cc:654] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.016513 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.017489 19214 consensus_queue.cc:1048] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.018476 18769 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:36.020115 19233 raft_consensus.cc:2749] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.021427 19088 raft_consensus.cc:2804] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.022378 19088 raft_consensus.cc:697] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:36.026087 19083 raft_consensus.cc:515] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.027720 19024 raft_consensus.cc:2393] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de62ff9486f7483e8f0e0b93dcca24bb in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:36.023969 19088 consensus_queue.cc:237] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.030725 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
W20260709 13:21:36.034999 18918 leader_election.cc:343] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.037281 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d350b38546104d82881fdcd2f9c251d3" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:36.038297 19074 ts_tablet_manager.cc:1434] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 13:21:36.040052 19074 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:36.045599 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.052870 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:36.054975 18917 leader_election.cc:343] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.055576 18917 leader_election.cc:304] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:36.056991 19184 raft_consensus.cc:2749] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.058079 19136 raft_consensus.cc:2749] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.048931 19083 leader_election.cc:290] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:36.060504 19074 tablet_bootstrap.cc:654] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.061885 19076 tablet_bootstrap.cc:654] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.083637 18994 leader_election.cc:304] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:36.085000 19227 raft_consensus.cc:2749] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.065838 19083 consensus_queue.cc:1048] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:21:36.093130 18800 raft_consensus.cc:2468] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:36.107939 18919 leader_election.cc:304] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:36.109721 19229 raft_consensus.cc:2804] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.110440 19229 raft_consensus.cc:697] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:36.111690 19229 consensus_queue.cc:237] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.123204 19073 ts_tablet_manager.cc:1434] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.157s	user 0.008s	sys 0.004s
I20260709 13:21:36.154362 19227 consensus_queue.cc:1048] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.156090 19073 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:36.157249 19136 raft_consensus.cc:493] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.158016 19136 raft_consensus.cc:515] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.163254 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.163439 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:36.164140 18801 raft_consensus.cc:2468] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:36.164971 19223 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.165560 19223 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:36.167271 18846 leader_election.cc:343] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.168541 19136 leader_election.cc:290] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:36.173691 19223 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:36.173804 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:36.180047 18845 leader_election.cc:304] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:36.180721 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.181720 19223 raft_consensus.cc:2804] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.182221 19223 raft_consensus.cc:493] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.182632 19223 raft_consensus.cc:3060] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.182807 18844 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:36.184499 19136 raft_consensus.cc:2749] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.186787 19227 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.187633 19074 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:36.187573 19227 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.188546 19074 ts_tablet_manager.cc:1403] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.149s	user 0.023s	sys 0.019s
I20260709 13:21:36.203574 19073 tablet_bootstrap.cc:654] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.196825 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:36.193591 19227 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:36.208241 18991 leader_election.cc:343] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.214197 19074 raft_consensus.cc:359] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.215837 19074 raft_consensus.cc:385] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.216354 19074 raft_consensus.cc:740] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.217702 19074 consensus_queue.cc:260] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.220027 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.224323 18878 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.225718 19076 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:36.231580 19076 ts_tablet_manager.cc:1403] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.249s	user 0.006s	sys 0.012s
I20260709 13:21:36.235769 19074 ts_tablet_manager.cc:1434] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.002s
W20260709 13:21:36.235929 18993 leader_election.cc:343] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.236575 18993 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:36.238718 19227 raft_consensus.cc:2749] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.241420 19223 raft_consensus.cc:515] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.245846 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:36.247267 19075 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:36.247972 19075 ts_tablet_manager.cc:1403] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.332s	user 0.019s	sys 0.004s
I20260709 13:21:36.248189 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:36.249312 18800 raft_consensus.cc:3060] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:36.253692 18846 leader_election.cc:343] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.254832 19223 leader_election.cc:290] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:36.259797 19074 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:36.264391 19076 raft_consensus.cc:359] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.265559 19076 raft_consensus.cc:385] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.266008 19076 raft_consensus.cc:740] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.267637 19076 consensus_queue.cc:260] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.268203 19074 tablet_bootstrap.cc:654] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.275519 19076 ts_tablet_manager.cc:1434] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.001s
I20260709 13:21:36.280098 19076 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:36.290230 18800 raft_consensus.cc:2468] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:36.292001 18845 leader_election.cc:304] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:36.292286 19075 raft_consensus.cc:359] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.293444 19075 raft_consensus.cc:385] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.293503 19223 raft_consensus.cc:2804] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.293875 19075 raft_consensus.cc:740] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.294133 19223 raft_consensus.cc:697] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:36.295286 19223 consensus_queue.cc:237] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.297535 19073 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:36.298105 19073 ts_tablet_manager.cc:1403] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.142s	user 0.021s	sys 0.003s
I20260709 13:21:36.301278 19073 raft_consensus.cc:359] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.302033 19073 raft_consensus.cc:385] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.302736 19073 raft_consensus.cc:740] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.301991 18702 catalog_manager.cc:5697] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.303802 19073 consensus_queue.cc:260] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.302671 19075 consensus_queue.cc:260] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.306967 19073 ts_tablet_manager.cc:1434] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:21:36.308408 19073 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:36.308396 19075 ts_tablet_manager.cc:1434] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
I20260709 13:21:36.310791 19076 tablet_bootstrap.cc:654] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.311381 19074 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:36.312081 19074 ts_tablet_manager.cc:1403] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.008s
I20260709 13:21:36.315905 19074 raft_consensus.cc:359] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.316736 19074 raft_consensus.cc:385] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.317020 19073 tablet_bootstrap.cc:654] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.317998 19075 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:36.317075 19074 raft_consensus.cc:740] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.324738 19074 consensus_queue.cc:260] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.328748 19074 ts_tablet_manager.cc:1434] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:21:36.330757 19074 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:36.333580 19229 raft_consensus.cc:493] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.336912 19075 tablet_bootstrap.cc:654] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.342922 19076 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:36.337704 19229 raft_consensus.cc:515] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.343976 19076 ts_tablet_manager.cc:1403] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.066s	user 0.020s	sys 0.017s
I20260709 13:21:36.348985 19076 raft_consensus.cc:359] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.350087 19076 raft_consensus.cc:385] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.350461 19076 raft_consensus.cc:740] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.351495 19076 consensus_queue.cc:260] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.352933 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.353679 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.348002 19229 leader_election.cc:290] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:36.354342 19076 ts_tablet_manager.cc:1434] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
W20260709 13:21:36.355605 18919 leader_election.cc:343] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.356117 19076 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:36.354558 19026 raft_consensus.cc:2468] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:36.366343 18918 leader_election.cc:304] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:36.370270 19076 tablet_bootstrap.cc:654] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.373492 19229 raft_consensus.cc:2804] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.374132 19229 raft_consensus.cc:493] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.374688 19229 raft_consensus.cc:3060] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.367583 19088 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.377780 19074 tablet_bootstrap.cc:654] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.377643 19088 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.383258 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.384330 19026 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:36.389350 18701 catalog_manager.cc:5697] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.401652 18769 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:36.402866 19233 raft_consensus.cc:2804] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.403383 19233 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.403769 19233 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.407039 19088 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:36.410471 19073 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:36.410264 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:36.411072 19073 ts_tablet_manager.cc:1403] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.103s	user 0.031s	sys 0.022s
W20260709 13:21:36.412249 18768 leader_election.cc:343] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.416904 19075 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:36.417510 19075 ts_tablet_manager.cc:1403] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.100s	user 0.012s	sys 0.006s
I20260709 13:21:36.414602 19073 raft_consensus.cc:359] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.419486 19073 raft_consensus.cc:385] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.419909 19073 raft_consensus.cc:740] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.420888 18800 raft_consensus.cc:3060] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.420365 18700 catalog_manager.cc:5697] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.420871 19073 consensus_queue.cc:260] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.425719 19073 ts_tablet_manager.cc:1434] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 13:21:36.427297 19073 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:36.432025 19233 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.436525 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:36.437649 19026 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.440816 19229 raft_consensus.cc:515] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.451329 19223 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.452051 19223 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.457944 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.459239 19024 raft_consensus.cc:2393] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate 878e795a8488495f978538952e7b2324 in this term.
I20260709 13:21:36.461295 19073 tablet_bootstrap.cc:654] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.463620 19075 raft_consensus.cc:359] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.464926 19075 raft_consensus.cc:385] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.465304 19075 raft_consensus.cc:740] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.466341 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.469660 18845 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:36.471287 19136 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.472157 19233 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:36.466414 19075 consensus_queue.cc:260] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.477530 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:36.478945 19074 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:36.479172 19026 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:36.479924 19074 ts_tablet_manager.cc:1403] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.150s	user 0.014s	sys 0.004s
I20260709 13:21:36.481515 18769 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:36.475441 19223 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:36.483911 19074 raft_consensus.cc:359] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.484773 19074 raft_consensus.cc:385] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.485113 19074 raft_consensus.cc:740] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.486075 19073 tablet_bootstrap.cc:492] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
W20260709 13:21:36.485119 18768 leader_election.cc:343] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.487537 19074 consensus_queue.cc:260] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.489329 19233 raft_consensus.cc:2804] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.489944 19233 raft_consensus.cc:697] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:36.490681 19229 leader_election.cc:290] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:36.490999 19233 consensus_queue.cc:237] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.493527 19227 raft_consensus.cc:493] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.494251 19227 raft_consensus.cc:515] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.489940 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:36.497025 19026 raft_consensus.cc:3060] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.498095 19227 leader_election.cc:290] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:36.501508 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:36.502388 19074 ts_tablet_manager.cc:1434] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:21:36.503218 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.503577 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:36.503974 19074 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:36.505151 18993 leader_election.cc:343] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:36.506454 18919 leader_election.cc:343] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:36.508669 18991 leader_election.cc:343] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:36.509218 19076 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:36.509331 18991 leader_election.cc:304] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:36.509711 19229 consensus_queue.cc:1048] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.510670 19075 ts_tablet_manager.cc:1434] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.001s
I20260709 13:21:36.511101 19227 raft_consensus.cc:2749] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.518381 19074 tablet_bootstrap.cc:654] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.519019 18703 catalog_manager.cc:5697] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.509850 19076 ts_tablet_manager.cc:1403] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.154s	user 0.035s	sys 0.024s
I20260709 13:21:36.512137 19075 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:36.539570 19073 ts_tablet_manager.cc:1403] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.113s	user 0.014s	sys 0.003s
I20260709 13:21:36.534819 19136 raft_consensus.cc:2749] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.544157 19074 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:36.544999 19074 ts_tablet_manager.cc:1403] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.004s
I20260709 13:21:36.561682 19026 raft_consensus.cc:2468] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:36.566100 19074 raft_consensus.cc:359] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.569198 19074 raft_consensus.cc:385] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.569561 19074 raft_consensus.cc:740] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.571893 18918 leader_election.cc:304] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:36.569676 19076 raft_consensus.cc:359] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.573866 19076 raft_consensus.cc:385] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.574627 19229 raft_consensus.cc:2804] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:36.574214 19076 raft_consensus.cc:740] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.576658 19229 raft_consensus.cc:697] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:36.575682 19073 raft_consensus.cc:359] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.577582 19073 raft_consensus.cc:385] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.577939 19073 raft_consensus.cc:740] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.578284 19075 tablet_bootstrap.cc:654] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.577754 19229 consensus_queue.cc:237] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.579936 19073 consensus_queue.cc:260] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.581478 19076 consensus_queue.cc:260] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.583395 19088 consensus_queue.cc:1048] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.583841 19076 ts_tablet_manager.cc:1434] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
I20260709 13:21:36.591882 19073 ts_tablet_manager.cc:1434] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:21:36.592696 19127 consensus_queue.cc:1048] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.594324 19073 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:36.594897 19074 consensus_queue.cc:260] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.592859 19076 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:36.607738 19074 ts_tablet_manager.cc:1434] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 13:21:36.610293 18701 catalog_manager.cc:5697] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.614341 19073 tablet_bootstrap.cc:654] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.643885 19229 raft_consensus.cc:493] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.644568 19229 raft_consensus.cc:515] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.646427 19074 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:36.646951 19229 leader_election.cc:290] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:36.648528 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:36.652745 18702 catalog_manager.cc:5697] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:36.648623 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c67b792f4ba14e55abb2a3a92d4d0b2b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.655149 19074 tablet_bootstrap.cc:654] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.656136 19076 tablet_bootstrap.cc:654] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.657166 19075 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:36.660373 19075 ts_tablet_manager.cc:1403] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.149s	user 0.023s	sys 0.020s
I20260709 13:21:36.661922 18800 raft_consensus.cc:2393] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de62ff9486f7483e8f0e0b93dcca24bb in current term 1: Already voted for candidate 039c551e55fb43cdb5c3880b7cb91a0b in this term.
I20260709 13:21:36.671458 19088 consensus_queue.cc:1048] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.672963 18919 leader_election.cc:304] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:36.674010 19229 raft_consensus.cc:3060] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.684170 19229 raft_consensus.cc:2749] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:36.685496 19075 raft_consensus.cc:359] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.686491 19075 raft_consensus.cc:385] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.686784 19075 raft_consensus.cc:740] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.701792 19075 consensus_queue.cc:260] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.712554 19217 consensus_queue.cc:1048] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.715945 19130 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.719466 19130 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.723023 19223 raft_consensus.cc:493] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.724779 19223 raft_consensus.cc:515] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.726795 19075 ts_tablet_manager.cc:1434] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:21:36.728732 19075 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:36.732378 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:36.733639 18880 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:36.735908 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:36.736248 19075 tablet_bootstrap.cc:654] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.736358 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:36.736822 18952 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:36.737485 18951 raft_consensus.cc:2468] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:36.744383 18768 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:36.747041 19233 raft_consensus.cc:2804] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.747697 19233 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.748124 19233 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.750700 19217 consensus_queue.cc:1048] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:36.733675 19073 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:36.752413 19073 ts_tablet_manager.cc:1403] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.158s	user 0.010s	sys 0.011s
I20260709 13:21:36.756904 18846 leader_election.cc:304] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:36.750728 19223 leader_election.cc:290] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:36.759330 19223 raft_consensus.cc:2804] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:36.759933 19223 raft_consensus.cc:493] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.760509 19223 raft_consensus.cc:3060] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.764329 19130 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:36.772770 19073 raft_consensus.cc:359] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.777477 19073 raft_consensus.cc:385] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.778990 19073 raft_consensus.cc:740] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.787541 19073 consensus_queue.cc:260] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.798830 19073 ts_tablet_manager.cc:1434] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 13:21:36.814280 19075 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:36.815562 19075 ts_tablet_manager.cc:1403] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.088s	user 0.018s	sys 0.007s
I20260709 13:21:36.817826 19074 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:36.818536 19074 ts_tablet_manager.cc:1403] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.189s	user 0.035s	sys 0.049s
I20260709 13:21:36.802572 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.822511 19074 raft_consensus.cc:359] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.822788 19026 raft_consensus.cc:2468] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:36.824092 19074 raft_consensus.cc:385] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.824536 19074 raft_consensus.cc:740] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.825512 19074 consensus_queue.cc:260] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.825997 19076 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:36.826736 19076 ts_tablet_manager.cc:1403] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.234s	user 0.014s	sys 0.013s
I20260709 13:21:36.848021 19233 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:36.842837 19076 raft_consensus.cc:359] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.854463 19076 raft_consensus.cc:385] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.855865 19073 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:36.854848 19076 raft_consensus.cc:740] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.860536 19074 ts_tablet_manager.cc:1434] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.001s
I20260709 13:21:36.862111 19076 consensus_queue.cc:260] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.872465 19074 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:36.873558 19076 ts_tablet_manager.cc:1434] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.001s
I20260709 13:21:36.878830 19223 raft_consensus.cc:515] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:36.882844 19223 leader_election.cc:290] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:36.883710 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:36.884120 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:36.884665 18951 raft_consensus.cc:3060] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.884970 19024 raft_consensus.cc:3060] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.890235 19076 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:36.900341 19073 tablet_bootstrap.cc:654] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:36.897037 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:36.906226 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:36.907145 18952 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.907696 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:36.908627 18878 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.913651 19233 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:36.937080 19075 raft_consensus.cc:359] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.939385 18878 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:36.939958 19217 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.942059 19217 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:36.944725 19217 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:36.947983 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:36.941118 19075 raft_consensus.cc:385] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:36.948868 19075 raft_consensus.cc:740] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:36.948997 19026 raft_consensus.cc:2393] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate 878e795a8488495f978538952e7b2324 in this term.
I20260709 13:21:36.949895 19073 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:36.958005 19073 ts_tablet_manager.cc:1403] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.027s
I20260709 13:21:36.951030 19074 tablet_bootstrap.cc:654] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.952216 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:36.957561 19075 consensus_queue.cc:260] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.954216 19076 tablet_bootstrap.cc:654] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:36.956295 18952 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:36.969715 19075 ts_tablet_manager.cc:1434] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.142s	user 0.008s	sys 0.002s
I20260709 13:21:36.970407 18768 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:36.971875 19075 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:36.990069 19083 raft_consensus.cc:493] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.991050 19083 raft_consensus.cc:515] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:36.992318 18951 raft_consensus.cc:2468] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:36.995755 18845 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:36.997622 19217 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:36.998215 19184 raft_consensus.cc:493] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.000969 19184 raft_consensus.cc:515] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.005584 18846 leader_election.cc:304] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:37.006990 19282 raft_consensus.cc:2804] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.007623 19282 raft_consensus.cc:697] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:37.008621 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:37.008721 19282 consensus_queue.cc:237] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.013505 19127 consensus_queue.cc:1048] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.017541 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.019003 18951 raft_consensus.cc:2468] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:37.029072 19130 raft_consensus.cc:493] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.030121 19130 raft_consensus.cc:515] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.035370 19130 leader_election.cc:290] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:37.036445 18994 leader_election.cc:304] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:37.037679 19227 raft_consensus.cc:2804] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.038362 19227 raft_consensus.cc:493] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.038787 19227 raft_consensus.cc:3060] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.040145 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.041230 19026 raft_consensus.cc:2393] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 878e795a8488495f978538952e7b2324 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
W20260709 13:21:37.026473 18917 leader_election.cc:343] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.045082 18801 raft_consensus.cc:3060] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.049430 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ac0dee3476f4361a8531c64161fee77" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.050156 19184 leader_election.cc:290] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:36.986274 19233 raft_consensus.cc:2804] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.051402 19233 raft_consensus.cc:697] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:37.053495 19233 consensus_queue.cc:237] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.055840 19024 raft_consensus.cc:2468] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:37.057971 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.049481 19083 leader_election.cc:290] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.073021 19088 consensus_queue.cc:1048] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.075078 18771 leader_election.cc:304] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:37.095328 18918 leader_election.cc:343] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.096567 19075 tablet_bootstrap.cc:654] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.095907 18918 leader_election.cc:304] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:37.099036 19229 raft_consensus.cc:2749] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:37.099682 19076 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:37.100302 19076 ts_tablet_manager.cc:1403] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.211s	user 0.039s	sys 0.046s
I20260709 13:21:37.106571 19076 raft_consensus.cc:359] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.107437 19076 raft_consensus.cc:385] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.107755 19076 raft_consensus.cc:740] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.108563 19076 consensus_queue.cc:260] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.113011 19130 raft_consensus.cc:2749] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:37.117005 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.117573 19217 raft_consensus.cc:2749] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:37.119334 18993 leader_election.cc:343] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.118786 18703 catalog_manager.cc:5697] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:37.138477 19127 consensus_queue.cc:1048] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.150054 18702 catalog_manager.cc:5697] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:37.158792 19076 ts_tablet_manager.cc:1434] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.001s
I20260709 13:21:37.164633 19073 raft_consensus.cc:359] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.165532 19234 consensus_queue.cc:1048] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.168646 19074 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:37.165581 19073 raft_consensus.cc:385] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.175163 19073 raft_consensus.cc:740] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.176199 19074 ts_tablet_manager.cc:1403] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.304s	user 0.039s	sys 0.038s
I20260709 13:21:37.186491 19073 consensus_queue.cc:260] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.188459 19076 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:37.193377 19073 ts_tablet_manager.cc:1434] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.234s	user 0.009s	sys 0.001s
I20260709 13:21:37.195045 19073 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:37.213443 19074 raft_consensus.cc:359] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.214349 19074 raft_consensus.cc:385] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.214649 19074 raft_consensus.cc:740] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.215756 19234 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.216565 19234 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.218575 19074 consensus_queue.cc:260] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.226559 19074 ts_tablet_manager.cc:1434] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:21:37.231216 19074 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:37.233974 19075 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:37.234696 19075 ts_tablet_manager.cc:1403] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.263s	user 0.024s	sys 0.025s
I20260709 13:21:37.246470 19223 raft_consensus.cc:493] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.247210 19223 raft_consensus.cc:515] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.248059 19073 tablet_bootstrap.cc:654] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.249676 19223 leader_election.cc:290] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.254505 19074 tablet_bootstrap.cc:654] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.282210 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.283529 19026 raft_consensus.cc:2468] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:37.286789 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.289417 18800 raft_consensus.cc:2468] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:37.243561 19075 raft_consensus.cc:359] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.291714 19075 raft_consensus.cc:385] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.292079 19075 raft_consensus.cc:740] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.292909 19075 consensus_queue.cc:260] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.290373 19227 raft_consensus.cc:515] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.295990 18845 leader_election.cc:304] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:37.297698 19227 leader_election.cc:290] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.298426 19217 raft_consensus.cc:2804] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.299212 19217 raft_consensus.cc:493] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.299716 19217 raft_consensus.cc:3060] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.302102 19076 tablet_bootstrap.cc:654] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.304095 19234 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:37.305419 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:37.307559 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:37.308652 18951 raft_consensus.cc:3060] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.307708 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.310876 18952 raft_consensus.cc:2468] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:37.311589 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:37.310851 18993 leader_election.cc:343] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:37.324499 18768 leader_election.cc:343] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.333642 19075 ts_tablet_manager.cc:1434] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.005s
I20260709 13:21:37.340384 19075 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:37.338397 18771 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:37.343887 19234 raft_consensus.cc:2804] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.344412 19234 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.344820 19234 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.351557 18951 raft_consensus.cc:2468] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:37.353399 18994 leader_election.cc:304] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:37.363508 19227 raft_consensus.cc:2804] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.365235 19227 raft_consensus.cc:697] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:37.366549 19227 consensus_queue.cc:237] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.377528 19075 tablet_bootstrap.cc:654] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.383029 19073 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:37.383895 19073 ts_tablet_manager.cc:1403] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.189s	user 0.039s	sys 0.082s
I20260709 13:21:37.388669 19074 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:37.389446 19074 ts_tablet_manager.cc:1403] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.159s	user 0.016s	sys 0.048s
I20260709 13:21:37.393079 19074 raft_consensus.cc:359] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.393890 19074 raft_consensus.cc:385] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.394198 19074 raft_consensus.cc:740] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.394644 19076 tablet_bootstrap.cc:492] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:37.395334 19234 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.405689 19076 ts_tablet_manager.cc:1403] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.218s	user 0.013s	sys 0.030s
I20260709 13:21:37.407637 19234 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:37.409531 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:37.398994 19074 consensus_queue.cc:260] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.404662 19217 raft_consensus.cc:515] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.411247 18702 catalog_manager.cc:5697] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:37.412215 18768 leader_election.cc:343] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.401155 19083 raft_consensus.cc:493] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.414458 19083 raft_consensus.cc:515] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.415805 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:37.416885 18801 raft_consensus.cc:3060] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.417150 19074 ts_tablet_manager.cc:1434] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 13:21:37.417009 19083 leader_election.cc:290] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.417896 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.418458 19074 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:37.420258 19075 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:37.415805 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:37.420926 19075 ts_tablet_manager.cc:1403] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.000s
I20260709 13:21:37.421151 18951 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.413375 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:37.422205 19024 raft_consensus.cc:3060] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:37.420120 18994 leader_election.cc:343] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.422304 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.398382 19073 raft_consensus.cc:359] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.424470 19073 raft_consensus.cc:385] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.425016 19073 raft_consensus.cc:740] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.426047 19073 consensus_queue.cc:260] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.429366 19074 tablet_bootstrap.cc:654] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.429891 19073 ts_tablet_manager.cc:1434] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.003s
I20260709 13:21:37.431303 19073 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
W20260709 13:21:37.424705 18993 leader_election.cc:343] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.433856 18993 leader_election.cc:304] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:37.435247 19083 raft_consensus.cc:2749] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:37.435137 19075 raft_consensus.cc:359] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.436115 19075 raft_consensus.cc:385] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.436434 19075 raft_consensus.cc:740] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.438232 19075 consensus_queue.cc:260] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.440515 19075 ts_tablet_manager.cc:1434] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:21:37.441841 19075 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:37.409559 19076 raft_consensus.cc:359] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.444130 19076 raft_consensus.cc:385] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.444487 19076 raft_consensus.cc:740] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.445381 19076 consensus_queue.cc:260] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.415944 19217 leader_election.cc:290] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.456650 19075 tablet_bootstrap.cc:654] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.450232 19024 raft_consensus.cc:2468] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:37.453742 19073 tablet_bootstrap.cc:654] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.490854 18844 leader_election.cc:304] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:37.492077 19217 raft_consensus.cc:2804] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.450392 18801 raft_consensus.cc:2468] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:37.493546 19217 raft_consensus.cc:697] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:37.495376 19074 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:37.494675 19217 consensus_queue.cc:237] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.496057 19074 ts_tablet_manager.cc:1403] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.078s	user 0.008s	sys 0.006s
I20260709 13:21:37.502341 19184 raft_consensus.cc:493] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.503075 19184 raft_consensus.cc:515] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.488519 19076 ts_tablet_manager.cc:1434] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.002s
I20260709 13:21:37.505218 19076 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:37.507665 19184 leader_election.cc:290] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:37.507934 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.509413 19073 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:37.511834 19073 ts_tablet_manager.cc:1403] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.081s	user 0.008s	sys 0.013s
W20260709 13:21:37.514377 18919 leader_election.cc:343] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.512506 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.515594 19024 raft_consensus.cc:2468] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:37.517234 18918 leader_election.cc:304] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:37.518385 19073 raft_consensus.cc:359] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.519490 19184 raft_consensus.cc:2804] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.519176 19074 raft_consensus.cc:359] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.520107 19184 raft_consensus.cc:493] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.520478 19074 raft_consensus.cc:385] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.520653 19184 raft_consensus.cc:3060] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.521028 19074 raft_consensus.cc:740] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.519243 19073 raft_consensus.cc:385] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.523787 19073 raft_consensus.cc:740] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.527045 18951 raft_consensus.cc:2468] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:37.527621 19073 consensus_queue.cc:260] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.527964 19184 raft_consensus.cc:515] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.531239 19073 ts_tablet_manager.cc:1434] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:21:37.533751 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:37.535079 19073 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:37.545266 18703 catalog_manager.cc:5697] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:37.547835 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:37.549067 19184 leader_election.cc:290] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:37.549767 18919 leader_election.cc:343] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.552484 19234 consensus_queue.cc:1048] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.553689 19024 raft_consensus.cc:3060] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.556100 19073 tablet_bootstrap.cc:654] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.557885 19234 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.558547 19234 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.563961 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.564329 19234 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:37.564818 19026 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:37.570070 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:37.572429 18768 leader_election.cc:343] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.572571 19024 raft_consensus.cc:2468] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:37.575101 18769 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:37.575766 18918 leader_election.cc:304] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:37.576365 19205 raft_consensus.cc:2804] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.576228 19234 consensus_queue.cc:1048] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.576960 19205 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.577418 19184 raft_consensus.cc:2804] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.577383 19205 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.578250 19184 raft_consensus.cc:697] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:37.580247 19217 raft_consensus.cc:493] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.580888 19217 raft_consensus.cc:515] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.583807 19217 leader_election.cc:290] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:37.586252 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.587558 18954 raft_consensus.cc:2468] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:37.589721 18846 leader_election.cc:304] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:37.579650 19184 consensus_queue.cc:237] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.591414 19223 consensus_queue.cc:1048] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.593792 19223 raft_consensus.cc:2804] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:37.594427 19223 raft_consensus.cc:493] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.594846 19223 raft_consensus.cc:3060] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.600634 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:37.601672 19026 raft_consensus.cc:2468] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 0.
I20260709 13:21:37.522305 19074 consensus_queue.cc:260] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.622727 19074 ts_tablet_manager.cc:1434] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.122s	user 0.010s	sys 0.000s
I20260709 13:21:37.630380 19073 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:37.631981 18771 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:37.635408 19130 raft_consensus.cc:2804] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.636080 19130 raft_consensus.cc:697] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:37.637048 19130 consensus_queue.cc:237] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.637930 19073 ts_tablet_manager.cc:1403] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.003s
I20260709 13:21:37.639604 19217 consensus_queue.cc:1048] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.644335 19074 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:37.653189 19074 tablet_bootstrap.cc:654] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.643898 19076 tablet_bootstrap.cc:654] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.642691 19073 raft_consensus.cc:359] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.659600 19073 raft_consensus.cc:385] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.660069 19073 raft_consensus.cc:740] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.661139 19073 consensus_queue.cc:260] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.664561 19073 ts_tablet_manager.cc:1434] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.003s
I20260709 13:21:37.667315 19073 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:37.676342 19073 tablet_bootstrap.cc:654] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.670104 18700 catalog_manager.cc:5697] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:37.656896 19223 raft_consensus.cc:515] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.703682 19075 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:37.704329 19075 ts_tablet_manager.cc:1403] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.263s	user 0.039s	sys 0.108s
I20260709 13:21:37.710595 19075 raft_consensus.cc:359] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.718096 19075 raft_consensus.cc:385] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.723518 19075 raft_consensus.cc:740] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.724726 19075 consensus_queue.cc:260] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.726456 19075 ts_tablet_manager.cc:1434] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:21:37.725950 19205 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.727658 19233 raft_consensus.cc:493] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.728288 19233 raft_consensus.cc:515] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:37.729339 19205 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:37.735790 19075 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:37.742652 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:37.756460 19024 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.757469 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:37.745000 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:37.759831 18951 raft_consensus.cc:3060] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.762933 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:37.743603 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:37.768474 19026 raft_consensus.cc:3060] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:37.768796 19227 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.751053 19223 leader_election.cc:290] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:37.769515 19227 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.781816 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.783329 19073 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:37.786952 19073 ts_tablet_manager.cc:1403] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.120s	user 0.020s	sys 0.001s
W20260709 13:21:37.789458 18768 leader_election.cc:343] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.793325 19227 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:37.796214 18993 leader_election.cc:343] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.796471 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:37.799196 18954 raft_consensus.cc:2393] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 878e795a8488495f978538952e7b2324 in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:37.799727 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f2be91edb9e4188a248db3e309f0e44" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:37.800473 18991 leader_election.cc:343] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.783232 19233 leader_election.cc:290] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:37.801286 18991 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:37.804214 19227 raft_consensus.cc:2749] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:37.808920 18771 leader_election.cc:304] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:37.811543 18801 raft_consensus.cc:3060] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:37.789805 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 44740.3us
I20260709 13:21:37.809649 19024 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:37.858968 19233 raft_consensus.cc:2749] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:37.859380 19227 consensus_queue.cc:1048] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.860941 18700 catalog_manager.cc:5697] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:37.834046 19026 raft_consensus.cc:2468] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:37.848654 19075 tablet_bootstrap.cc:654] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:37.865031 18769 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:37.868944 18844 leader_election.cc:304] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:37.872351 19130 raft_consensus.cc:2804] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.873059 19130 raft_consensus.cc:697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:37.874464 19130 consensus_queue.cc:237] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.850483 18951 raft_consensus.cc:2468] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:37.884055 19073 raft_consensus.cc:359] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.886314 19073 raft_consensus.cc:385] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.886763 19073 raft_consensus.cc:740] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.901041 19073 consensus_queue.cc:260] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.904228 19074 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:37.904939 19074 ts_tablet_manager.cc:1403] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.274s	user 0.017s	sys 0.008s
I20260709 13:21:37.896047 19229 raft_consensus.cc:493] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.907374 19313 raft_consensus.cc:2804] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:37.907420 19229 raft_consensus.cc:515] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.926692 19074 raft_consensus.cc:359] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.927814 19074 raft_consensus.cc:385] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:37.928157 19074 raft_consensus.cc:740] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.930402 19074 consensus_queue.cc:260] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:37.944023 19076 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:37.944796 19076 ts_tablet_manager.cc:1403] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.440s	user 0.050s	sys 0.042s
I20260709 13:21:37.927542 19313 raft_consensus.cc:697] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:37.948706 19074 ts_tablet_manager.cc:1434] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.003s
I20260709 13:21:37.950139 19073 ts_tablet_manager.cc:1434] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.163s	user 0.001s	sys 0.010s
I20260709 13:21:37.950855 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:37.938743 19229 leader_election.cc:290] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:37.947526 19083 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.955216 19073 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:37.952891 18802 raft_consensus.cc:2468] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:37.959102 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:37.961472 19074 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:37.961769 18917 leader_election.cc:343] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:37.967926 19083 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:37.962730 18703 catalog_manager.cc:5697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:37.972611 19227 consensus_queue.cc:1048] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.982016 19311 consensus_queue.cc:1048] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:37.984081 19313 consensus_queue.cc:237] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:37.979820 19083 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:37.986500 19075 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:37.994711 19075 ts_tablet_manager.cc:1403] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.262s	user 0.026s	sys 0.017s
I20260709 13:21:37.984691 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:37.983247 19076 raft_consensus.cc:359] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.002063 19076 raft_consensus.cc:385] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.002561 19076 raft_consensus.cc:740] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:37.998893 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:38.009047 19073 tablet_bootstrap.cc:654] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.011631 19076 consensus_queue.cc:260] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.016857 19075 raft_consensus.cc:359] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.017567 19075 raft_consensus.cc:385] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.017810 19075 raft_consensus.cc:740] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.020316 19076 ts_tablet_manager.cc:1434] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.075s	user 0.004s	sys 0.005s
I20260709 13:21:38.021116 19075 consensus_queue.cc:260] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.027074 19075 ts_tablet_manager.cc:1434] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.001s
I20260709 13:21:38.031785 19075 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
W20260709 13:21:38.032598 18991 leader_election.cc:343] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.036161 19074 tablet_bootstrap.cc:654] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:38.039945 19321 log.cc:927] Time spent T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:21:38.041960 19223 consensus_queue.cc:1048] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:38.043797 18993 leader_election.cc:343] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.003661 18919 leader_election.cc:304] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.044787 18993 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:38.045478 19214 raft_consensus.cc:2804] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.046048 19214 raft_consensus.cc:493] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.046170 19227 raft_consensus.cc:2749] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:38.046553 19214 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:38.049350 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 49149us
I20260709 13:21:38.054246 19075 tablet_bootstrap.cc:654] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.060906 19076 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:38.092891 19076 tablet_bootstrap.cc:654] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.135847 19074 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.136709 19074 ts_tablet_manager.cc:1403] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.176s	user 0.019s	sys 0.016s
I20260709 13:21:38.144183 19074 raft_consensus.cc:359] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.145040 19074 raft_consensus.cc:385] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.145365 19074 raft_consensus.cc:740] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.146170 19074 consensus_queue.cc:260] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:38.151541 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:38.111392 19214 raft_consensus.cc:515] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.156307 19074 ts_tablet_manager.cc:1434] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:21:38.158885 19322 consensus_queue.cc:1048] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.165314 18701 catalog_manager.cc:5697] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:38.172506 19233 consensus_queue.cc:1048] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.177855 19214 leader_election.cc:290] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:38.179257 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:38.189577 19074 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:38.196229 18917 leader_election.cc:343] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.190258 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:38.200711 18800 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.206915 19076 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:38.207744 19076 ts_tablet_manager.cc:1403] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.151s	user 0.014s	sys 0.002s
W20260709 13:21:38.209188 18768 consensus_peers.cc:597] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:38.193256 19073 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:38.212910 19073 ts_tablet_manager.cc:1403] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.261s	user 0.011s	sys 0.012s
I20260709 13:21:38.218372 19076 raft_consensus.cc:359] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:38.193130 18991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 100946us
I20260709 13:21:38.219348 19076 raft_consensus.cc:385] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.219720 19076 raft_consensus.cc:740] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.222127 19074 tablet_bootstrap.cc:654] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.230201 19076 consensus_queue.cc:260] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.244422 19073 raft_consensus.cc:359] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.242458 19076 ts_tablet_manager.cc:1434] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 13:21:38.247987 19073 raft_consensus.cc:385] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.248555 19073 raft_consensus.cc:740] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.250521 19076 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:38.251643 19073 consensus_queue.cc:260] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.254585 19073 ts_tablet_manager.cc:1434] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:21:38.273155 18800 raft_consensus.cc:2468] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:38.277457 19075 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:38.278232 19075 ts_tablet_manager.cc:1403] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.247s	user 0.024s	sys 0.011s
I20260709 13:21:38.279021 19076 tablet_bootstrap.cc:654] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.282001 18919 leader_election.cc:304] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.283815 19214 raft_consensus.cc:2804] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:38.284482 19214 raft_consensus.cc:697] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:38.285429 19214 consensus_queue.cc:237] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:38.286046 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:38.298636 19075 raft_consensus.cc:359] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.300197 19075 raft_consensus.cc:385] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.300705 19075 raft_consensus.cc:740] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.302104 19075 consensus_queue.cc:260] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.310899 19073 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:38.313203 19075 ts_tablet_manager.cc:1434] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.004s
I20260709 13:21:38.316092 19075 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:38.327946 18880 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.343205 19074 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.343900 19074 ts_tablet_manager.cc:1403] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.155s	user 0.019s	sys 0.031s
I20260709 13:21:38.345623 19075 tablet_bootstrap.cc:654] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.346613 19074 raft_consensus.cc:359] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.347488 19074 raft_consensus.cc:385] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.347800 19074 raft_consensus.cc:740] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.349594 19074 consensus_queue.cc:260] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.350842 19076 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:38.350358 18702 catalog_manager.cc:5697] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:38.356490 19073 tablet_bootstrap.cc:654] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.356206 19314 consensus_queue.cc:1048] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.364181 19074 ts_tablet_manager.cc:1434] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:21:38.351500 19076 ts_tablet_manager.cc:1403] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.101s	user 0.023s	sys 0.007s
I20260709 13:21:38.366526 19074 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.381557 19075 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:38.382319 19075 ts_tablet_manager.cc:1403] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.003s
I20260709 13:21:38.383498 19314 raft_consensus.cc:493] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.383841 19073 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:38.384338 19074 tablet_bootstrap.cc:654] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.384610 19073 ts_tablet_manager.cc:1403] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.120s	user 0.024s	sys 0.008s
I20260709 13:21:38.385975 19075 raft_consensus.cc:359] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.387375 19075 raft_consensus.cc:385] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.387749 19075 raft_consensus.cc:740] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.384187 19314 raft_consensus.cc:515] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.388700 19075 consensus_queue.cc:260] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.391571 19075 ts_tablet_manager.cc:1434] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:21:38.392172 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.388609 19073 raft_consensus.cc:359] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.394304 19073 raft_consensus.cc:385] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.394610 19073 raft_consensus.cc:740] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.395534 19073 consensus_queue.cc:260] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.402988 19075 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:38.405861 19314 leader_election.cc:290] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.409845 19074 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.410028 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac00a69e4894ac79bdbbb74eaa3ad51" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.410636 19074 ts_tablet_manager.cc:1403] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.004s
I20260709 13:21:38.412927 18769 leader_election.cc:304] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:38.414120 19074 raft_consensus.cc:359] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.415009 19074 raft_consensus.cc:385] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.414880 19130 raft_consensus.cc:2749] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:38.415870 19314 consensus_queue.cc:1048] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.419943 19075 tablet_bootstrap.cc:654] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.416206 19223 consensus_queue.cc:1048] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:38.415959 19074 raft_consensus.cc:740] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.422112 19074 consensus_queue.cc:260] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.424937 19074 ts_tablet_manager.cc:1434] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 13:21:38.426410 19074 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.428875 19073 ts_tablet_manager.cc:1434] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.003s
I20260709 13:21:38.432610 19076 raft_consensus.cc:359] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.433511 19223 raft_consensus.cc:493] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.434150 19223 raft_consensus.cc:515] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.439389 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.443970 19223 leader_election.cc:290] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:38.432778 19073 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:38.446128 19214 raft_consensus.cc:493] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.447001 19214 raft_consensus.cc:515] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.450745 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:38.452667 18917 leader_election.cc:343] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.453696 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.454583 19024 raft_consensus.cc:2468] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:38.454934 19075 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:38.455611 19075 ts_tablet_manager.cc:1403] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.003s
I20260709 13:21:38.455953 18918 leader_election.cc:304] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.456744 19074 tablet_bootstrap.cc:654] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.457021 19229 raft_consensus.cc:2804] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.457471 19229 raft_consensus.cc:493] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.457856 19229 raft_consensus.cc:3060] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.459405 19075 raft_consensus.cc:359] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.460224 19075 raft_consensus.cc:385] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.460526 19075 raft_consensus.cc:740] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.461410 19075 consensus_queue.cc:260] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.462716 19214 leader_election.cc:290] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.466727 19073 tablet_bootstrap.cc:654] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.471020 19075 ts_tablet_manager.cc:1434] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:21:38.472033 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:38.472647 19075 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:38.473378 18800 raft_consensus.cc:2393] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:38.481411 19074 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.482278 19074 ts_tablet_manager.cc:1403] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.006s
I20260709 13:21:38.483809 18845 leader_election.cc:304] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:38.484834 19223 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.486101 19074 raft_consensus.cc:359] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.487296 19074 raft_consensus.cc:385] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.487747 19074 raft_consensus.cc:740] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.488672 19074 consensus_queue.cc:260] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.490931 19074 ts_tablet_manager.cc:1434] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:21:38.492255 19074 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.494410 19075 tablet_bootstrap.cc:654] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.496254 19223 raft_consensus.cc:2749] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:38.433573 19076 raft_consensus.cc:385] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.552059 19076 raft_consensus.cc:740] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.552309 19074 tablet_bootstrap.cc:654] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.552459 19073 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:38.552989 19076 consensus_queue.cc:260] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.499050 19229 raft_consensus.cc:515] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.559446 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:38.559626 19076 ts_tablet_manager.cc:1434] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.194s	user 0.007s	sys 0.002s
W20260709 13:21:38.561357 18917 leader_election.cc:343] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.560712 19229 leader_election.cc:290] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.562404 19073 ts_tablet_manager.cc:1403] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.130s	user 0.028s	sys 0.017s
I20260709 13:21:38.562592 19229 raft_consensus.cc:493] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.562433 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:38.563498 19076 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:38.563233 19229 raft_consensus.cc:515] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.565758 19322 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.566422 19322 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.563706 19027 raft_consensus.cc:3060] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.565760 19073 raft_consensus.cc:359] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.569629 19322 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:38.569969 19073 raft_consensus.cc:385] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.570330 19073 raft_consensus.cc:740] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.571192 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:38.571080 19073 consensus_queue.cc:260] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:38.575655 18917 leader_election.cc:343] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.577404 19074 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.578133 19074 ts_tablet_manager.cc:1403] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.086s	user 0.021s	sys 0.044s
I20260709 13:21:38.578622 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:38.580211 19130 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.575687 19073 ts_tablet_manager.cc:1434] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:21:38.580967 19130 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.583401 19075 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:38.583582 19076 tablet_bootstrap.cc:654] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.584112 19075 ts_tablet_manager.cc:1403] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.112s	user 0.030s	sys 0.008s
I20260709 13:21:38.584951 19130 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:38.581862 19074 raft_consensus.cc:359] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.587644 19074 raft_consensus.cc:385] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.587719 19075 raft_consensus.cc:359] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:38.588176 18917 leader_election.cc:343] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.588167 19074 raft_consensus.cc:740] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.591609 19075 raft_consensus.cc:385] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.591888 19075 raft_consensus.cc:740] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.589063 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.593948 18954 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:38.592578 19075 consensus_queue.cc:260] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.597184 19075 ts_tablet_manager.cc:1434] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:21:38.596398 19074 consensus_queue.cc:260] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.598563 19075 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:38.599157 19223 consensus_queue.cc:1048] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.599777 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:38.602135 19074 ts_tablet_manager.cc:1434] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.002s
W20260709 13:21:38.602183 18768 leader_election.cc:343] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.603670 19074 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.604570 18771 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.606127 19075 tablet_bootstrap.cc:654] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.606493 19229 leader_election.cc:290] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.606515 19314 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.607965 19314 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.616163 19073 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:38.618323 19083 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.619087 19083 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.611541 19314 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.607987 19130 raft_consensus.cc:2804] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.621703 19083 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:38.621961 19130 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.622490 19130 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.623726 19076 tablet_bootstrap.cc:492] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:38.623790 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.624591 19076 ts_tablet_manager.cc:1403] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.001s
I20260709 13:21:38.624891 18954 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:38.623318 19073 tablet_bootstrap.cc:654] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.627381 18994 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:38.628311 19076 raft_consensus.cc:359] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.628492 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.629189 19076 raft_consensus.cc:385] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.629547 18954 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:38.629810 19076 raft_consensus.cc:740] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.630239 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.631204 19028 raft_consensus.cc:2468] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:38.631088 19076 consensus_queue.cc:260] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.634616 19076 ts_tablet_manager.cc:1434] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 13:21:38.629876 19083 raft_consensus.cc:2804] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.636991 19083 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.635937 19314 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.643245 19076 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:38.646330 18771 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:38.650804 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.653070 19083 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.656344 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:38.659905 19088 raft_consensus.cc:2804] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.667255 18918 leader_election.cc:304] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.668330 19214 raft_consensus.cc:2804] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.668843 19214 raft_consensus.cc:493] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.669301 19214 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.666283 18800 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:38.670197 19088 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.670567 19074 tablet_bootstrap.cc:654] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.677727 19075 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:38.678753 19075 ts_tablet_manager.cc:1403] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.080s	user 0.021s	sys 0.013s
I20260709 13:21:38.670754 19088 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.683028 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:38.687304 19075 raft_consensus.cc:359] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.688333 19075 raft_consensus.cc:385] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.688668 19075 raft_consensus.cc:740] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.655905 19314 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.690871 19130 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.689566 19075 consensus_queue.cc:260] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.688040 18802 raft_consensus.cc:3060] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.711783 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:38.713755 18768 leader_election.cc:343] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.712852 19130 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:38.716137 18800 raft_consensus.cc:2393] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de62ff9486f7483e8f0e0b93dcca24bb in current term 1: Already voted for candidate 878e795a8488495f978538952e7b2324 in this term.
I20260709 13:21:38.717495 19075 ts_tablet_manager.cc:1434] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:21:38.718883 19075 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:38.728998 19073 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:38.730031 19073 ts_tablet_manager.cc:1403] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.010s
I20260709 13:21:38.731006 19314 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:38.732123 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:38.734724 18768 leader_election.cc:343] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.728549 19088 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.701361 19214 raft_consensus.cc:515] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.754907 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.749876 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:38.761047 18951 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.736310 19027 raft_consensus.cc:2468] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:38.763652 18953 raft_consensus.cc:2468] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:38.756201 19076 tablet_bootstrap.cc:654] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.757156 19083 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.768283 19074 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.769033 19074 ts_tablet_manager.cc:1403] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.166s	user 0.015s	sys 0.012s
I20260709 13:21:38.769655 18918 leader_election.cc:304] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.774698 19229 raft_consensus.cc:2804] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:38.775487 19229 raft_consensus.cc:697] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:38.777915 19073 raft_consensus.cc:359] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.779618 19073 raft_consensus.cc:385] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.780001 19024 raft_consensus.cc:2393] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 878e795a8488495f978538952e7b2324 in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:38.779835 18771 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:38.779997 19073 raft_consensus.cc:740] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.781810 19073 consensus_queue.cc:260] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.783802 19229 consensus_queue.cc:237] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.784575 19073 ts_tablet_manager.cc:1434] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:21:38.784266 19074 raft_consensus.cc:359] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.784787 19130 raft_consensus.cc:2804] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.785135 19074 raft_consensus.cc:385] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.785284 19130 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.785552 19074 raft_consensus.cc:740] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.785934 19073 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:38.786564 19074 consensus_queue.cc:260] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.789726 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:38.791725 19088 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.791489 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:38.792595 18952 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.793114 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:38.794131 18803 raft_consensus.cc:2393] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate 878e795a8488495f978538952e7b2324 in this term.
I20260709 13:21:38.794274 19083 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:38.794966 19074 ts_tablet_manager.cc:1434] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
W20260709 13:21:38.795466 18917 leader_election.cc:343] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.796357 19074 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.797245 19214 leader_election.cc:290] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.802289 18952 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:38.804557 19074 tablet_bootstrap.cc:654] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.807471 19073 tablet_bootstrap.cc:654] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.808166 19223 raft_consensus.cc:493] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.808844 19223 raft_consensus.cc:515] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:38.785769 19130 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.813128 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:38.813983 19024 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:38.770593 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 65957.2us
I20260709 13:21:38.817445 18994 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:38.818895 19083 raft_consensus.cc:2804] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:38.819500 19083 raft_consensus.cc:697] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:38.820376 19223 leader_election.cc:290] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:38.820652 19083 consensus_queue.cc:237] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.827173 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.828516 19075 tablet_bootstrap.cc:654] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.828794 19027 raft_consensus.cc:2393] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:38.832333 19076 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:38.833096 19076 ts_tablet_manager.cc:1403] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.190s	user 0.013s	sys 0.003s
I20260709 13:21:38.834574 18919 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:38.837244 19076 raft_consensus.cc:359] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.838851 19076 raft_consensus.cc:385] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.839371 19076 raft_consensus.cc:740] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.845670 19076 consensus_queue.cc:260] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.841959 19205 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.824510 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:38.857281 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:38.869086 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3f20d931bf640d09b913bc7a1e8e3a8" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:38.857280 19205 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.876478 19205 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:38.852154 19322 raft_consensus.cc:493] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.884884 19076 ts_tablet_manager.cc:1434] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.051s	user 0.013s	sys 0.000s
I20260709 13:21:38.886930 19205 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.892982 18952 raft_consensus.cc:2393] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 878e795a8488495f978538952e7b2324 in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:38.903610 19074 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:38.904354 19074 ts_tablet_manager.cc:1403] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.007s
I20260709 13:21:38.878002 19322 raft_consensus.cc:515] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:38.909324 19076 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:38.930629 19322 leader_election.cc:290] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:38.931550 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:38.932478 19027 raft_consensus.cc:2468] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:38.935750 18846 leader_election.cc:304] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:38.937358 18874 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.940121 19074 raft_consensus.cc:359] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.941083 19074 raft_consensus.cc:385] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:38.942618 19074 raft_consensus.cc:740] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:38.944648 19074 consensus_queue.cc:260] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:38.912058 19205 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:38.950493 18918 leader_election.cc:304] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:38.952963 19073 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:38.953924 19073 ts_tablet_manager.cc:1403] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.168s	user 0.018s	sys 0.000s
I20260709 13:21:38.957046 19074 ts_tablet_manager.cc:1434] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.001s
I20260709 13:21:38.926488 18951 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:38.963215 19229 raft_consensus.cc:2804] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:38.925983 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 48028.7us
I20260709 13:21:38.945129 19223 raft_consensus.cc:3060] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.977841 19076 tablet_bootstrap.cc:654] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.967310 19229 raft_consensus.cc:493] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.984203 19229 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.984001 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:38.989812 19074 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:38.967070 19214 raft_consensus.cc:2749] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:38.991915 19322 consensus_queue.cc:1048] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:38.992511 18880 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:38.995509 18768 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:21:38.967063 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:38.996686 19088 raft_consensus.cc:2804] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:38.997198 19088 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.997591 19088 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:38.987891 19205 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:38.974360 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:38.998792 18768 leader_election.cc:343] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:38.999892 19027 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:39.007795 18771 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:39.011848 19233 raft_consensus.cc:2804] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.012611 19233 raft_consensus.cc:697] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:39.013949 19233 consensus_queue.cc:237] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.001520 19024 raft_consensus.cc:2468] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:39.019212 19073 raft_consensus.cc:359] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.000267 18699 catalog_manager.cc:5697] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:39.021476 19073 raft_consensus.cc:385] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.022063 19073 raft_consensus.cc:740] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.023221 19073 consensus_queue.cc:260] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.029009 18918 leader_election.cc:304] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:39.030496 19363 raft_consensus.cc:2804] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.034559 19363 raft_consensus.cc:697] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:39.035449 18769 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:39.040297 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 50854.1us
I20260709 13:21:39.042237 19205 raft_consensus.cc:2749] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:39.043990 19363 consensus_queue.cc:237] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:39.044117 19184 consensus_queue.cc:1048] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:39.050628 19074 tablet_bootstrap.cc:654] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:38.982872 19130 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.055532 19130 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:39.057987 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:39.042385 18769 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:39.081104 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:39.081928 18951 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.085144 19223 raft_consensus.cc:2749] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:39.087379 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:39.089368 19364 raft_consensus.cc:2804] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:39.089987 19364 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:39.090137 18768 leader_election.cc:343] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.090528 19364 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.092464 19073 ts_tablet_manager.cc:1434] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.138s	user 0.010s	sys 0.004s
I20260709 13:21:39.093151 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:39.095909 19073 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:39.089818 19027 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
W20260709 13:21:39.103067 18919 leader_election.cc:343] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.109951 18951 raft_consensus.cc:2468] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:39.111838 19229 raft_consensus.cc:515] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.113123 18771 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:39.114507 19130 raft_consensus.cc:2804] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.115056 19130 raft_consensus.cc:697] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:39.116043 19130 consensus_queue.cc:237] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.120385 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:39.138743 19229 leader_election.cc:290] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:39.141800 19088 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:39.160065 18919 leader_election.cc:343] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.160506 19088 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:39.151607 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:39.164216 19027 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.164623 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:39.165305 19025 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.168934 19075 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:39.169816 19075 ts_tablet_manager.cc:1403] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.451s	user 0.030s	sys 0.042s
I20260709 13:21:39.144912 19364 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.176960 19074 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:39.182014 19074 ts_tablet_manager.cc:1403] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.207s	user 0.020s	sys 0.012s
I20260709 13:21:39.188540 19073 tablet_bootstrap.cc:654] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.196702 19364 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:39.225394 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:39.226264 19026 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.226824 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:39.235260 18875 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:39.248783 19075 raft_consensus.cc:359] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:39.266290 19075 raft_consensus.cc:385] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.267359 19075 raft_consensus.cc:740] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.268297 19075 consensus_queue.cc:260] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:39.272535 19027 raft_consensus.cc:2468] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:39.279376 19229 consensus_queue.cc:1048] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:39.278353 19083 consensus_queue.cc:1048] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:21:39.281917 19075 ts_tablet_manager.cc:1434] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.112s	user 0.009s	sys 0.000s
I20260709 13:21:39.299535 19075 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:39.284236 19074 raft_consensus.cc:359] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.306524 19074 raft_consensus.cc:385] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.306914 19074 raft_consensus.cc:740] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.308080 19074 consensus_queue.cc:260] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.294384 18875 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:39.315614 18918 leader_election.cc:304] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:39.321346 19074 ts_tablet_manager.cc:1434] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.139s	user 0.010s	sys 0.001s
I20260709 13:21:39.287345 19025 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:39.289800 19026 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:39.326292 18769 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:39.330968 18769 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:39.322799 19229 raft_consensus.cc:2804] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.336830 19074 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:39.332291 19088 raft_consensus.cc:2804] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.334251 19205 raft_consensus.cc:2804] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:39.350191 19076 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:39.350898 19076 ts_tablet_manager.cc:1403] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.454s	user 0.021s	sys 0.031s
I20260709 13:21:39.337779 19229 raft_consensus.cc:697] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:39.356293 19229 consensus_queue.cc:237] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.340440 19311 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:39.357820 19311 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.369556 19205 raft_consensus.cc:697] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:39.371093 19205 consensus_queue.cc:237] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.369482 19088 raft_consensus.cc:697] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:39.380280 19088 consensus_queue.cc:237] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.389432 19130 raft_consensus.cc:493] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.390096 19130 raft_consensus.cc:515] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.392146 19311 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:39.393973 19130 leader_election.cc:290] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:39.392153 19076 raft_consensus.cc:359] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.397017 19076 raft_consensus.cc:385] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.398412 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:39.398545 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:39.397976 18702 catalog_manager.cc:5697] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:39.397439 19076 raft_consensus.cc:740] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.406885 19076 consensus_queue.cc:260] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:39.419173 19083 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.419785 19083 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.421120 19076 ts_tablet_manager.cc:1434] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.001s
I20260709 13:21:39.426877 19074 tablet_bootstrap.cc:654] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.427470 19073 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:39.428220 19073 ts_tablet_manager.cc:1403] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.333s	user 0.017s	sys 0.013s
I20260709 13:21:39.430876 19075 tablet_bootstrap.cc:654] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:39.417430 18771 leader_election.cc:343] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.436445 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:39.418571 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:39.442277 19083 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:39.443540 18703 catalog_manager.cc:5697] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:39.453008 19083 consensus_queue.cc:1048] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:39.461019 18993 leader_election.cc:343] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.465217 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:39.467629 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:39.469432 19322 consensus_queue.cc:1048] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:39.456451 18846 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:39.475641 19380 raft_consensus.cc:2749] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:39.478118 19073 raft_consensus.cc:359] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.479519 19073 raft_consensus.cc:385] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.479919 19073 raft_consensus.cc:740] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.480954 19073 consensus_queue.cc:260] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:39.487802 18768 leader_election.cc:343] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.493775 18768 leader_election.cc:304] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
W20260709 13:21:39.495680 18768 leader_election.cc:343] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.495877 19233 raft_consensus.cc:2749] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:39.500577 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:39.513998 19075 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:39.514678 19075 ts_tablet_manager.cc:1403] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.217s	user 0.017s	sys 0.001s
I20260709 13:21:39.516057 19073 ts_tablet_manager.cc:1434] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.087s	user 0.010s	sys 0.001s
I20260709 13:21:39.516531 19074 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:39.517153 19074 ts_tablet_manager.cc:1403] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.183s	user 0.026s	sys 0.005s
W20260709 13:21:39.515765 18991 leader_election.cc:343] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:39.518445 18991 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:39.527415 19075 raft_consensus.cc:359] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.528146 19075 raft_consensus.cc:385] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.528395 19075 raft_consensus.cc:740] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.529078 19075 consensus_queue.cc:260] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:39.532982 19075 ts_tablet_manager.cc:1434] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.006s
I20260709 13:21:39.537719 19075 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:39.538532 19083 raft_consensus.cc:2749] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:39.540416 19076 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:39.551522 19073 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:39.550252 19074 raft_consensus.cc:359] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.565371 19074 raft_consensus.cc:385] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:39.565799 19074 raft_consensus.cc:740] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:39.535246 19130 consensus_queue.cc:1048] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
W20260709 13:21:39.575006 18917 consensus_peers.cc:597] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.566593 19074 consensus_queue.cc:260] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:39.581067 19184 raft_consensus.cc:493] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.592147 19074 ts_tablet_manager.cc:1434] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.002s
I20260709 13:21:39.604650 19074 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:39.609601 19076 tablet_bootstrap.cc:654] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.612797 19074 tablet_bootstrap.cc:654] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.598798 19184 raft_consensus.cc:515] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:39.638860 19205 consensus_queue.cc:1048] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:39.659996 19184 leader_election.cc:290] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:39.903865 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 201705us
I20260709 13:21:39.949257 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:39.962512 19075 tablet_bootstrap.cc:654] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:39.966503 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:39.976044 19398 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.977232 19398 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.055330 19398 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:40.062498 19073 tablet_bootstrap.cc:654] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.087672 19074 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:40.088585 19074 ts_tablet_manager.cc:1403] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.484s	user 0.050s	sys 0.030s
W20260709 13:21:40.099220 18802 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.015s	sys 0.003s
W20260709 13:21:40.116690 18802 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.015s	sys 0.003s
I20260709 13:21:40.098135 19074 raft_consensus.cc:359] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.113256 18703 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48490 (request call id 62) took 1088 ms. Trace:
I20260709 13:21:40.118372 18703 rpcz_store.cc:276] 0709 13:21:39.024710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.024912 (+   202us) service_pool.cc:224] Handling call
0709 13:21:40.113139 (+1088227us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:39.594428 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:39.661544 (+ 67116us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:39.661606 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:21:39.661639 (+    33us) tablet.cc:662] Decoding operations
0709 13:21:39.900464 (+238825us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:39.911489 (+ 11025us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:39.913049 (+  1560us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:39.928795 (+ 15746us) tablet.cc:701] Row locks acquired
0709 13:21:39.928818 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:21:39.945240 (+ 16422us) write_op.cc:270] Start()
0709 13:21:39.951926 (+  6686us) write_op.cc:276] Timestamp: P: 1783603299943020 usec, L: 0
0709 13:21:39.951981 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:21:40.003018 (+ 51037us) log.cc:844] Serialized 537 byte log entry
0709 13:21:40.006416 (+  3398us) op_driver.cc:464] REPLICATION: finished
0709 13:21:40.007253 (+   837us) write_op.cc:301] APPLY: starting
0709 13:21:40.007367 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:40.007536 (+   169us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:40.007736 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:40.008202 (+   466us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:40.008462 (+   260us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:21:40.008542 (+    80us) write_op.cc:312] APPLY: finished
0709 13:21:40.008872 (+   330us) log.cc:844] Serialized 22 byte log entry
0709 13:21:40.009146 (+   274us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:40.009300 (+   154us) write_op.cc:454] Released schema lock
0709 13:21:40.009361 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:40.009875 (+   514us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:40.009922 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":192442,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":500,"apply.run_cpu_time_us":2553,"apply.run_wall_time_us":2551,"num_ops":1,"prepare.queue_time_us":14516,"prepare.run_cpu_time_us":21775,"prepare.run_wall_time_us":409120,"replication_time_us":34744}]]}
I20260709 13:21:40.113387 19322 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.123232 19322 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.124675 19365 consensus_queue.cc:1048] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:40.126132 19322 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:40.127163 19074 raft_consensus.cc:385] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:21:40.127954 18918 leader_election.cc:343] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.128523 19074 raft_consensus.cc:740] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.131173 19074 consensus_queue.cc:260] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.129882 19322 consensus_queue.cc:1048] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:40.140136 18917 leader_election.cc:343] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.142196 18917 leader_election.cc:304] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:40.143215 19402 raft_consensus.cc:2749] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:40.145897 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:40.146966 19025 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:40.162360 19223 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.182264 19223 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.186827 18918 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:40.190793 19223 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:40.192446 19073 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:40.199251 19073 ts_tablet_manager.cc:1403] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.655s	user 0.026s	sys 0.014s
I20260709 13:21:40.199401 19074 ts_tablet_manager.cc:1434] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.110s	user 0.020s	sys 0.008s
I20260709 13:21:40.200649 19402 raft_consensus.cc:2804] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.201362 19402 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.201799 19402 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:40.212108 19401 log.cc:927] Time spent T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Append to log took a long time: real 0.282s	user 0.005s	sys 0.000s
I20260709 13:21:40.223976 19073 raft_consensus.cc:359] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.225095 19073 raft_consensus.cc:385] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.225451 19073 raft_consensus.cc:740] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.226416 19073 consensus_queue.cc:260] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.236032 19076 tablet_bootstrap.cc:492] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:40.236685 19073 ts_tablet_manager.cc:1434] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.002s
I20260709 13:21:40.236726 19076 ts_tablet_manager.cc:1403] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.697s	user 0.012s	sys 0.013s
I20260709 13:21:40.242497 19402 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.245575 19402 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:40.248385 19075 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:40.249081 19075 ts_tablet_manager.cc:1403] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.712s	user 0.027s	sys 0.040s
I20260709 13:21:40.250118 19074 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:40.258888 19075 raft_consensus.cc:359] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.259899 19075 raft_consensus.cc:385] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.260282 19075 raft_consensus.cc:740] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.261102 19075 consensus_queue.cc:260] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.264110 19073 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:40.268383 19075 ts_tablet_manager.cc:1434] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.001s
I20260709 13:21:40.276933 19076 raft_consensus.cc:359] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.278863 19076 raft_consensus.cc:385] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.279155 19076 raft_consensus.cc:740] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.279810 19076 consensus_queue.cc:260] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.294943 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.296036 18801 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:40.322075 19414 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.323239 19414 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.327747 19415 raft_consensus.cc:493] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:40.328543 18768 consensus_peers.cc:597] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.328522 19415 raft_consensus.cc:515] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.331929 19415 leader_election.cc:290] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:40.337368 18798 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.344303 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.344671 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:40.345369 18800 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:40.353440 18879 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.360901 19414 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.363238 18874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:40.365494 18768 leader_election.cc:343] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.367894 19414 consensus_queue.cc:1048] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:40.372604 19409 consensus_queue.cc:1048] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:40.414453 18798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.415546 18993 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:40.418682 19396 raft_consensus.cc:2804] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.419310 19396 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.419792 19396 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.431388 19414 consensus_queue.cc:1048] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:40.450480 19396 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.455719 19396 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:40.456979 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:40.457844 18801 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.473197 18801 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:40.474864 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:40.476056 18952 raft_consensus.cc:2393] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:40.475929 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:40.476926 18953 raft_consensus.cc:2393] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3e2fa7cf3974c27a19386f362688d29 in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
W20260709 13:21:40.482841 18768 consensus_peers.cc:597] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.474884 18993 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:40.492463 19396 raft_consensus.cc:2804] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:40.493373 19396 raft_consensus.cc:697] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:40.494450 19396 consensus_queue.cc:237] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:40.580541 19073 tablet_bootstrap.cc:654] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.587598 19073 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:40.588244 19073 ts_tablet_manager.cc:1403] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.325s	user 0.016s	sys 0.003s
I20260709 13:21:40.591596 19073 raft_consensus.cc:359] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.592460 19073 raft_consensus.cc:385] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.592767 19073 raft_consensus.cc:740] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.593735 19073 consensus_queue.cc:260] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.594585 19075 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:40.595736 19073 ts_tablet_manager.cc:1434] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:40.597224 19073 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:40.598476 18699 catalog_manager.cc:5697] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:40.602301 19363 raft_consensus.cc:493] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.603330 19363 raft_consensus.cc:515] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.607519 19073 tablet_bootstrap.cc:654] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.608202 19076 ts_tablet_manager.cc:1434] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.371s	user 0.011s	sys 0.001s
I20260709 13:21:40.609668 19076 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:40.607560 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.611794 18804 raft_consensus.cc:2468] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:40.616066 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:40.619374 18917 leader_election.cc:343] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.620007 19323 raft_consensus.cc:493] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.620795 19363 leader_election.cc:290] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.620688 19323 raft_consensus.cc:515] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.623790 19076 tablet_bootstrap.cc:654] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.623528 19323 leader_election.cc:290] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:40.629989 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:40.631441 19023 raft_consensus.cc:2468] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:40.632792 19075 tablet_bootstrap.cc:654] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.644050 18919 leader_election.cc:304] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:40.646564 19074 tablet_bootstrap.cc:654] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.646673 19184 raft_consensus.cc:2804] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.647244 19184 raft_consensus.cc:493] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.647650 19184 raft_consensus.cc:3060] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.658609 19088 consensus_queue.cc:1048] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:40.658828 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.661592 19184 raft_consensus.cc:515] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.662935 18769 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:40.664117 19088 raft_consensus.cc:2804] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:40.664714 19088 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.665202 19088 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.665494 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:40.665668 19419 consensus_queue.cc:1048] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:40.666348 18804 raft_consensus.cc:3060] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.667640 19425 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:40.668339 19425 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.670545 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:40.666877 18991 leader_election.cc:343] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.673514 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.679972 19392 raft_consensus.cc:493] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.689885 19073 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:40.690788 19073 ts_tablet_manager.cc:1403] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.010s
I20260709 13:21:40.696959 19076 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:40.698776 19076 ts_tablet_manager.cc:1403] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.089s	user 0.022s	sys 0.009s
I20260709 13:21:40.689101 19392 raft_consensus.cc:515] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.718618 19088 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.721179 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:40.723165 18991 leader_election.cc:343] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.726084 19074 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:40.726689 19074 ts_tablet_manager.cc:1403] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.477s	user 0.011s	sys 0.007s
I20260709 13:21:40.700356 19425 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.700091 19184 leader_election.cc:290] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:40.748977 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:40.733418 18877 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.733556 19392 leader_election.cc:290] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:40.736243 18702 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48472 (request call id 54) took 1444 ms. Trace:
I20260709 13:21:40.737056 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:40.753356 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 271241us
I20260709 13:21:40.748212 18699 catalog_manager.cc:5697] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:40.755959 18698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48480 (request call id 55) took 1271 ms. Trace:
I20260709 13:21:40.756510 18698 rpcz_store.cc:276] 0709 13:21:39.486555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.486799 (+   244us) service_pool.cc:224] Handling call
0709 13:21:40.755870 (+1269071us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230630,"reported_tablets":2}
I20260709 13:21:40.757951 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:40.760015 19075 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:40.760792 19075 ts_tablet_manager.cc:1403] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.167s	user 0.014s	sys 0.007s
I20260709 13:21:40.761629 19425 raft_consensus.cc:493] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 039c551e55fb43cdb5c3880b7cb91a0b)
I20260709 13:21:40.762306 19425 raft_consensus.cc:515] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.763170 19398 raft_consensus.cc:493] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 039c551e55fb43cdb5c3880b7cb91a0b)
I20260709 13:21:40.763837 19398 raft_consensus.cc:515] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.765833 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.766242 19398 leader_election.cc:290] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.702912 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.760107 19074 raft_consensus.cc:359] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.716548 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:40.802248 19074 raft_consensus.cc:385] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.802770 19074 raft_consensus.cc:740] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.766891 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:40.748426 19073 raft_consensus.cc:359] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.745018 19076 raft_consensus.cc:359] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.804718 19425 leader_election.cc:290] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:40.805449 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:40.804028 18917 leader_election.cc:343] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.804474 19398 raft_consensus.cc:493] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 039c551e55fb43cdb5c3880b7cb91a0b)
I20260709 13:21:40.810076 19398 raft_consensus.cc:515] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.812781 19398 leader_election.cc:290] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:40.814803 18991 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:40.816988 19398 raft_consensus.cc:2749] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.817425 18804 raft_consensus.cc:2468] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:40.818367 19088 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.804076 19184 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:40.818922 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:40.820883 19075 raft_consensus.cc:359] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.828915 19075 raft_consensus.cc:385] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.829506 19075 raft_consensus.cc:740] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.752556 18702 rpcz_store.cc:276] 0709 13:21:39.292282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.292500 (+   218us) service_pool.cc:224] Handling call
0709 13:21:39.351604 (+ 59104us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b5400960854. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa27800cf3a 00007fa27ddc31cc 00007fa27dec137e 00007fa27c3822a4 00007fa27c382232 00007fa27c382161 00007fa27c3820dc 00007fa27c381392 00007fa27633f48c 00007fa27633e8c6
0709 13:21:40.736152 (+1384548us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:39.583996 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:39.591984 (+  7988us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:39.592027 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:21:39.592045 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:39.592487 (+   442us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:39.592545 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:39.592573 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:39.592819 (+   246us) tablet.cc:701] Row locks acquired
0709 13:21:39.592838 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:21:39.593019 (+   181us) write_op.cc:270] Start()
0709 13:21:39.593163 (+   144us) write_op.cc:276] Timestamp: P: 1783603299592965 usec, L: 0
0709 13:21:39.593195 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:21:39.593822 (+   627us) log.cc:844] Serialized 537 byte log entry
0709 13:21:39.615987 (+ 22165us) op_driver.cc:464] REPLICATION: finished
0709 13:21:39.621072 (+  5085us) write_op.cc:301] APPLY: starting
0709 13:21:39.627787 (+  6715us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:39.635615 (+  7828us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:39.635813 (+   198us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:39.636253 (+   440us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:39.636462 (+   209us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:21:39.636524 (+    62us) write_op.cc:312] APPLY: finished
0709 13:21:39.636796 (+   272us) log.cc:844] Serialized 22 byte log entry
0709 13:21:39.637054 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:39.637185 (+   131us) write_op.cc:454] Released schema lock
0709 13:21:39.637245 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:39.637710 (+   465us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:39.637746 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":169839,"reported_tablets":5,"spinlock_wait_cycles":106852608,"child_traces":[["op",{"apply.queue_time_us":198,"apply.run_cpu_time_us":2601,"apply.run_wall_time_us":16621,"num_ops":1,"prepare.queue_time_us":2357,"prepare.run_cpu_time_us":3150,"prepare.run_wall_time_us":11974,"replication_time_us":22647}]]}
I20260709 13:21:40.803611 19074 consensus_queue.cc:260] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.807029 19076 raft_consensus.cc:385] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.831005 19076 raft_consensus.cc:740] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.832069 19076 consensus_queue.cc:260] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.832629 19075 consensus_queue.cc:260] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.824067 19073 raft_consensus.cc:385] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:40.834040 19073 raft_consensus.cc:740] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:40.846474 19075 ts_tablet_manager.cc:1434] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.006s
I20260709 13:21:40.841032 19073 consensus_queue.cc:260] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:40.823272 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccb9f9f9dbb04b459b94b15ddb6864d5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.835135 19184 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:40.831453 18954 raft_consensus.cc:2468] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:40.858408 19184 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:40.867980 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:40.868952 19074 ts_tablet_manager.cc:1434] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.142s	user 0.009s	sys 0.001s
I20260709 13:21:40.869480 18878 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.870515 19074 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:40.872685 19076 ts_tablet_manager.cc:1434] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.173s	user 0.004s	sys 0.005s
I20260709 13:21:40.873595 19073 ts_tablet_manager.cc:1434] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.182s	user 0.010s	sys 0.000s
I20260709 13:21:40.886523 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.906136 19076 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:40.911247 19073 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:40.922482 19075 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:40.926925 18991 leader_election.cc:304] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:40.915799 18919 leader_election.cc:304] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:40.931891 19365 raft_consensus.cc:2804] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:40.932499 19365 raft_consensus.cc:697] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:40.911103 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:40.933532 19365 consensus_queue.cc:237] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:40.934466 18768 leader_election.cc:343] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.904909 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:40.942731 19022 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:40.943810 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:40.944113 19392 raft_consensus.cc:2749] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:40.942113 19088 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.953496 19088 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:40.930980 19074 tablet_bootstrap.cc:654] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.976975 19088 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:40.989974 18994 leader_election.cc:343] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:40.994423 19075 tablet_bootstrap.cc:654] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.998149 19020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:41.001631 19073 tablet_bootstrap.cc:654] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:40.990885 18994 leader_election.cc:304] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:41.004914 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:41.007539 18769 leader_election.cc:343] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.009833 19396 raft_consensus.cc:2749] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.010521 19076 tablet_bootstrap.cc:654] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.012032 18994 leader_election.cc:304] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
W20260709 13:21:41.020567 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 119541us
I20260709 13:21:41.025020 18918 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:41.023777 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 115355us
W20260709 13:21:41.025789 18845 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 147529us
I20260709 13:21:41.031455 19409 raft_consensus.cc:2749] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.031562 19398 raft_consensus.cc:2804] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.061605 19398 raft_consensus.cc:493] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.062111 19398 raft_consensus.cc:3060] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.058790 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "211a9a1cbd3f4dca903a20b4ecb32209" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:41.059749 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c692689ede48466a882155f06b0763ad" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:41.089715 18771 leader_election.cc:304] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:41.073513 19229 raft_consensus.cc:493] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.062788 18846 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:41.091786 19414 raft_consensus.cc:2749] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.124734 18994 leader_election.cc:304] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:41.126950 18994 leader_election.cc:304] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
W20260709 13:21:41.163865 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 87777.5us
I20260709 13:21:41.105310 19229 raft_consensus.cc:515] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.158246 19323 raft_consensus.cc:2749] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.165758 19426 raft_consensus.cc:2749] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.164223 19446 raft_consensus.cc:2749] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:41.158092 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 95420.3us
I20260709 13:21:41.173563 19380 raft_consensus.cc:493] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de62ff9486f7483e8f0e0b93dcca24bb)
I20260709 13:21:41.200577 19229 leader_election.cc:290] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:41.179172 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 52373.8us
I20260709 13:21:41.212570 19022 raft_consensus.cc:2468] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:41.184870 19380 raft_consensus.cc:515] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.233803 19398 raft_consensus.cc:515] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.245635 19075 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:41.249523 19074 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:41.249922 19075 ts_tablet_manager.cc:1403] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.334s	user 0.017s	sys 0.006s
I20260709 13:21:41.249708 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:41.250381 19074 ts_tablet_manager.cc:1403] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.380s	user 0.017s	sys 0.013s
I20260709 13:21:41.253618 19075 raft_consensus.cc:359] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.254673 19075 raft_consensus.cc:385] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.255074 19075 raft_consensus.cc:740] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.258406 19452 raft_consensus.cc:493] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.259040 19452 raft_consensus.cc:515] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.260221 19380 leader_election.cc:290] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:41.256992 19075 consensus_queue.cc:260] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.261628 19452 leader_election.cc:290] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:41.268519 19396 consensus_queue.cc:1048] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.273998 19073 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:41.274838 19073 ts_tablet_manager.cc:1403] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.367s	user 0.020s	sys 0.000s
I20260709 13:21:41.284600 19074 raft_consensus.cc:359] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.286198 19074 raft_consensus.cc:385] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.286993 19074 raft_consensus.cc:740] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.288785 18702 catalog_manager.cc:5697] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:41.294967 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:41.295459 18699 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48488 (request call id 58) took 1185 ms. Trace:
W20260709 13:21:41.298063 18826 debug-util.cc:398] Leaking SignalData structure 0x7b08002ac5e0 after lost signal to thread 8928
I20260709 13:21:41.268590 19398 leader_election.cc:290] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:41.300753 19076 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:41.312386 19076 ts_tablet_manager.cc:1403] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.412s	user 0.025s	sys 0.007s
I20260709 13:21:41.316310 19074 consensus_queue.cc:260] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.318614 19075 ts_tablet_manager.cc:1434] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.003s
I20260709 13:21:41.319509 19074 ts_tablet_manager.cc:1434] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 13:21:41.319698 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:41.320210 19075 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:41.320815 19074 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:41.301600 18826 debug-util.cc:398] Leaking SignalData structure 0x7b08002be780 after lost signal to thread 18737
W20260709 13:21:41.332115 18826 debug-util.cc:398] Leaking SignalData structure 0x7b080023cae0 after lost signal to thread 18829
W20260709 13:21:41.330844 18991 leader_election.cc:343] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.332814 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:41.333940 18826 debug-util.cc:398] Leaking SignalData structure 0x7b0800202b80 after lost signal to thread 18904
I20260709 13:21:41.334229 18801 raft_consensus.cc:2468] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:41.308336 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:41.336972 18951 raft_consensus.cc:2468] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
W20260709 13:21:41.308779 18768 leader_election.cc:343] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.338266 18768 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:41.336958 18702 catalog_manager.cc:5697] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } }
W20260709 13:21:41.336012 18826 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8a20 after lost signal to thread 18978
W20260709 13:21:41.361371 18826 debug-util.cc:398] Leaking SignalData structure 0x7b080034bd40 after lost signal to thread 18979
I20260709 13:21:41.333316 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:41.339521 19074 tablet_bootstrap.cc:654] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.311659 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:41.301303 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:41.343611 19418 raft_consensus.cc:2749] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.345124 18769 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
W20260709 13:21:41.366312 18991 leader_election.cc:343] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.298025 18699 rpcz_store.cc:276] 0709 13:21:40.110570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.111311 (+   741us) service_pool.cc:224] Handling call
0709 13:21:41.295365 (+1184054us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:40.948810 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:40.949485 (+   675us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:40.949529 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:21:40.949549 (+    20us) tablet.cc:662] Decoding operations
0709 13:21:40.949979 (+   430us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:40.950076 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:40.950103 (+    27us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:21:40.950439 (+   336us) tablet.cc:701] Row locks acquired
0709 13:21:40.950457 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:21:40.950608 (+   151us) write_op.cc:270] Start()
0709 13:21:40.950736 (+   128us) write_op.cc:276] Timestamp: P: 1783603300950565 usec, L: 0
0709 13:21:40.950757 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:21:41.003505 (+ 52748us) log.cc:844] Serialized 922 byte log entry
0709 13:21:41.046402 (+ 42897us) op_driver.cc:464] REPLICATION: finished
0709 13:21:41.208120 (+161718us) write_op.cc:301] APPLY: starting
0709 13:21:41.208249 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:41.208434 (+   185us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:41.208619 (+   185us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:41.209040 (+   421us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:41.209256 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:21:41.209299 (+    43us) write_op.cc:312] APPLY: finished
0709 13:21:41.209548 (+   249us) log.cc:844] Serialized 28 byte log entry
0709 13:21:41.209758 (+   210us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:41.209873 (+   115us) write_op.cc:454] Released schema lock
0709 13:21:41.209917 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:41.211217 (+  1300us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:41.211246 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":271191,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":159956,"apply.run_cpu_time_us":2291,"apply.run_wall_time_us":3088,"num_ops":2,"prepare.queue_time_us":185724,"prepare.run_cpu_time_us":3681,"prepare.run_wall_time_us":55066,"replication_time_us":95440,"thread_start_us":262127,"threads_started":2}]]}
I20260709 13:21:41.302623 18917 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:41.306649 19073 raft_consensus.cc:359] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.374507 19073 raft_consensus.cc:385] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.374919 19073 raft_consensus.cc:740] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.377081 19073 consensus_queue.cc:260] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.379530 19076 raft_consensus.cc:359] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.380381 19076 raft_consensus.cc:385] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.380693 19076 raft_consensus.cc:740] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.384460 19440 raft_consensus.cc:2804] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.385169 19440 raft_consensus.cc:697] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:41.386109 19440 consensus_queue.cc:237] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.388134 18846 leader_election.cc:304] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:41.389513 19380 raft_consensus.cc:2749] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:41.382597 18919 leader_election.cc:304] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:41.396214 19363 raft_consensus.cc:2749] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:41.379017 18826 debug-util.cc:398] Leaking SignalData structure 0x7b08000969a0 after lost signal to thread 19052
I20260709 13:21:41.405455 19076 consensus_queue.cc:260] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.417099 19073 ts_tablet_manager.cc:1434] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.142s	user 0.007s	sys 0.003s
I20260709 13:21:41.428260 18703 catalog_manager.cc:5697] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } }
I20260709 13:21:41.437031 19073 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:41.421983 19407 raft_consensus.cc:2804] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.438740 19407 raft_consensus.cc:493] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.439208 19407 raft_consensus.cc:3060] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.442278 19076 ts_tablet_manager.cc:1434] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.000s
I20260709 13:21:41.443720 19076 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
W20260709 13:21:41.407301 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 104842us
I20260709 13:21:41.451956 19075 tablet_bootstrap.cc:654] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.473434 19311 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.474154 19311 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:41.474835 18917 leader_election.cc:343] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.478171 18994 leader_election.cc:304] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:41.482197 19396 raft_consensus.cc:2804] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.482717 19396 raft_consensus.cc:493] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.483145 19396 raft_consensus.cc:3060] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.479982 19461 consensus_queue.cc:1048] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.494408 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:41.495605 18950 raft_consensus.cc:3060] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.498283 19074 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:41.499761 19074 ts_tablet_manager.cc:1403] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.179s	user 0.018s	sys 0.010s
I20260709 13:21:41.510756 19074 raft_consensus.cc:359] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.511690 19074 raft_consensus.cc:385] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.512900 19074 raft_consensus.cc:740] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.533732 19074 consensus_queue.cc:260] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.537263 19074 ts_tablet_manager.cc:1434] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:21:41.550974 19073 tablet_bootstrap.cc:654] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.554124 19074 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:41.567972 19076 tablet_bootstrap.cc:654] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.589707 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:41.581712 19311 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:41.608882 19396 raft_consensus.cc:515] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.612589 19074 tablet_bootstrap.cc:654] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.681363 19073 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:41.682475 19073 ts_tablet_manager.cc:1403] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.261s	user 0.024s	sys 0.010s
I20260709 13:21:41.689163 19396 leader_election.cc:290] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:41.692492 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:41.703008 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:41.698653 18950 raft_consensus.cc:2468] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:41.730499 19075 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:41.731271 19075 ts_tablet_manager.cc:1403] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.411s	user 0.060s	sys 0.052s
I20260709 13:21:41.727234 18949 raft_consensus.cc:3060] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.716336 19456 consensus_queue.cc:1048] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
W20260709 13:21:41.734400 18991 leader_election.cc:343] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.744012 18804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:41.763823 19392 raft_consensus.cc:493] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.777479 18872 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.770669 19392 raft_consensus.cc:515] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.782390 19392 leader_election.cc:290] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:41.743495 19407 raft_consensus.cc:515] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.792912 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:41.793856 18799 raft_consensus.cc:2468] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
I20260709 13:21:41.793902 18845 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:41.795315 18917 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.801280 18993 leader_election.cc:304] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
W20260709 13:21:41.805474 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 76316.6us
I20260709 13:21:41.806545 19396 raft_consensus.cc:2804] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.812692 19396 raft_consensus.cc:493] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.813175 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:41.813759 18994 leader_election.cc:304] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:41.815076 19398 raft_consensus.cc:2804] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:41.815683 19398 raft_consensus.cc:697] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:41.816645 19398 consensus_queue.cc:237] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.813215 19396 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.820528 19419 consensus_queue.cc:1048] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:41.825161 19407 leader_election.cc:290] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:41.828994 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:41.829953 18799 raft_consensus.cc:3060] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.831912 19073 raft_consensus.cc:359] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.832835 19073 raft_consensus.cc:385] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.833124 19073 raft_consensus.cc:740] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.833956 19073 consensus_queue.cc:260] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.844781 19074 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:41.845781 19074 ts_tablet_manager.cc:1403] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.307s	user 0.028s	sys 0.036s
I20260709 13:21:41.851104 18949 raft_consensus.cc:2468] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:41.870651 18799 raft_consensus.cc:2468] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
W20260709 13:21:41.872294 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 59895.5us
I20260709 13:21:41.873716 19075 raft_consensus.cc:359] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.874608 19075 raft_consensus.cc:385] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.874917 19075 raft_consensus.cc:740] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.875730 19075 consensus_queue.cc:260] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:41.876515 19076 tablet_bootstrap.cc:492] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:41.877082 19076 ts_tablet_manager.cc:1403] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.434s	user 0.009s	sys 0.023s
I20260709 13:21:41.880088 19392 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.880918 19392 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:41.882371 19076 raft_consensus.cc:359] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.883481 19076 raft_consensus.cc:385] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.883852 19076 raft_consensus.cc:740] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.855368 19073 ts_tablet_manager.cc:1434] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.172s	user 0.006s	sys 0.004s
I20260709 13:21:41.884649 19405 raft_consensus.cc:2749] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:41.885279 19392 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:41.885955 19073 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:41.855253 19074 raft_consensus.cc:359] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.888056 19074 raft_consensus.cc:385] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:41.888437 19074 raft_consensus.cc:740] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:41.889442 19074 consensus_queue.cc:260] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:41.894479 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:41.895704 18799 raft_consensus.cc:2468] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
W20260709 13:21:41.896023 18768 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.897367 18993 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:41.897320 19076 consensus_queue.cc:260] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:41.898124 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:41.900074 19076 ts_tablet_manager.cc:1434] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 13:21:41.895725 19074 ts_tablet_manager.cc:1434] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.003s
I20260709 13:21:41.901577 19076 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:41.901954 19074 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:41.904044 19398 raft_consensus.cc:2804] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:41.911541 19398 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:41.911968 19398 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:41.916307 18698 catalog_manager.cc:5697] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:41.931479 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 64837us
I20260709 13:21:41.933115 18919 leader_election.cc:304] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
W20260709 13:21:41.939889 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 66116.4us
I20260709 13:21:41.967322 19073 tablet_bootstrap.cc:654] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:41.967325 19076 tablet_bootstrap.cc:654] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:41.973429 18991 leader_election.cc:343] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:41.973970 19074 tablet_bootstrap.cc:654] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:41.987061 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:41.990447 19073 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:41.993044 19073 ts_tablet_manager.cc:1403] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.107s	user 0.018s	sys 0.004s
I20260709 13:21:42.001891 19453 raft_consensus.cc:2804] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.002900 19453 raft_consensus.cc:697] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:42.008937 19453 consensus_queue.cc:237] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:42.016707 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 43564.6us
I20260709 13:21:42.025250 19075 ts_tablet_manager.cc:1434] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.293s	user 0.017s	sys 0.006s
I20260709 13:21:42.036680 19405 raft_consensus.cc:493] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.037484 19405 raft_consensus.cc:515] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:42.041924 19405 leader_election.cc:290] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:42.049949 19076 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:42.049544 19021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:42.050648 19076 ts_tablet_manager.cc:1403] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.149s	user 0.012s	sys 0.003s
I20260709 13:21:42.051517 19075 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:42.058790 19074 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:42.059669 19074 ts_tablet_manager.cc:1403] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.158s	user 0.015s	sys 0.010s
I20260709 13:21:41.997473 19073 raft_consensus.cc:359] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.061720 19073 raft_consensus.cc:385] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.062114 19073 raft_consensus.cc:740] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
W20260709 13:21:42.071158 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 130116us
W20260709 13:21:42.072790 18994 leader_election.cc:343] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.075945 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:42.069237 19073 consensus_queue.cc:260] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.086068 19074 raft_consensus.cc:359] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.091495 19074 raft_consensus.cc:385] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.080401 19396 raft_consensus.cc:515] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.092562 18994 leader_election.cc:304] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:42.093689 19392 raft_consensus.cc:2804] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.094264 19392 raft_consensus.cc:697] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:42.095249 19392 consensus_queue.cc:237] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:42.096261 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:42.097085 18799 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.089870 19076 raft_consensus.cc:359] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.099933 19076 raft_consensus.cc:385] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.104935 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4490b378f034620bcb7df67470bdab9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:42.088641 19073 ts_tablet_manager.cc:1434] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.002s
I20260709 13:21:42.092186 19074 raft_consensus.cc:740] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.112358 18950 raft_consensus.cc:2393] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate f3e2fa7cf3974c27a19386f362688d29 in this term.
I20260709 13:21:42.115592 19073 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:42.124699 19074 consensus_queue.cc:260] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.111833 19076 raft_consensus.cc:740] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.115715 19398 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.130802 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:42.131886 18802 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:42.136078 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 44607.2us
I20260709 13:21:42.138088 19396 leader_election.cc:290] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:42.139951 18846 leader_election.cc:304] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:42.141552 19380 raft_consensus.cc:3060] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.143496 19076 consensus_queue.cc:260] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.145394 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:42.146026 19076 ts_tablet_manager.cc:1434] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.095s	user 0.002s	sys 0.009s
I20260709 13:21:42.146216 18952 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.147362 19076 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:42.152786 19398 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:42.153484 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:42.155063 18991 leader_election.cc:343] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.155608 19074 ts_tablet_manager.cc:1434] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.096s	user 0.010s	sys 0.002s
I20260709 13:21:42.158295 19075 tablet_bootstrap.cc:654] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:42.161784 18917 leader_election.cc:343] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.168483 19074 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:42.159235 18802 raft_consensus.cc:2468] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:42.190572 19076 tablet_bootstrap.cc:654] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.194767 19380 raft_consensus.cc:2749] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.189589 18952 raft_consensus.cc:2468] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:42.199831 18994 leader_election.cc:304] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:42.200853 19398 raft_consensus.cc:2804] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.201483 19398 raft_consensus.cc:697] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:42.202584 19398 consensus_queue.cc:237] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.204533 19073 tablet_bootstrap.cc:654] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.206472 18799 raft_consensus.cc:2468] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:42.218667 19074 tablet_bootstrap.cc:654] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:42.227377 18771 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 67778.9us
I20260709 13:21:42.236470 18993 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:42.237864 19396 raft_consensus.cc:2804] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.238583 19396 raft_consensus.cc:697] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:42.239593 19396 consensus_queue.cc:237] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.264211 19075 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:42.264974 19075 ts_tablet_manager.cc:1403] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.238s	user 0.020s	sys 0.007s
I20260709 13:21:42.293704 19076 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:42.294484 19076 ts_tablet_manager.cc:1403] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.147s	user 0.014s	sys 0.003s
I20260709 13:21:42.298157 19076 raft_consensus.cc:359] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.299041 19076 raft_consensus.cc:385] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.303155 19076 raft_consensus.cc:740] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.310387 19076 consensus_queue.cc:260] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.310767 19075 raft_consensus.cc:359] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:42.315268 19075 raft_consensus.cc:385] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.315624 19075 raft_consensus.cc:740] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.316286 19075 consensus_queue.cc:260] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:42.329639 19398 consensus_queue.cc:1048] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.331269 19076 ts_tablet_manager.cc:1434] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.036s	user 0.011s	sys 0.000s
I20260709 13:21:42.332751 19076 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:42.333990 18698 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48472 (request call id 55) took 1306 ms. Trace:
I20260709 13:21:42.337836 18699 catalog_manager.cc:5697] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.341619 19075 ts_tablet_manager.cc:1434] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:21:42.344237 19074 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:42.344955 19074 ts_tablet_manager.cc:1403] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.182s	user 0.010s	sys 0.011s
I20260709 13:21:42.347440 19493 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.348069 19493 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.349773 19073 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:42.350487 19073 ts_tablet_manager.cc:1403] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.235s	user 0.010s	sys 0.044s
I20260709 13:21:42.352001 19493 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:42.352161 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:42.354175 19074 raft_consensus.cc:359] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
W20260709 13:21:42.354422 18768 leader_election.cc:343] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.355027 19074 raft_consensus.cc:385] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.355437 19074 raft_consensus.cc:740] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.356307 19074 consensus_queue.cc:260] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.358973 19076 tablet_bootstrap.cc:654] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.334501 18698 rpcz_store.cc:276] 0709 13:21:41.027886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.028070 (+   184us) service_pool.cc:224] Handling call
0709 13:21:42.333909 (+1305839us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:42.041366 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:42.042259 (+   893us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:42.042308 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:21:42.042338 (+    30us) tablet.cc:662] Decoding operations
0709 13:21:42.042852 (+   514us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:42.042971 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:42.043011 (+    40us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:42.068693 (+ 25682us) tablet.cc:701] Row locks acquired
0709 13:21:42.074584 (+  5891us) write_op.cc:260] PREPARE: finished
0709 13:21:42.074781 (+   197us) write_op.cc:270] Start()
0709 13:21:42.074925 (+   144us) write_op.cc:276] Timestamp: P: 1783603302074721 usec, L: 0
0709 13:21:42.074949 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:42.100133 (+ 25184us) log.cc:844] Serialized 537 byte log entry
0709 13:21:42.192342 (+ 92209us) op_driver.cc:464] REPLICATION: finished
0709 13:21:42.223799 (+ 31457us) write_op.cc:301] APPLY: starting
0709 13:21:42.223937 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:42.224098 (+   161us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:42.224306 (+   208us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:42.224665 (+   359us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:42.224971 (+   306us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:21:42.225062 (+    91us) write_op.cc:312] APPLY: finished
0709 13:21:42.225397 (+   335us) log.cc:844] Serialized 22 byte log entry
0709 13:21:42.225688 (+   291us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:42.225833 (+   145us) write_op.cc:454] Released schema lock
0709 13:21:42.225905 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:42.226448 (+   543us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:42.226498 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":119749,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":31076,"apply.run_cpu_time_us":2659,"apply.run_wall_time_us":2657,"num_ops":1,"prepare.queue_time_us":47593,"prepare.run_cpu_time_us":4205,"prepare.run_wall_time_us":74166,"replication_time_us":115038,"thread_start_us":80384,"threads_started":2}]]}
I20260709 13:21:42.343055 19075 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:42.363677 19074 ts_tablet_manager.cc:1434] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:21:42.363161 18702 catalog_manager.cc:5697] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } }
I20260709 13:21:42.365073 19073 raft_consensus.cc:359] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.380431 19073 raft_consensus.cc:385] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.370424 19493 raft_consensus.cc:493] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.381683 19074 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:42.382186 19493 raft_consensus.cc:515] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.381987 18702 catalog_manager.cc:5697] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.361507 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:42.386675 19022 raft_consensus.cc:2468] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:42.380745 19073 raft_consensus.cc:740] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.389266 19073 consensus_queue.cc:260] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.397703 18702 catalog_manager.cc:5697] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:42.399376 19396 consensus_queue.cc:1048] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.399897 18702 catalog_manager.cc:5697] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.401571 18769 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:42.402422 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:42.402621 19205 raft_consensus.cc:2804] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.403357 19205 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.403800 19205 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:42.404812 18768 leader_election.cc:343] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.405707 19074 tablet_bootstrap.cc:654] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.408497 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:42.410458 18771 leader_election.cc:343] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.411057 18771 leader_election.cc:304] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:42.412497 19419 raft_consensus.cc:2749] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.401995 18702 catalog_manager.cc:5697] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.416162 19073 ts_tablet_manager.cc:1434] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.002s
I20260709 13:21:42.400004 19493 leader_election.cc:290] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:42.422283 19073 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:42.428403 19075 tablet_bootstrap.cc:654] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.436282 19405 raft_consensus.cc:493] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.436992 19405 raft_consensus.cc:515] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.440775 19076 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:42.441529 19076 ts_tablet_manager.cc:1403] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.109s	user 0.020s	sys 0.009s
I20260709 13:21:42.441212 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:42.445118 19076 raft_consensus.cc:359] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.446039 19405 leader_election.cc:290] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:42.446136 19076 raft_consensus.cc:385] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.446853 19076 raft_consensus.cc:740] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.448309 19405 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.449011 19405 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.452490 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed73ebff034411ab7b405ac83d2e34e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:42.456117 19405 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:42.456691 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:42.458460 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:42.461700 19076 consensus_queue.cc:260] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.465555 18844 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:42.475093 19205 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.475057 19405 raft_consensus.cc:2749] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.462886 18846 leader_election.cc:304] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:42.492277 19380 raft_consensus.cc:2749] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:42.496978 18880 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.505395 19076 ts_tablet_manager.cc:1434] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:21:42.502671 19073 tablet_bootstrap.cc:654] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.516076 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:42.498212 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:42.523409 19026 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.526562 18872 raft_consensus.cc:3060] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.493563 19205 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:42.531958 18768 leader_election.cc:343] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.554083 19076 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:42.560384 19074 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:42.574488 19026 raft_consensus.cc:2468] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:42.577175 18769 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:42.551347 19398 consensus_queue.cc:1048] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:21:42.579310 19440 raft_consensus.cc:2804] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.580341 19440 raft_consensus.cc:697] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:42.581717 19440 consensus_queue.cc:237] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:42.590394 19074 ts_tablet_manager.cc:1403] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.209s	user 0.021s	sys 0.007s
I20260709 13:21:42.594221 19074 raft_consensus.cc:359] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.595232 19074 raft_consensus.cc:385] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.595608 19074 raft_consensus.cc:740] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.597114 19074 consensus_queue.cc:260] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.599694 19074 ts_tablet_manager.cc:1434] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 13:21:42.601097 19074 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:42.628590 19426 consensus_queue.cc:1048] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.638056 19405 raft_consensus.cc:493] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.648161 19405 raft_consensus.cc:515] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.650141 19440 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.650797 19440 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.653198 19440 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:42.653862 18991 consensus_peers.cc:597] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:42.655928 19075 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:42.656685 19075 ts_tablet_manager.cc:1403] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.314s	user 0.030s	sys 0.009s
I20260709 13:21:42.656816 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:42.657641 19026 raft_consensus.cc:2468] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:42.659055 19426 consensus_queue.cc:1048] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.660580 18769 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:42.660804 19457 consensus_queue.cc:1048] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:42.662981 19205 raft_consensus.cc:2804] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.663507 19205 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.663882 19205 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.668282 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:42.670603 18771 leader_election.cc:343] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.675256 18877 raft_consensus.cc:3060] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.674860 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:42.678079 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "706f5712f7ac487b97054415f672886f" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:42.679265 19405 leader_election.cc:290] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:42.681788 18845 leader_election.cc:304] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:42.681422 18699 catalog_manager.cc:5697] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:42.714105 19075 raft_consensus.cc:359] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:42.717044 18991 consensus_peers.cc:597] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:42.731323 19073 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:42.732021 19073 ts_tablet_manager.cc:1403] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.310s	user 0.019s	sys 0.017s
I20260709 13:21:42.743667 19073 raft_consensus.cc:359] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.744619 19073 raft_consensus.cc:385] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.744953 19073 raft_consensus.cc:740] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.755951 19073 consensus_queue.cc:260] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.766872 19073 ts_tablet_manager.cc:1434] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.001s
I20260709 13:21:42.768322 19074 tablet_bootstrap.cc:654] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.777055 19073 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:42.776715 19398 consensus_queue.cc:1048] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:42.715039 19075 raft_consensus.cc:385] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:42.779574 19075 raft_consensus.cc:740] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.780884 19075 consensus_queue.cc:260] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.782241 19457 raft_consensus.cc:493] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.782927 19457 raft_consensus.cc:515] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:42.787689 19457 leader_election.cc:290] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:42.788391 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:42.789366 19026 raft_consensus.cc:2468] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:42.792037 18918 leader_election.cc:304] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:42.805048 19075 ts_tablet_manager.cc:1434] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.148s	user 0.007s	sys 0.002s
I20260709 13:21:42.788337 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:42.808454 19453 raft_consensus.cc:2804] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:42.823130 19453 raft_consensus.cc:493] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.823686 19453 raft_consensus.cc:3060] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.825439 19493 raft_consensus.cc:493] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.826105 19493 raft_consensus.cc:515] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:42.827440 19075 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:42.831387 19205 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.832484 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:42.834098 19205 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:42.836069 19380 raft_consensus.cc:2749] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:42.835000 18771 leader_election.cc:343] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.839426 19073 tablet_bootstrap.cc:654] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:42.839756 18917 leader_election.cc:343] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.847944 19076 tablet_bootstrap.cc:654] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.848521 19075 tablet_bootstrap.cc:654] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:42.828641 19493 leader_election.cc:290] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:42.857390 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:42.873962 19026 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:42.949613 19026 raft_consensus.cc:2468] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:42.964700 18769 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:42.972154 19493 raft_consensus.cc:2804] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:42.980836 19493 raft_consensus.cc:697] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:42.982093 19493 consensus_queue.cc:237] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:42.996431 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
W20260709 13:21:42.999332 18771 leader_election.cc:343] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:42.960021 19402 consensus_queue.cc:1048] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:21:43.021551 19074 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:43.022485 19074 ts_tablet_manager.cc:1403] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.422s	user 0.042s	sys 0.017s
I20260709 13:21:43.028913 19453 raft_consensus.cc:515] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.034765 19074 raft_consensus.cc:359] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.035778 19074 raft_consensus.cc:385] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.036129 19074 raft_consensus.cc:740] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:42.981925 18703 catalog_manager.cc:5697] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:43.037074 19074 consensus_queue.cc:260] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.069551 19074 ts_tablet_manager.cc:1434] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.001s
I20260709 13:21:43.088485 19075 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:43.089675 19075 ts_tablet_manager.cc:1403] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.268s	user 0.010s	sys 0.011s
I20260709 13:21:43.111303 19074 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:43.136075 19073 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:43.137019 19073 ts_tablet_manager.cc:1403] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.361s	user 0.023s	sys 0.015s
W20260709 13:21:43.156159 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:21:43.175848 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 111809us
I20260709 13:21:43.184027 19419 consensus_queue.cc:1048] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:43.143548 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:43.171876 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 97896.9us
I20260709 13:21:43.198925 19073 raft_consensus.cc:359] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.202881 19073 raft_consensus.cc:385] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.205214 19076 tablet_bootstrap.cc:492] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:43.217655 19076 ts_tablet_manager.cc:1403] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.689s	user 0.016s	sys 0.017s
I20260709 13:21:43.203392 19073 raft_consensus.cc:740] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.234155 19073 consensus_queue.cc:260] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.224402 19453 leader_election.cc:290] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:43.247716 19073 ts_tablet_manager.cc:1434] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.000s
I20260709 13:21:43.237303 19521 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.253518 19075 raft_consensus.cc:359] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.254845 19075 raft_consensus.cc:385] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.255683 19075 raft_consensus.cc:740] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.262766 19074 tablet_bootstrap.cc:654] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.243955 18873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:43.259675 19521 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.282698 19521 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:43.293998 19076 raft_consensus.cc:359] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.294994 19076 raft_consensus.cc:385] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.257802 19532 raft_consensus.cc:493] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.295387 19076 raft_consensus.cc:740] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.295686 19532 raft_consensus.cc:515] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.298461 19073 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:43.298678 19532 leader_election.cc:290] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:43.312372 18768 leader_election.cc:343] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.314235 18768 leader_election.cc:304] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:43.298794 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:43.320026 19531 raft_consensus.cc:2749] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:43.328325 18993 leader_election.cc:343] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.328403 18703 catalog_manager.cc:5697] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:43.290138 19075 consensus_queue.cc:260] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.335750 19075 ts_tablet_manager.cc:1434] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.245s	user 0.010s	sys 0.003s
I20260709 13:21:43.337015 19075 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:43.341008 19076 consensus_queue.cc:260] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.346730 18873 raft_consensus.cc:3060] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.353085 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:43.356107 18768 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:43.358173 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 80408.1us
I20260709 13:21:43.365557 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:43.366443 18799 raft_consensus.cc:2468] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:43.373438 18919 leader_election.cc:304] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:43.388007 19021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f9127f17e646d7a2d0f25d4a4b0d4f" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:43.390388 19021 raft_consensus.cc:3060] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.392406 19532 raft_consensus.cc:2804] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.394378 19532 raft_consensus.cc:493] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.394877 19532 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.399523 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:43.427307 19073 tablet_bootstrap.cc:654] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.429500 18873 raft_consensus.cc:2468] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
W20260709 13:21:43.436159 18917 leader_election.cc:343] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.434695 19532 raft_consensus.cc:515] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.440011 19074 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:43.437700 19075 tablet_bootstrap.cc:654] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.440661 19074 ts_tablet_manager.cc:1403] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.341s	user 0.014s	sys 0.002s
W20260709 13:21:43.441243 18994 leader_election.cc:343] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.441808 18994 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:43.443918 19398 raft_consensus.cc:2749] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.446792 19532 leader_election.cc:290] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:43.448097 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:43.450052 18799 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.456216 19073 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:43.467432 19073 ts_tablet_manager.cc:1403] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.169s	user 0.012s	sys 0.005s
I20260709 13:21:43.485642 19531 consensus_queue.cc:1048] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:43.481791 18799 raft_consensus.cc:2468] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:43.489737 19076 ts_tablet_manager.cc:1434] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.272s	user 0.010s	sys 0.000s
I20260709 13:21:43.496024 18919 leader_election.cc:304] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:43.497213 19532 raft_consensus.cc:2804] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:43.497877 19532 raft_consensus.cc:697] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:43.473342 19074 raft_consensus.cc:359] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.503742 19074 raft_consensus.cc:385] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.503311 19076 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:43.504730 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:43.499017 19532 consensus_queue.cc:237] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.520941 19531 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.526182 19531 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.528465 19074 raft_consensus.cc:740] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.529067 19531 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:43.531276 18917 leader_election.cc:304] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:43.533061 19407 raft_consensus.cc:2804] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:43.533543 18917 leader_election.cc:343] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.533668 19407 raft_consensus.cc:697] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:43.536136 19407 consensus_queue.cc:237] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.542346 19076 tablet_bootstrap.cc:654] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.543588 18873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:43.531713 19073 raft_consensus.cc:359] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.547103 19074 consensus_queue.cc:260] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.548429 19075 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:43.622406 19075 ts_tablet_manager.cc:1403] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.286s	user 0.017s	sys 0.017s
I20260709 13:21:43.623767 19074 ts_tablet_manager.cc:1434] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.183s	user 0.011s	sys 0.000s
I20260709 13:21:43.556689 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:43.625274 19074 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:43.626525 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 74301.2us
I20260709 13:21:43.628324 19486 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.628964 19486 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.631156 19486 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:43.630667 19075 raft_consensus.cc:359] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.631951 19075 raft_consensus.cc:385] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.632283 19075 raft_consensus.cc:740] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
W20260709 13:21:43.632477 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 67303.7us
I20260709 13:21:43.633023 19075 consensus_queue.cc:260] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.565330 19021 raft_consensus.cc:2468] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:43.627240 19073 raft_consensus.cc:385] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.636313 19073 raft_consensus.cc:740] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.637146 19073 consensus_queue.cc:260] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.638185 19026 raft_consensus.cc:2468] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:43.639381 19075 ts_tablet_manager.cc:1434] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:21:43.641045 19075 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
W20260709 13:21:43.655334 18768 leader_election.cc:343] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.659096 18769 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:43.661008 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:43.660271 19493 raft_consensus.cc:2804] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:43.662547 19493 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.663045 19493 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.674643 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:43.681023 18949 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.682358 19074 tablet_bootstrap.cc:654] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.687814 19073 ts_tablet_manager.cc:1434] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.202s	user 0.013s	sys 0.001s
I20260709 13:21:43.706102 19073 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:43.708662 18918 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:43.721544 19396 raft_consensus.cc:493] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.722610 19396 raft_consensus.cc:515] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:43.725365 19076 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:43.726198 19076 ts_tablet_manager.cc:1403] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.235s	user 0.014s	sys 0.004s
I20260709 13:21:43.727969 19075 tablet_bootstrap.cc:654] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.746718 19076 raft_consensus.cc:359] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.748086 19076 raft_consensus.cc:385] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.748590 19076 raft_consensus.cc:740] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.741259 19396 leader_election.cc:290] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:43.752683 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:43.759644 19537 raft_consensus.cc:2749] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.760888 19076 consensus_queue.cc:260] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:43.761526 18994 leader_election.cc:343] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.765503 19076 ts_tablet_manager.cc:1434] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.001s
W20260709 13:21:43.739768 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 63204.9us
I20260709 13:21:43.786281 19531 consensus_queue.cc:1048] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:43.791776 18873 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.811542 19076 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:43.806967 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:43.826575 19493 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:43.835218 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
W20260709 13:21:43.837724 18768 leader_election.cc:343] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:21:43.841511 18991 leader_election.cc:343] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.843046 19534 raft_consensus.cc:493] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.843711 19534 raft_consensus.cc:515] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.842258 18991 leader_election.cc:304] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:43.853080 19074 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:43.853844 19074 ts_tablet_manager.cc:1403] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.229s	user 0.028s	sys 0.011s
I20260709 13:21:43.855168 19493 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:43.849344 19398 raft_consensus.cc:2749] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:43.872884 19534 leader_election.cc:290] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:43.873075 19021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:43.874115 19021 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:43.878111 19076 tablet_bootstrap.cc:654] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.885835 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:43.888602 18917 leader_election.cc:343] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:43.890070 19074 raft_consensus.cc:359] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:43.893750 19073 tablet_bootstrap.cc:654] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:43.890991 19074 raft_consensus.cc:385] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:43.915648 19074 raft_consensus.cc:740] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:43.916706 19074 consensus_queue.cc:260] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:43.983246 18771 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 68778.6us
W20260709 13:21:43.984578 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 70583.7us
I20260709 13:21:43.991878 19074 ts_tablet_manager.cc:1434] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.138s	user 0.008s	sys 0.002s
I20260709 13:21:43.994231 19492 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.994890 19492 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:43.997563 19492 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:44.005386 18917 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:44.008533 19075 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:44.009357 19075 ts_tablet_manager.cc:1403] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.369s	user 0.033s	sys 0.037s
I20260709 13:21:44.013429 19021 raft_consensus.cc:2468] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:44.018215 19074 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:44.018798 19075 raft_consensus.cc:359] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.019711 19075 raft_consensus.cc:385] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.020110 19075 raft_consensus.cc:740] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.033697 18698 catalog_manager.cc:5697] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.035836 18698 catalog_manager.cc:5697] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.033447 19075 consensus_queue.cc:260] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.040958 19534 consensus_queue.cc:1048] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:21:44.047655 19075 ts_tablet_manager.cc:1434] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:21:44.113268 19075 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
W20260709 13:21:44.118826 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 131476us
I20260709 13:21:44.121815 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:44.137413 18771 leader_election.cc:343] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.149428 19223 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3e2fa7cf3974c27a19386f362688d29)
I20260709 13:21:44.150115 19223 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.161713 19074 tablet_bootstrap.cc:654] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.164920 19076 tablet_bootstrap.cc:492] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:44.165717 19076 ts_tablet_manager.cc:1403] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.355s	user 0.028s	sys 0.003s
I20260709 13:21:44.201020 19073 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:44.202155 19073 ts_tablet_manager.cc:1403] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.499s	user 0.017s	sys 0.009s
I20260709 13:21:44.203066 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:44.203972 19022 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:44.208869 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:44.210071 18801 raft_consensus.cc:2468] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:44.217801 19407 raft_consensus.cc:493] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.221297 19407 raft_consensus.cc:515] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.223804 19407 leader_election.cc:290] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:44.223409 19076 raft_consensus.cc:359] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.224372 19076 raft_consensus.cc:385] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.224709 19076 raft_consensus.cc:740] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.240324 19223 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:44.240981 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:44.234236 19073 raft_consensus.cc:359] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.243075 19073 raft_consensus.cc:385] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.243522 19073 raft_consensus.cc:740] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.269228 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:44.333798 19073 consensus_queue.cc:260] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:44.336632 18917 leader_election.cc:343] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.336254 19073 ts_tablet_manager.cc:1434] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.133s	user 0.001s	sys 0.013s
W20260709 13:21:44.394546 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 144691us
W20260709 13:21:44.338335 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 66879.8us
W20260709 13:21:44.338246 18768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 90344.6us
I20260709 13:21:44.355705 19076 consensus_queue.cc:260] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.361582 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:44.399467 19026 raft_consensus.cc:2468] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:44.366329 18769 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.386736 19074 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:44.354327 19075 tablet_bootstrap.cc:700] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: Time spent opening tablet: real 0.140s	user 0.003s	sys 0.002s
I20260709 13:21:44.404795 19074 ts_tablet_manager.cc:1403] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.387s	user 0.019s	sys 0.000s
I20260709 13:21:44.406256 18918 leader_election.cc:304] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.407287 19075 tablet_bootstrap.cc:654] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.414275 19076 ts_tablet_manager.cc:1434] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.248s	user 0.008s	sys 0.000s
I20260709 13:21:44.418048 18769 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.358456 19559 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.420781 19073 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:44.426405 19492 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.425174 19559 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.426987 19492 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.429037 19492 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.414059 19402 raft_consensus.cc:2804] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.431746 19402 raft_consensus.cc:493] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.433557 19402 raft_consensus.cc:3060] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.434125 19559 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:44.437115 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:44.416461 19517 raft_consensus.cc:2804] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.439558 19517 raft_consensus.cc:697] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:44.440636 19517 consensus_queue.cc:237] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:44.448594 18771 leader_election.cc:343] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.369930 19532 consensus_queue.cc:1048] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 13:21:44.457156 19074 raft_consensus.cc:359] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.458046 19074 raft_consensus.cc:385] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.458326 19074 raft_consensus.cc:740] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.459266 19074 consensus_queue.cc:260] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.456445 19492 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.467233 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:44.469864 19074 ts_tablet_manager.cc:1434] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 13:21:44.472091 19492 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:44.472963 19074 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:44.475466 18768 leader_election.cc:343] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.476136 18768 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.479897 19419 raft_consensus.cc:2749] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.488193 19074 tablet_bootstrap.cc:654] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.490653 19076 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:44.495409 19073 tablet_bootstrap.cc:654] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.506342 19074 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: No bootstrap required, opened a new log
I20260709 13:21:44.507277 19074 ts_tablet_manager.cc:1403] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.000s
I20260709 13:21:44.510438 19076 tablet_bootstrap.cc:654] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.510706 19074 raft_consensus.cc:359] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.511651 19074 raft_consensus.cc:385] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.511983 19074 raft_consensus.cc:740] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.513531 19074 consensus_queue.cc:260] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.515496 19402 raft_consensus.cc:515] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.516332 19074 ts_tablet_manager.cc:1434] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:21:44.531955 19398 raft_consensus.cc:493] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.532732 19398 raft_consensus.cc:515] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.537014 19398 leader_election.cc:290] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:44.537039 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:44.538095 19402 leader_election.cc:290] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:44.538902 19073 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
W20260709 13:21:44.539043 18991 leader_election.cc:343] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.539512 19073 ts_tablet_manager.cc:1403] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.121s	user 0.015s	sys 0.000s
I20260709 13:21:44.553925 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
W20260709 13:21:44.556252 18994 leader_election.cc:343] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.559423 18994 leader_election.cc:304] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.566115 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
W20260709 13:21:44.566231 18769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 145529us
I20260709 13:21:44.567166 19026 raft_consensus.cc:3060] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.564879 19396 raft_consensus.cc:2749] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.568848 18919 leader_election.cc:304] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.570664 19486 raft_consensus.cc:2804] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:21:44.578925 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:44.580091 19073 raft_consensus.cc:359] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.581054 19073 raft_consensus.cc:385] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.581367 19073 raft_consensus.cc:740] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.563467 18873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee6b2223720435195a84c3570506ea5" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:44.573860 19486 raft_consensus.cc:493] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.588271 19486 raft_consensus.cc:3060] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:44.591094 18917 leader_election.cc:343] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.591636 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:44.592092 19565 raft_consensus.cc:493] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.592782 19565 raft_consensus.cc:515] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.606345 19073 consensus_queue.cc:260] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.613134 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:44.614838 19027 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.631939 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:44.634868 18873 raft_consensus.cc:3060] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:44.640493 18771 leader_election.cc:343] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.645363 19026 raft_consensus.cc:2468] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:44.646641 19398 raft_consensus.cc:493] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.672528 19398 raft_consensus.cc:515] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.676455 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:44.652733 19565 leader_election.cc:290] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
W20260709 13:21:44.678149 18994 leader_election.cc:343] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.654184 19022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:44.655189 19075 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:44.681814 19075 ts_tablet_manager.cc:1403] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.600s	user 0.019s	sys 0.014s
W20260709 13:21:44.685194 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 113305us
I20260709 13:21:44.685442 19075 raft_consensus.cc:359] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.686339 19075 raft_consensus.cc:385] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.686641 19075 raft_consensus.cc:740] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.688393 19398 leader_election.cc:290] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:44.682564 19486 raft_consensus.cc:515] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.692714 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:44.693560 18801 raft_consensus.cc:2468] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 0.
W20260709 13:21:44.693603 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 45979.9us
I20260709 13:21:44.695819 18993 leader_election.cc:304] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.697211 19398 raft_consensus.cc:2804] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.697691 19398 raft_consensus.cc:493] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.698290 19398 raft_consensus.cc:3060] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.664127 19073 ts_tablet_manager.cc:1434] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.124s	user 0.009s	sys 0.000s
I20260709 13:21:44.664688 19027 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:44.670786 18845 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:44.709218 19073 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: Bootstrap starting.
I20260709 13:21:44.709405 19556 raft_consensus.cc:2749] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:44.706068 18702 catalog_manager.cc:5697] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.713814 19486 leader_election.cc:290] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:44.730831 19398 raft_consensus.cc:515] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.715430 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:44.715489 18769 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.720184 19534 consensus_queue.cc:1048] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:44.734549 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:44.721802 19075 consensus_queue.cc:260] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.736217 19493 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:21:44.736539 18917 leader_election.cc:343] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.736873 19493 raft_consensus.cc:697] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:44.737718 19076 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
W20260709 13:21:44.737556 18994 leader_election.cc:343] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.738370 19076 ts_tablet_manager.cc:1403] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.248s	user 0.020s	sys 0.035s
I20260709 13:21:44.737955 19493 consensus_queue.cc:237] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.730320 19073 tablet_bootstrap.cc:654] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.746917 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:44.747792 18801 raft_consensus.cc:3060] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.748621 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:44.749375 19075 ts_tablet_manager.cc:1434] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:21:44.749523 18803 raft_consensus.cc:3060] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.748718 19398 leader_election.cc:290] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:44.750774 19075 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:44.756573 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6808727e98af41939b051f079357a49f" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:44.764503 18803 raft_consensus.cc:2468] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:44.766304 18993 leader_election.cc:304] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:44.767951 19398 raft_consensus.cc:2804] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.769531 19398 raft_consensus.cc:697] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:44.770861 19398 consensus_queue.cc:237] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.773687 19075 tablet_bootstrap.cc:654] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.775954 18918 leader_election.cc:304] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.741690 19076 raft_consensus.cc:359] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.788839 19076 raft_consensus.cc:385] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.789410 19076 raft_consensus.cc:740] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.797535 19076 consensus_queue.cc:260] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.800488 19076 ts_tablet_manager.cc:1434] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.003s
I20260709 13:21:44.801787 19076 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:44.808995 19076 tablet_bootstrap.cc:654] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.813367 19073 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: No bootstrap required, opened a new log
I20260709 13:21:44.814205 19073 ts_tablet_manager.cc:1403] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: Time spent bootstrapping tablet: real 0.105s	user 0.013s	sys 0.001s
I20260709 13:21:44.836864 19076 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:44.837546 19076 ts_tablet_manager.cc:1403] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.013s
I20260709 13:21:44.792068 19486 raft_consensus.cc:2804] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.849539 19486 raft_consensus.cc:697] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:44.850503 19073 raft_consensus.cc:359] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.851549 19073 raft_consensus.cc:385] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.850806 19486 consensus_queue.cc:237] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:44.851863 19073 raft_consensus.cc:740] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.852838 19073 consensus_queue.cc:260] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.835410 18801 raft_consensus.cc:2468] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:44.856177 19073 ts_tablet_manager.cc:1434] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.000s
I20260709 13:21:44.856642 18846 leader_election.cc:304] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:44.858418 19223 raft_consensus.cc:2749] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:44.872468 19075 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:44.866689 19559 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.893093 19075 ts_tablet_manager.cc:1403] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.004s
I20260709 13:21:44.893501 19559 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.898032 19559 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:44.899696 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:44.900851 19076 raft_consensus.cc:359] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:44.901585 18768 leader_election.cc:343] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:44.901713 19076 raft_consensus.cc:385] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.902168 19076 raft_consensus.cc:740] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.902405 19075 raft_consensus.cc:359] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.903226 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:44.904224 19027 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 0.
I20260709 13:21:44.906025 18769 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.911720 19076 consensus_queue.cc:260] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.903286 19075 raft_consensus.cc:385] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:44.915400 19075 raft_consensus.cc:740] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:44.916317 19075 consensus_queue.cc:260] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:44.922560 18702 catalog_manager.cc:5697] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 1 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:44.926616 19076 ts_tablet_manager.cc:1434] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.002s
I20260709 13:21:44.932528 19076 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:44.935707 19075 ts_tablet_manager.cc:1434] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:21:44.938987 18698 catalog_manager.cc:5697] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:44.911020 19559 raft_consensus.cc:2804] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:44.944084 19559 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:44.944636 19559 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:44.937708 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 94757.7us
I20260709 13:21:44.963940 19076 tablet_bootstrap.cc:654] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:44.966030 18919 leader_election.cc:304] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:44.966321 18873 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:44.967026 19486 raft_consensus.cc:2804] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:44.967850 19486 raft_consensus.cc:697] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:44.969036 19486 consensus_queue.cc:237] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:44.975662 19075 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:44.980538 19559 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:44.994527 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:45.020026 19027 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:45.022424 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 56502.6us
I20260709 13:21:44.991719 18698 catalog_manager.cc:5697] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:45.028446 19075 tablet_bootstrap.cc:654] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.035482 19565 raft_consensus.cc:493] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.039170 19565 raft_consensus.cc:515] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:45.039878 19532 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:44.993721 19559 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:45.044044 19027 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 1.
I20260709 13:21:45.049122 18769 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:45.011638 19517 consensus_queue.cc:1048] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 13:21:45.040740 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:45.057245 19559 raft_consensus.cc:2804] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:45.042784 19532 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:45.076610 19559 raft_consensus.cc:697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:21:45.095470 19532 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:45.096524 19559 consensus_queue.cc:237] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.098662 19076 tablet_bootstrap.cc:492] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:45.099413 19076 ts_tablet_manager.cc:1403] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.007s
W20260709 13:21:45.092094 18768 leader_election.cc:343] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.092785 19565 leader_election.cc:290] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:45.102650 19076 raft_consensus.cc:359] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.104619 19076 raft_consensus.cc:385] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.104956 19076 raft_consensus.cc:740] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
W20260709 13:21:45.112860 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 69517.6us
I20260709 13:21:45.075932 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:45.113538 19076 consensus_queue.cc:260] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.129221 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcb4a0d3f9924af3881fae78b771264e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:45.133994 19076 ts_tablet_manager.cc:1434] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:21:45.145990 19076 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:45.168926 18801 raft_consensus.cc:2393] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 039c551e55fb43cdb5c3880b7cb91a0b in current term 1: Already voted for candidate de62ff9486f7483e8f0e0b93dcca24bb in this term.
I20260709 13:21:45.184913 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:45.194346 19076 tablet_bootstrap.cc:654] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
W20260709 13:21:45.197913 18917 leader_election.cc:343] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.222291 18702 catalog_manager.cc:5697] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:45.226382 18768 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:45.239663 18845 leader_election.cc:304] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:45.241209 19600 raft_consensus.cc:3060] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.249419 19562 consensus_queue.cc:1048] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.262179 19076 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:45.268024 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:45.287817 18805 raft_consensus.cc:2468] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
W20260709 13:21:45.254616 18994 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:45.293552 19076 ts_tablet_manager.cc:1403] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.150s	user 0.028s	sys 0.035s
I20260709 13:21:45.313257 19537 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.314005 19537 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:45.317469 19537 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:45.321848 18951 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.323065 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:45.302358 19517 consensus_queue.cc:1048] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.324146 19027 raft_consensus.cc:2393] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de62ff9486f7483e8f0e0b93dcca24bb in current term 1: Already voted for candidate 878e795a8488495f978538952e7b2324 in this term.
I20260709 13:21:45.303768 19075 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:45.325590 19076 raft_consensus.cc:359] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.326470 19076 raft_consensus.cc:385] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.325385 19075 ts_tablet_manager.cc:1403] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.350s	user 0.013s	sys 0.006s
I20260709 13:21:45.326848 19076 raft_consensus.cc:740] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.300045 18698 catalog_manager.cc:5697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:45.294945 18919 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:45.336956 19537 raft_consensus.cc:2804] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.337533 19537 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.337930 19537 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.339377 18805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:45.345460 18918 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:45.346841 19600 raft_consensus.cc:2749] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.361040 19076 consensus_queue.cc:260] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.333209 19075 raft_consensus.cc:359] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.369074 19075 raft_consensus.cc:385] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.369609 19075 raft_consensus.cc:740] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.374140 19076 ts_tablet_manager.cc:1434] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.062s	user 0.002s	sys 0.005s
I20260709 13:21:45.375573 19076 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:45.386730 19076 tablet_bootstrap.cc:654] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.407400 19076 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:45.408192 19076 ts_tablet_manager.cc:1403] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.001s
I20260709 13:21:45.414341 19076 raft_consensus.cc:359] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.418212 19596 consensus_queue.cc:1048] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.420612 19076 raft_consensus.cc:385] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.421057 19076 raft_consensus.cc:740] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.424300 19076 consensus_queue.cc:260] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.426848 19076 ts_tablet_manager.cc:1434] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:21:45.370563 19075 consensus_queue.cc:260] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.433813 19407 consensus_queue.cc:1048] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.434343 19076 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:45.446121 19076 tablet_bootstrap.cc:654] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.462321 18871 raft_consensus.cc:3060] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.476679 19596 consensus_queue.cc:1048] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:45.486840 19075 ts_tablet_manager.cc:1434] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.160s	user 0.009s	sys 0.000s
I20260709 13:21:45.492944 19589 raft_consensus.cc:2749] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.495699 19075 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: Bootstrap starting.
I20260709 13:21:45.544808 19599 consensus_queue.cc:1048] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:21:45.529915 19596 raft_consensus.cc:493] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.548810 19596 raft_consensus.cc:515] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.601279 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:45.602145 19027 raft_consensus.cc:2468] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 0.
I20260709 13:21:45.604460 18918 leader_election.cc:304] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:45.615427 19589 raft_consensus.cc:2804] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:21:45.589143 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:45.594607 19596 leader_election.cc:290] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:45.623956 19223 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.631564 19223 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.634581 19223 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:45.642936 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:45.644937 18917 leader_election.cc:343] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:45.615974 19589 raft_consensus.cc:493] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:45.656440 19589 raft_consensus.cc:3060] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.661871 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:45.663094 18768 consensus_peers.cc:597] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:45.669570 19223 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.670724 19223 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:45.701408 19075 tablet_bootstrap.cc:654] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:45.599715 19537 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:45.713985 19223 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:45.720382 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 115997us
I20260709 13:21:45.722599 19076 tablet_bootstrap.cc:492] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:45.725463 19076 ts_tablet_manager.cc:1403] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.291s	user 0.034s	sys 0.031s
I20260709 13:21:45.701277 19419 consensus_queue.cc:1048] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:21:45.736707 19407 consensus_queue.cc:1048] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:45.758123 19591 log.cc:927] Time spent T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:21:45.780741 19537 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:45.782923 19589 raft_consensus.cc:515] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.705773 18845 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:45.860508 18845 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 178288us
W20260709 13:21:45.867956 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 115280us
W20260709 13:21:45.874747 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 120739us
I20260709 13:21:45.884886 19600 raft_consensus.cc:2749] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:45.894348 19589 leader_election.cc:290] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:21:45.939869 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:45.944629 19026 raft_consensus.cc:3060] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:45.933735 19076 raft_consensus.cc:359] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.968638 19076 raft_consensus.cc:385] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:45.969187 19076 raft_consensus.cc:740] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:45.970314 19076 consensus_queue.cc:260] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:21:45.974543 18879 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.002133 18801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324"
I20260709 13:21:46.003496 18801 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:21:46.003567 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 304833us
W20260709 13:21:46.015303 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 66031.7us
I20260709 13:21:46.026957 19075 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: No bootstrap required, opened a new log
I20260709 13:21:46.028658 19075 ts_tablet_manager.cc:1403] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent bootstrapping tablet: real 0.533s	user 0.023s	sys 0.017s
I20260709 13:21:46.033069 18872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:46.033937 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0240263315db416dbece9ba6b467818d" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:46.035035 18877 raft_consensus.cc:3060] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.061465 19075 raft_consensus.cc:359] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:46.063486 19076 ts_tablet_manager.cc:1434] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.338s	user 0.011s	sys 0.005s
I20260709 13:21:46.065657 19076 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:46.062904 19075 raft_consensus.cc:385] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.073210 18801 raft_consensus.cc:2468] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:46.068192 19075 raft_consensus.cc:740] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Initialized, Role: FOLLOWER
W20260709 13:21:46.093295 18917 leader_election.cc:343] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:21:46.099532 19026 raft_consensus.cc:2468] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:46.103435 19075 consensus_queue.cc:260] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:46.120009 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:46.128116 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 107676us
I20260709 13:21:46.130374 19075 ts_tablet_manager.cc:1434] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: Time spent starting tablet: real 0.101s	user 0.010s	sys 0.002s
I20260709 13:21:46.179316 18918 leader_election.cc:304] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
W20260709 13:21:46.204766 18750 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:21:46.214059 18803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
W20260709 13:21:46.243003 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 69196.3us
I20260709 13:21:46.247581 19589 raft_consensus.cc:2804] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.248432 19589 raft_consensus.cc:697] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:46.250793 18880 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.250696 19589 consensus_queue.cc:237] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
W20260709 13:21:46.277012 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 61579.9us
I20260709 13:21:46.288236 18919 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:21:46.291467 19076 tablet_bootstrap.cc:654] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.297120 19537 raft_consensus.cc:2804] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:21:46.298164 19537 raft_consensus.cc:697] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
W20260709 13:21:46.267575 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 66816.7us
I20260709 13:21:46.310972 19532 raft_consensus.cc:493] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.311872 19532 raft_consensus.cc:515] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:46.297351 18877 raft_consensus.cc:2468] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:46.323405 19537 consensus_queue.cc:237] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:46.328536 19532 leader_election.cc:290] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:46.359227 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:46.402561 18845 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:46.452314 19623 raft_consensus.cc:2749] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.462169 18802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd6bf8d58b494607937dbeb92cdb2015" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:46.497331 18919 leader_election.cc:304] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:46.505364 19486 raft_consensus.cc:2749] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:21:46.554291 18845 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 56999.7us
I20260709 13:21:46.597232 19076 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:46.597966 19076 ts_tablet_manager.cc:1403] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.533s	user 0.048s	sys 0.036s
I20260709 13:21:46.610159 18951 raft_consensus.cc:3060] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:46.621862 19076 raft_consensus.cc:359] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:46.623143 19076 raft_consensus.cc:385] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.624660 19076 raft_consensus.cc:740] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.643875 19076 consensus_queue.cc:260] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:46.670967 19537 consensus_queue.cc:1048] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:21:46.753376 19486 consensus_queue.cc:1048] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:21:46.760627 19076 ts_tablet_manager.cc:1434] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.162s	user 0.008s	sys 0.003s
I20260709 13:21:46.763389 19076 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: Bootstrap starting.
I20260709 13:21:46.758422 18698 catalog_manager.cc:5697] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } }
I20260709 13:21:46.792976 19076 tablet_bootstrap.cc:654] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:21:46.867878 19534 consensus_queue.cc:1048] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:21:46.871490 18917 consensus_peers.cc:597] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:46.879345 18699 catalog_manager.cc:5697] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 0 to 1, leader changed from <none> to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 1 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } }
I20260709 13:21:46.903756 19623 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.904636 19623 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:46.908924 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:46.921159 19623 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:46.926220 19076 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: No bootstrap required, opened a new log
I20260709 13:21:46.929124 19076 ts_tablet_manager.cc:1403] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent bootstrapping tablet: real 0.167s	user 0.016s	sys 0.006s
I20260709 13:21:46.931327 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:46.941100 18845 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:46.947451 19623 raft_consensus.cc:2749] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:46.969760 19076 raft_consensus.cc:359] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:46.970808 19076 raft_consensus.cc:385] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:21:46.971189 19076 raft_consensus.cc:740] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Initialized, Role: FOLLOWER
I20260709 13:21:46.972086 19076 consensus_queue.cc:260] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:46.974110 19076 ts_tablet_manager.cc:1434] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
I20260709 13:21:47.120683 18880 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.197918 19623 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.198740 19623 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:47.243590 18800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878e795a8488495f978538952e7b2324" is_pre_election: true
I20260709 13:21:47.286350 19623 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:47.306031 18880 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:21:47.316025 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:47.320892 18846 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:47.355445 19623 raft_consensus.cc:2749] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:21:48.711544  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
W20260709 13:21:48.719765 18919 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 13:21:48.758841 18919 consensus_peers.cc:597] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.800375 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.817790 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.828310 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.830132 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.844609 18845 consensus_peers.cc:597] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.846004 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:48.866986  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:48.869050  8925 tablet_replica.cc:333] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.870782  8925 raft_consensus.cc:2243] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:48.871037 18919 consensus_peers.cc:597] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:48.871731  8925 raft_consensus.cc:2272] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.875957  8925 tablet_replica.cc:333] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.877043  8925 raft_consensus.cc:2243] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.877694  8925 raft_consensus.cc:2272] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.880986  8925 tablet_replica.cc:333] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.881953  8925 raft_consensus.cc:2243] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.882630  8925 raft_consensus.cc:2272] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.885871  8925 tablet_replica.cc:333] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.886608  8925 raft_consensus.cc:2243] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.887182  8925 raft_consensus.cc:2272] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.890789  8925 tablet_replica.cc:333] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.892009  8925 raft_consensus.cc:2243] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.892778  8925 raft_consensus.cc:2272] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.896438  8925 tablet_replica.cc:333] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.897611  8925 raft_consensus.cc:2243] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.898453  8925 raft_consensus.cc:2272] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.913517  8925 tablet_replica.cc:333] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.917562  8925 raft_consensus.cc:2243] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:48.928319  8925 raft_consensus.cc:2272] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.933362  8925 tablet_replica.cc:333] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.934748  8925 raft_consensus.cc:2243] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.935812  8925 raft_consensus.cc:2272] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.940951  8925 tablet_replica.cc:333] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.942399  8925 raft_consensus.cc:2243] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:48.944476  8925 raft_consensus.cc:2272] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:48.950172 18919 consensus_peers.cc:597] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:48.952531  8925 tablet_replica.cc:333] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.953876  8925 raft_consensus.cc:2243] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:48.955351  8925 raft_consensus.cc:2272] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.963064  8925 tablet_replica.cc:333] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.965422  8925 raft_consensus.cc:2243] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.966773  8925 raft_consensus.cc:2272] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.971066  8925 tablet_replica.cc:333] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.972385  8925 raft_consensus.cc:2243] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.973235  8925 raft_consensus.cc:2272] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:48.981786 18993 consensus_peers.cc:597] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:48.986114 18993 consensus_peers.cc:597] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:48.986120  8925 tablet_replica.cc:333] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.988042  8925 raft_consensus.cc:2243] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:48.988897  8925 raft_consensus.cc:2272] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:48.992956  8925 tablet_replica.cc:333] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:48.994238  8925 raft_consensus.cc:2243] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:48.995827  8925 raft_consensus.cc:2272] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.000972  8925 tablet_replica.cc:333] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: stopping tablet replica
W20260709 13:21:49.001354 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.002164  8925 raft_consensus.cc:2243] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.003885  8925 raft_consensus.cc:2272] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.008116  8925 tablet_replica.cc:333] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: stopping tablet replica
W20260709 13:21:49.012948 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:49.013355 18919 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.014091  8925 raft_consensus.cc:2243] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.015055  8925 raft_consensus.cc:2272] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.019194  8925 tablet_replica.cc:333] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.020349  8925 raft_consensus.cc:2243] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.021251  8925 raft_consensus.cc:2272] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.025144  8925 tablet_replica.cc:333] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.028219  8925 raft_consensus.cc:2243] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.029022  8925 raft_consensus.cc:2272] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.034353 18993 consensus_peers.cc:597] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.034474  8925 tablet_replica.cc:333] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.035894  8925 raft_consensus.cc:2243] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.036669  8925 raft_consensus.cc:2272] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.037958 18919 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.041766  8925 tablet_replica.cc:333] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.043268  8925 raft_consensus.cc:2243] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.044871  8925 raft_consensus.cc:2272] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.049093  8925 tablet_replica.cc:333] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.050380  8925 raft_consensus.cc:2243] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.051988  8925 raft_consensus.cc:2272] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.055892  8925 tablet_replica.cc:333] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: stopping tablet replica
W20260709 13:21:49.056871 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.057060  8925 raft_consensus.cc:2243] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.058164  8925 raft_consensus.cc:2272] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.062345  8925 tablet_replica.cc:333] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.063792  8925 raft_consensus.cc:2243] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.065384  8925 raft_consensus.cc:2272] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.069520  8925 tablet_replica.cc:333] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.070673  8925 raft_consensus.cc:2243] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.071513  8925 raft_consensus.cc:2272] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.075541  8925 tablet_replica.cc:333] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.076664  8925 raft_consensus.cc:2243] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.079885  8925 raft_consensus.cc:2272] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.080155 18993 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:49.083283 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.084739  8925 tablet_replica.cc:333] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.086043  8925 raft_consensus.cc:2243] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.087306  8925 raft_consensus.cc:2272] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.091536  8925 tablet_replica.cc:333] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.096029  8925 raft_consensus.cc:2243] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.096961  8925 raft_consensus.cc:2272] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.100558 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.101121  8925 tablet_replica.cc:333] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.102555  8925 raft_consensus.cc:2243] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.104315  8925 raft_consensus.cc:2272] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.108317  8925 tablet_replica.cc:333] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.109462  8925 raft_consensus.cc:2243] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.110991  8925 raft_consensus.cc:2272] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.114916  8925 tablet_replica.cc:333] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.116155  8925 raft_consensus.cc:2243] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.117614  8925 raft_consensus.cc:2272] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.121646  8925 tablet_replica.cc:333] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.122803  8925 raft_consensus.cc:2243] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.124428  8925 raft_consensus.cc:2272] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.128249  8925 tablet_replica.cc:333] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.129339  8925 raft_consensus.cc:2243] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:21:49.129868 18993 consensus_peers.cc:597] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.130704  8925 raft_consensus.cc:2272] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.134676  8925 tablet_replica.cc:333] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.135885  8925 raft_consensus.cc:2243] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.137411  8925 raft_consensus.cc:2272] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.141269  8925 tablet_replica.cc:333] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.142513  8925 raft_consensus.cc:2243] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.143343  8925 raft_consensus.cc:2272] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.147104  8925 tablet_replica.cc:333] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.148198  8925 raft_consensus.cc:2243] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.149753  8925 raft_consensus.cc:2272] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.153594  8925 tablet_replica.cc:333] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.154702  8925 raft_consensus.cc:2243] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.156067  8925 raft_consensus.cc:2272] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.160100  8925 tablet_replica.cc:333] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.161247  8925 raft_consensus.cc:2243] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.162043  8925 raft_consensus.cc:2272] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.166483  8925 tablet_replica.cc:333] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.167729  8925 raft_consensus.cc:2243] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.168623  8925 raft_consensus.cc:2272] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.166579 18919 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.172417  8925 tablet_replica.cc:333] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.173475  8925 raft_consensus.cc:2243] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.176837  8925 raft_consensus.cc:2272] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.180580  8925 tablet_replica.cc:333] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.181531  8925 raft_consensus.cc:2243] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.182168  8925 raft_consensus.cc:2272] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.185379  8925 tablet_replica.cc:333] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.186242  8925 raft_consensus.cc:2243] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.187485  8925 raft_consensus.cc:2272] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.191738  8925 tablet_replica.cc:333] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.192502  8925 raft_consensus.cc:2243] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.193089  8925 raft_consensus.cc:2272] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.196431  8925 tablet_replica.cc:333] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.197540  8925 raft_consensus.cc:2243] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.198129  8925 raft_consensus.cc:2272] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.202003  8925 tablet_replica.cc:333] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: stopping tablet replica
W20260709 13:21:49.202279 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:49.202862  8925 raft_consensus.cc:2243] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:21:49.204130  8925 raft_consensus.cc:2272] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:49.208241  8925 tablet_replica.cc:333] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: stopping tablet replica
I20260709 13:21:49.209012  8925 raft_consensus.cc:2243] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:49.209775  8925 raft_consensus.cc:2272] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:49.212466 18993 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:49.214150 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:49.343371 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:50.036037 19623 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.036821 19623 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.045966 19600 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.046728 19600 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.049455 19600 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.054672 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.056454 19623 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:50.058419 18845 leader_election.cc:336] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.059149 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.059057 19623 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.061280 19673 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.068827 19673 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.071256 18844 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.072647 19600 raft_consensus.cc:2749] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.073117 19623 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.075853 19623 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.072636 19673 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.080855 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.086087 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:50.084875 18993 leader_election.cc:336] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.070873 18845 leader_election.cc:336] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.095976 18844 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.097187 19623 raft_consensus.cc:2749] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.098652 19673 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.099375 18991 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.099404 19673 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.101138 19660 raft_consensus.cc:2749] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.104518 19673 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.107323 19623 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.108040 19623 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:50.112223 18993 leader_election.cc:336] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.108620 18845 leader_election.cc:336] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.114341 19623 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.115052 19600 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.115355 18844 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.115929 19600 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.116346 19669 raft_consensus.cc:2749] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.118672 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.118724 19600 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.121032 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.121737 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
W20260709 13:21:50.122943 18845 leader_election.cc:336] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.123462 18991 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.125854 19589 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
W20260709 13:21:50.126602 18845 leader_election.cc:336] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.126561 19589 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.124926 19673 raft_consensus.cc:2749] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.128358 18844 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.129549 19623 raft_consensus.cc:2749] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.130249 18844 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.131552 19623 raft_consensus.cc:2749] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.133853 19589 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.134621 19673 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.135195 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.135322 19673 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.138839 19673 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.140614 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:50.141932 18919 leader_election.cc:336] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.143219 18919 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
W20260709 13:21:50.144009 18993 leader_election.cc:336] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.144168 19589 raft_consensus.cc:2749] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.144747 18993 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.145989 19673 raft_consensus.cc:2749] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.184686 19673 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.185369 19673 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.187865 19673 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.189330 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.190274 18876 raft_consensus.cc:2468] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.191546 18991 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.192387 19673 raft_consensus.cc:2804] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:50.192674 18993 leader_election.cc:336] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.192953 19673 raft_consensus.cc:493] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.193573 19673 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.197505 19660 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.198428 19660 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.201814 19660 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.202615 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.202415 19673 raft_consensus.cc:515] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.205389 19673 leader_election.cc:290] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.206812 19623 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.207324 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d850a82d849e47ef9576b6d0b73700c0" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.208158 18876 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.207778 19623 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.221958 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.222905 19026 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.223073 19623 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.224488 18844 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.226161 19669 raft_consensus.cc:2804] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.226640 19669 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.227022 19669 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.232112 19623 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.232648 19600 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.232986 19623 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.233278 19600 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.233891 19407 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.234633 19407 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.236580 19600 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.238408 19662 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.239238 19662 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.240245 19673 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.242790 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.243222 18876 raft_consensus.cc:2468] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:50.245306 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.249480 19623 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.250890 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.252018 19023 raft_consensus.cc:2468] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.241111 19673 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.253472 19589 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.253799 18844 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.254623 19589 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.264529 19662 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.257004 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.257303 19407 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.257700 19623 raft_consensus.cc:2804] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.258885 18991 leader_election.cc:304] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.272658 19660 raft_consensus.cc:2804] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.273310 19660 raft_consensus.cc:697] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:50.274665 19660 consensus_queue.cc:237] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.278534 19589 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.256131 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.279852 19623 raft_consensus.cc:493] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.259967 19669 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.283471 19623 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:50.281286 18845 leader_election.cc:336] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.286531 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
W20260709 13:21:50.290315 18993 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.289180 19600 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.290943 18993 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:50.289703 19673 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.291236 19600 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.292980 19669 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.290686 19023 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.294409 19665 raft_consensus.cc:2749] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.295531 19665 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.296293 19665 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.286564 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.300406 19026 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:50.300601 19665 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.299526 19600 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:50.300894 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.301836 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.302596 18876 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.303920 18991 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.304922 19665 raft_consensus.cc:2804] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.305456 19665 raft_consensus.cc:493] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.305990 19665 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:50.309649 18993 leader_election.cc:336] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.312263 18919 leader_election.cc:336] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.312925 18919 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:50.313148 18845 leader_election.cc:336] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.318001 18845 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.324299 19600 raft_consensus.cc:2749] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.323210 18918 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.325634 19589 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.322993 19023 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:50.326179 19589 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.326735 19589 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.327813 18844 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
W20260709 13:21:50.329049  8925 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 18826)
I20260709 13:21:50.329560 19600 raft_consensus.cc:2804] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.330319 19600 raft_consensus.cc:697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:50.331447 19600 consensus_queue.cc:237] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:50.326198 18845 leader_election.cc:336] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.343325 19669 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.324476 19662 raft_consensus.cc:2749] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.344368 19669 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.347533 19669 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.364650 19678 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.365587 19678 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:50.366617 18993 leader_election.cc:336] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.367391 18993 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.368413 19678 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.368674 19677 raft_consensus.cc:2749] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.369807 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.370689 18880 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.357942 19665 raft_consensus.cc:515] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.372272 18991 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.374040 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.331475 19623 raft_consensus.cc:515] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.373262 19677 raft_consensus.cc:2804] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.375919 19677 raft_consensus.cc:493] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.376543 19677 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:50.370985 18993 leader_election.cc:336] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.383142 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f439429f2a945c5b906f0a9e2d57e41" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:50.384177 19023 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.384246 19623 leader_election.cc:290] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:50.349844 18919 leader_election.cc:336] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.392372 18845 leader_election.cc:336] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.394193 18845 leader_election.cc:336] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.393493 18919 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
W20260709 13:21:50.396050 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 46240us
I20260709 13:21:50.394719 18845 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:50.396778 19407 raft_consensus.cc:2749] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.399646 19623 raft_consensus.cc:2749] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.392537 19674 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.400725 19674 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.404664 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7175a5fa52a94e2d8504a19eb7aa1421" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.405560 18880 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.406188 19665 leader_election.cc:290] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.406925 19674 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:50.408164 18993 leader_election.cc:336] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.413339 19665 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.414182 19665 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.414804 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:50.416111 18993 leader_election.cc:336] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.436367 19023 raft_consensus.cc:2468] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:50.437121 19677 raft_consensus.cc:515] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.438992 19674 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.448514 19674 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.448971 19677 leader_election.cc:290] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.449337 18702 catalog_manager.cc:5697] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:50.451081 19674 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.452733 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.454545 19679 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.460826 19679 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.455049 18876 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:50.453248 18844 leader_election.cc:304] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.464529 18917 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
W20260709 13:21:50.453089 18845 leader_election.cc:336] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.465704 18845 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29
I20260709 13:21:50.468922 19660 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.451745 18701 catalog_manager.cc:5697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:50.469895 19660 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.441671 19589 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.472604 19660 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.468866 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3b2d906586545bda1c94132ed20782a" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.470095 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.473624 19682 raft_consensus.cc:2804] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.473650 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.474583 18871 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.474906 19682 raft_consensus.cc:697] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:50.474996 18877 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.456290 19665 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.476225 19682 consensus_queue.cc:237] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.477092 18991 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.478523 19623 raft_consensus.cc:2749] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.479919 18879 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
W20260709 13:21:50.480409 18919 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.473991 19674 raft_consensus.cc:2804] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.481230 19674 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.481609 19674 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.484180 18991 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.485391 19660 raft_consensus.cc:2804] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:21:50.473742 18845 leader_election.cc:336] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.487257 19665 raft_consensus.cc:2804] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.487879 19665 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.488298 19665 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.474190 19589 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.485867 19660 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.492012 19660 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.493283 18880 raft_consensus.cc:2468] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:50.453449 19669 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.495560 19669 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.497021 18991 leader_election.cc:304] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:50.498481 19677 raft_consensus.cc:2804] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.499287 19677 raft_consensus.cc:697] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
W20260709 13:21:50.499648 18919 leader_election.cc:336] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.500669 19677 consensus_queue.cc:237] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.503166 19623 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.503937 19623 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.514029 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.514914 18951 raft_consensus.cc:2468] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.516589 18994 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.517580 19673 raft_consensus.cc:2804] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.518018 19673 raft_consensus.cc:493] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.518433 19673 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.518996 19669 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:50.523293 19679 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.523721 19623 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
W20260709 13:21:50.525017 18993 leader_election.cc:336] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.525724 18993 leader_election.cc:336] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.522677 18919 leader_election.cc:336] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.527412 18919 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.527866 19673 raft_consensus.cc:515] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.528426 19407 raft_consensus.cc:2749] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:50.528581 18845 leader_election.cc:336] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.532449 19407 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.533078 19407 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.533806 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35fbf5e9b0e446b39953dd28e5aa71be" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:50.534513 18951 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.535382 19407 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:50.536788 18845 leader_election.cc:336] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.540849 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:50.543907 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 72119.6us
W20260709 13:21:50.544113 18993 leader_election.cc:336] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.544637 19677 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.545023 19623 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.538268 18871 raft_consensus.cc:2468] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:50.547875 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:50.549121 19026 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.549302 18951 raft_consensus.cc:2468] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
W20260709 13:21:50.550836 18993 leader_election.cc:336] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.542357 18919 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.557154 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.558115 18952 raft_consensus.cc:2468] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.559037 19623 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.559554 18949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.560568 18949 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.560719 18991 leader_election.cc:304] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:50.562186 19623 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:50.559836 19673 leader_election.cc:290] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.563436 19673 raft_consensus.cc:2804] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.565122 19677 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.569100 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.569941 18950 raft_consensus.cc:2468] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.570394 19623 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.571434 19623 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.572383 19673 raft_consensus.cc:697] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:50.573208 19677 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.574620 19623 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.575337 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.598042 19023 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.580593 19674 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.601188 19674 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.589980 19026 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 2.
I20260709 13:21:50.576630 18994 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.602900 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.579810 19665 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.583576 18846 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:50.573611 19673 consensus_queue.cc:237] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.596886 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.603801 18871 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.610508 18844 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.617272 19682 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.619300 19682 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.596158 19677 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.623345 19678 raft_consensus.cc:2804] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.623896 19678 raft_consensus.cc:493] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.623788 19677 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.624398 19678 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.627760 19681 raft_consensus.cc:2804] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.628432 19681 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.628839 19681 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.629065 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.629905 18879 raft_consensus.cc:2468] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.631558 18991 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.633255 19679 raft_consensus.cc:2804] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.633845 19679 raft_consensus.cc:493] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.634379 19679 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.633970 18918 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:21:50.639283 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 55883.7us
I20260709 13:21:50.612857 18994 leader_election.cc:304] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.644966 18846 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:50.641587 19623 raft_consensus.cc:2804] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.640959 19677 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.648922 19675 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.648680 19600 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.681098 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:50.697911 18950 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
I20260709 13:21:50.697695 19600 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.672215 19682 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.686276 18871 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 2.
I20260709 13:21:50.659286 19623 raft_consensus.cc:493] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.688920 19678 raft_consensus.cc:515] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.679987 19677 raft_consensus.cc:2804] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.680966 19589 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
W20260709 13:21:50.693197 18845 leader_election.cc:336] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.694525 19665 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:50.706955 18991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 54502.1us
W20260709 13:21:50.710522 18845 leader_election.cc:336] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.716292 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 74395.1us
W20260709 13:21:50.718990 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 106303us
I20260709 13:21:50.710089 19623 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.710249 19682 raft_consensus.cc:2804] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.681327 19660 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.719065 18917 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:50.708890 19675 raft_consensus.cc:697] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:50.716133 19677 raft_consensus.cc:697] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:50.727098 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.732587 18877 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.733614 19673 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.734381 19673 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.734171 19677 consensus_queue.cc:237] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.724359 19682 raft_consensus.cc:493] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.734616 19675 consensus_queue.cc:237] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:50.735538 19682 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.737224 19673 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.738517 19662 raft_consensus.cc:2804] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.714816 19589 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.739347 19662 raft_consensus.cc:697] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:50.740398 19662 consensus_queue.cc:237] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.741174 19589 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:21:50.743637 18845 leader_election.cc:336] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.747707 18993 leader_election.cc:336] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.790174 19678 leader_election.cc:290] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:50.729555 19600 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:50.795399 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 153071us
W20260709 13:21:50.805253 18993 leader_election.cc:336] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.810168 18993 leader_election.cc:336] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.817203 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.818109 19679 raft_consensus.cc:515] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.819555 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
I20260709 13:21:50.820528 18871 raft_consensus.cc:2468] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 1.
I20260709 13:21:50.830963 18877 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
W20260709 13:21:50.846253 18846 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 128991us
I20260709 13:21:50.818189 18879 raft_consensus.cc:2468] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 1.
I20260709 13:21:50.811925 19681 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.840981 18917 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:50.850045 19685 raft_consensus.cc:2804] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.850500 19685 raft_consensus.cc:493] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.850878 19685 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.856443 19681 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:21:50.861603 18919 leader_election.cc:336] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.861872 19660 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.863229 19623 raft_consensus.cc:515] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:50.863163 18919 leader_election.cc:336] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.864820 19679 leader_election.cc:290] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.866202 18991 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.867205 19679 raft_consensus.cc:2804] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.867668 19679 raft_consensus.cc:493] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.868067 19679 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.847553 18846 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:50.871388 19682 raft_consensus.cc:515] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:50.872594 18991 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:50.873912 19660 raft_consensus.cc:2804] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:50.872462 19681 raft_consensus.cc:2749] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.874536 19660 raft_consensus.cc:697] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:50.875696 19660 consensus_queue.cc:237] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.876300 19623 leader_election.cc:290] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:50.877497 19679 raft_consensus.cc:515] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:50.881248 18846 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:21:50.880626 18919 leader_election.cc:336] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.885936 18919 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:21:50.895313 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a01b903fffc64c949e13c3c74abe8489" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.896219 18879 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:50.899791 18845 leader_election.cc:336] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.900696 19698 raft_consensus.cc:2749] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:21:50.904165 18877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.905109 18877 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.914954 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:21:50.916139 19026 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 1.
W20260709 13:21:50.926633 18993 leader_election.cc:336] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.929598 18993 leader_election.cc:336] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:50.930990 18993 leader_election.cc:336] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.928229 19623 raft_consensus.cc:2804] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:50.934051 18950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b57e1d1c76904a8492670c30af464d8d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:50.973385 18950 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.973888 19679 leader_election.cc:290] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:50.964450 19623 raft_consensus.cc:493] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:50.976008 18877 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:50.976363 19623 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.977972 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:50.978703 19026 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.971987 19682 leader_election.cc:290] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:50.985507 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab7b798c63c94bce8e69fefd480e295b" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:50.986297 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04089f925f3e44ab917020605bd8a75e" candidate_uuid: "f3e2fa7cf3974c27a19386f362688d29" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:50.987222 18880 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:50.990042 18993 leader_election.cc:336] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.986347 18952 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.990427 18844 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:50.990859 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcc4d045faa6417e894366a8fb7b9f52" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
W20260709 13:21:50.991150 18993 leader_election.cc:336] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:50.991636 18951 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:50.996279 18871 raft_consensus.cc:1275] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:50.980381 19685 raft_consensus.cc:515] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:51.001602 19685 leader_election.cc:290] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Requested vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:21:51.007642 19681 raft_consensus.cc:2804] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:21:51.008338 19681 raft_consensus.cc:493] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:21:51.008914 19681 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:51.008462 18845 leader_election.cc:336] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:51.012522 18991 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:21:51.009775 18845 leader_election.cc:336] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:51.013809 18845 leader_election.cc:336] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:51.019568 18871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f302bf297c554158b7f4ebcd5c7760bc" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:21:51.031206 19027 raft_consensus.cc:1275] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.033211 19023 raft_consensus.cc:1275] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.011889 18699 catalog_manager.cc:5697] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 2 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.035570 18879 raft_consensus.cc:2468] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:51.042814 18991 leader_election.cc:304] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:21:51.023825 18919 leader_election.cc:336] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:51.049045 18702 catalog_manager.cc:5697] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.025735 19677 raft_consensus.cc:2804] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.026189 18951 raft_consensus.cc:2468] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:51.027974 19691 consensus_queue.cc:1048] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.030244 18871 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:51.057554 18845 consensus_peers.cc:597] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.060429 18701 catalog_manager.cc:5697] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.071938 18702 catalog_manager.cc:5697] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.082824 18702 catalog_manager.cc:5697] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.084857 18702 catalog_manager.cc:5697] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.051992 18952 raft_consensus.cc:2468] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:51.088865 19677 raft_consensus.cc:697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:51.090104 19677 consensus_queue.cc:237] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.095016 18846 leader_election.cc:304] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:51.096689 19702 raft_consensus.cc:2804] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.097221 19702 raft_consensus.cc:697] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:51.098121 19702 consensus_queue.cc:237] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:51.062541 19026 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
W20260709 13:21:51.069322 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 45547.8us
I20260709 13:21:51.063709 18880 raft_consensus.cc:2468] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e2fa7cf3974c27a19386f362688d29 in term 2.
I20260709 13:21:51.112383 18991 leader_election.cc:304] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:21:51.109349 18699 catalog_manager.cc:5697] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 2 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.071942 19665 raft_consensus.cc:2804] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.067658 19681 raft_consensus.cc:515] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.117244 19681 leader_election.cc:290] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:51.128330 19665 raft_consensus.cc:697] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:51.129106 19660 raft_consensus.cc:2804] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:21:51.129581 18845 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.129817 19660 raft_consensus.cc:697] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:51.129652 19665 consensus_queue.cc:237] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.131572 19623 raft_consensus.cc:515] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.130959 19660 consensus_queue.cc:237] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.136852 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09abb2d24f6748bf938539c210dc01fc" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:21:51.137691 18951 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:51.144246 18844 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:51.151901 18951 raft_consensus.cc:2468] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:51.153580 18994 leader_election.cc:304] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:51.043054 18950 raft_consensus.cc:2468] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:51.154642 18846 leader_election.cc:304] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:21:51.157383 19693 raft_consensus.cc:2804] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.158006 19693 raft_consensus.cc:697] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:51.158274 19701 raft_consensus.cc:2804] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.158880 19701 raft_consensus.cc:697] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Leader. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:21:51.159325 19696 consensus_queue.cc:1048] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:21:51.159205 19693 consensus_queue.cc:237] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.160161 19701 consensus_queue.cc:237] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:51.155010 19682 raft_consensus.cc:2804] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.162453 19682 raft_consensus.cc:697] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:51.163770 19682 consensus_queue.cc:237] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.165663 19023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e076455b2f73491282b009a90112d07d" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:21:51.166567 19023 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:51.173012 18993 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.173446 18846 leader_election.cc:304] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:21:51.173647 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 104794us
W20260709 13:21:51.174154 18993 leader_election.cc:336] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:51.184422 19623 leader_election.cc:290] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:51.181192 19696 raft_consensus.cc:2804] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.187503 19696 raft_consensus.cc:697] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:51.188592 19696 consensus_queue.cc:237] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:51.189618 18845 leader_election.cc:336] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:21:51.196009 18880 raft_consensus.cc:1275] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.198889 19694 consensus_queue.cc:1048] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.153400 19681 consensus_queue.cc:1048] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:21:51.221239 18879 raft_consensus.cc:1275] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer de62ff9486f7483e8f0e0b93dcca24bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.241495 19407 consensus_queue.cc:1048] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:21:51.244974 18845 leader_election.cc:336] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:21:51.251299 18993 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.259593 18871 raft_consensus.cc:2468] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 2.
I20260709 13:21:51.275982 18702 catalog_manager.cc:5697] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:51.329679  8925 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 18826)
I20260709 13:21:51.362232 18871 raft_consensus.cc:1275] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:51.365569 18917 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 126063us
I20260709 13:21:51.369609 18917 leader_election.cc:304] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:51.390130 18877 raft_consensus.cc:1275] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.396682 19685 raft_consensus.cc:2804] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:21:51.397516 19685 raft_consensus.cc:697] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Becoming Leader. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:21:51.406208 19027 raft_consensus.cc:1275] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer de62ff9486f7483e8f0e0b93dcca24bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.398509 19685 consensus_queue.cc:237] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:51.415343 18919 consensus_peers.cc:597] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.418532 18919 consensus_peers.cc:597] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.423182 19712 log.cc:927] Time spent T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:21:51.424955 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.402818 19023 raft_consensus.cc:2468] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:21:51.431530 18844 leader_election.cc:304] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
I20260709 13:21:51.432850 19681 raft_consensus.cc:2804] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:21:51.454416 18919 consensus_peers.cc:597] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.446657 19681 raft_consensus.cc:697] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:21:51.456784 19681 consensus_queue.cc:237] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:51.457301 18950 raft_consensus.cc:1275] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.460484 19407 consensus_queue.cc:1048] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.472795 19673 consensus_queue.cc:1048] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:21:51.470979 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.481247 18993 consensus_peers.cc:597] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.481815 18993 consensus_peers.cc:597] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.481987 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.485250 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.491871 18702 catalog_manager.cc:5697] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.506553 18702 catalog_manager.cc:5697] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.512705 18702 catalog_manager.cc:5697] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.526194 18702 catalog_manager.cc:5697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68). New cstate: current_term: 2 leader_uuid: "f3e2fa7cf3974c27a19386f362688d29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:51.506297 19678 consensus_queue.cc:1048] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:21:51.506651 19677 consensus_queue.cc:1048] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 13:21:51.555353 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 44975us
W20260709 13:21:51.592866 19719 log.cc:927] Time spent T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: Append to log took a long time: real 0.141s	user 0.003s	sys 0.001s
W20260709 13:21:51.612514 19716 log.cc:927] Time spent T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: Append to log took a long time: real 0.139s	user 0.002s	sys 0.000s
I20260709 13:21:51.614842 18948 raft_consensus.cc:1275] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:51.612753 18919 consensus_peers.cc:597] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.616695 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.624318 19693 consensus_queue.cc:1048] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:51.641423 18845 consensus_peers.cc:597] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.690579 18876 raft_consensus.cc:1275] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.710098 19677 consensus_queue.cc:1048] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:51.701048 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 180657us
W20260709 13:21:51.716825 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.717881 18919 consensus_peers.cc:597] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.718520 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.719067 18919 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.719688 18919 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.712266 18993 consensus_peers.cc:597] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.742460 19734 log.cc:927] Time spent T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:21:51.743922 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 71408.4us
I20260709 13:21:51.762529 18951 raft_consensus.cc:1275] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:51.773414 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.790778 18845 consensus_peers.cc:597] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.791704 18845 consensus_peers.cc:597] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.792309 18845 consensus_peers.cc:597] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.792832 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.798466 18880 raft_consensus.cc:1275] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.798142 19696 consensus_queue.cc:1048] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.800194 18699 catalog_manager.cc:5697] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:51.802129 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 48405.4us
I20260709 13:21:51.820534 19679 consensus_queue.cc:1048] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:21:51.814256 18699 catalog_manager.cc:5697] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:51.826546 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.828159 18993 consensus_peers.cc:597] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.826485 18699 catalog_manager.cc:5697] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:51.856446 18919 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.860342 18993 consensus_peers.cc:597] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.861429 18993 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.858862 19028 raft_consensus.cc:1275] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:51.878530 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.895696 18951 raft_consensus.cc:1275] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.906174 19682 consensus_queue.cc:1048] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:51.910178 18993 consensus_peers.cc:597] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:51.925486 19735 consensus_queue.cc:1048] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:51.934537 18845 consensus_peers.cc:597] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.940318 18993 consensus_peers.cc:597] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.941195 18993 consensus_peers.cc:597] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:51.941701 18993 consensus_peers.cc:597] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:51.942135 18993 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.951505 18702 catalog_manager.cc:5697] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67). New cstate: current_term: 2 leader_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:51.925817 18951 raft_consensus.cc:1275] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:51.966509 18877 raft_consensus.cc:1275] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer de62ff9486f7483e8f0e0b93dcca24bb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:21:51.967193 18845 consensus_peers.cc:597] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:51.972587 19681 consensus_queue.cc:1048] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:21:51.972820 19741 consensus_queue.cc:1048] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:51.994263 18993 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:52.020278 18873 raft_consensus.cc:1275] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer f3e2fa7cf3974c27a19386f362688d29: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.023578 19726 consensus_queue.cc:1048] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:52.028765 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:52.068488 18919 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:52.081380 19020 raft_consensus.cc:1275] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:21:52.082422 18699 catalog_manager.cc:5697] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from 878e795a8488495f978538952e7b2324 (127.8.183.65) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:52.087062 19696 consensus_queue.cc:1048] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:21:52.113438 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:52.295454 18993 consensus_peers.cc:597] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:52.347070  8925 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 18826)
W20260709 13:21:53.253656 18993 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.347862  8925 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 18826)
W20260709 13:21:53.555505 18993 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.605171 18993 consensus_peers.cc:597] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.610719 18845 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.614005 18919 consensus_peers.cc:597] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.653366 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.691372 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.733771 18845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111) [suppressed 403 similar messages]
W20260709 13:21:53.737169 18845 consensus_peers.cc:597] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.747968 18845 consensus_peers.cc:597] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.791273 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.794894 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.817258 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.825598 18919 consensus_peers.cc:597] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.840461 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.842530 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.870538 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.949895 18919 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.959235 18993 consensus_peers.cc:597] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:53.960096 18993 consensus_peers.cc:597] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:53.977937 18993 consensus_peers.cc:597] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.000233 18919 consensus_peers.cc:597] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.090433 18919 consensus_peers.cc:597] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.116137 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.127389 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.135847 18919 consensus_peers.cc:597] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.142808 18993 consensus_peers.cc:597] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.185150 18993 consensus_peers.cc:597] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.197650 18993 consensus_peers.cc:597] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.228066 18993 consensus_peers.cc:597] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.244089 18919 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.267490 18993 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.296128 18845 consensus_peers.cc:597] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.302374 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.310545 18845 consensus_peers.cc:597] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.310603 18919 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.348630  8925 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 18826)
W20260709 13:21:54.355423 18845 consensus_peers.cc:597] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.362332 18993 consensus_peers.cc:597] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.364338 18845 consensus_peers.cc:597] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.389819 18845 consensus_peers.cc:597] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.430748 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.442938 18993 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.457093 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.508232 18919 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:54.556644 18993 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:54.747822 18993 consensus_peers.cc:597] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:55.349321  8925 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 18826)
W20260709 13:21:56.067353 18845 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.091988 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 54508.1us
W20260709 13:21:56.114410 18919 consensus_peers.cc:597] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.208698 18845 consensus_peers.cc:597] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.241333 18845 consensus_peers.cc:597] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.244915 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.247584 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.299431 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.305872 18919 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.326342 18919 consensus_peers.cc:597] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.345952 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.350272  8925 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 18826)
W20260709 13:21:56.406288 18919 consensus_peers.cc:597] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.444150 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.472685 18993 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.478353 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.541558 18993 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.544335 18919 consensus_peers.cc:597] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.600831 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.622812 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.627573 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.648154 18993 consensus_peers.cc:597] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.685385 18919 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.716960 18919 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.737767 18845 consensus_peers.cc:597] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.760095 18919 consensus_peers.cc:597] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.767966 18919 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.811053 18993 consensus_peers.cc:597] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.835521 18845 consensus_peers.cc:597] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:56.837868  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:21:56.877177 18845 consensus_peers.cc:597] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:56.892522 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.897576 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.920418 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:56.993826 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:57.020126 18845 consensus_peers.cc:597] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:57.020655  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:21:57.031778 19771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:57.031868 19772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:21:57.033236 19774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:21:57.038749  8925 server_base.cc:1061] running on GCE node
W20260709 13:21:57.040030 18845 consensus_peers.cc:597] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:57.042734  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:21:57.043200  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:21:57.043496  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603317043474 us; error 0 us; skew 500 ppm
I20260709 13:21:57.044492  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:21:57.048635  8925 webserver.cc:533] Webserver started at http://127.8.183.65:40061/ using document root <none> and password file <none>
I20260709 13:21:57.049459  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:21:57.049712  8925 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:21:57.054207  8925 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:21:57.065804 19779 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:21:57.067318  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:21:57.067909  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "878e795a8488495f978538952e7b2324"
format_stamp: "Formatted at 2026-07-09 13:21:30 on dist-test-slave-ch3x"
I20260709 13:21:57.068785  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:21:57.091794  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:21:57.093796  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:21:57.096498  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:21:57.104274 19786 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:21:57.146212 18993 consensus_peers.cc:597] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:57.167578 18993 consensus_peers.cc:597] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:57.177141 18993 consensus_peers.cc:597] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:57.225821 18993 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:57.304098 18993 consensus_peers.cc:597] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:57.366597 18993 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:57.442152 18993 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:57.473290 18993 consensus_peers.cc:597] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:57.515097  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:21:57.515741  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.413s	user 0.009s	sys 0.003s
I20260709 13:21:57.516184  8925 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
W20260709 13:21:57.524201 18993 consensus_peers.cc:597] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:57.526290 19786 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:57.556627 19786 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:57.558169 19786 tablet_bootstrap.cc:492] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:57.559303 19786 ts_tablet_manager.cc:1403] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.004s
I20260709 13:21:57.563905 19786 raft_consensus.cc:359] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:57.565570 19786 raft_consensus.cc:740] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:57.567030 19786 consensus_queue.cc:260] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:57.654714 19786 ts_tablet_manager.cc:1434] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.095s	user 0.020s	sys 0.046s
I20260709 13:21:57.655983 19786 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:57.877811 19786 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:57.911896 19786 tablet_bootstrap.cc:492] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:57.956770 19786 ts_tablet_manager.cc:1403] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.301s	user 0.027s	sys 0.004s
I20260709 13:21:57.983632 19786 raft_consensus.cc:359] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:57.984822 19786 raft_consensus.cc:740] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.041114 19786 consensus_queue.cc:260] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.044142 19786 ts_tablet_manager.cc:1434] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 13:21:58.077313 19786 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:58.085242 18993 consensus_peers.cc:597] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:58.225345 18918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 166555us
W20260709 13:21:58.221616 18993 consensus_peers.cc:597] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:58.343032 19786 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.354992 19786 tablet_bootstrap.cc:492] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.429575 19786 ts_tablet_manager.cc:1403] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.353s	user 0.037s	sys 0.016s
W20260709 13:21:58.374086 18991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:37707, user_credentials={real_user=slave}} blocked reactor thread for 51671.8us
W20260709 13:21:58.422876 18919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 112161us
I20260709 13:21:58.440446 19786 raft_consensus.cc:359] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.451488 19786 raft_consensus.cc:740] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.454295 19786 consensus_queue.cc:260] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.456547 19786 ts_tablet_manager.cc:1434] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:21:58.457499 19786 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.477561 19786 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.478791 19786 tablet_bootstrap.cc:492] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.479755 19786 ts_tablet_manager.cc:1403] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:21:58.483050 19786 raft_consensus.cc:359] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.483944 19786 raft_consensus.cc:740] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.484824 19786 consensus_queue.cc:260] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.487210 19786 ts_tablet_manager.cc:1434] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:21:58.488190 19786 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.505867 19786 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.506970 19786 tablet_bootstrap.cc:492] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.507958 19786 ts_tablet_manager.cc:1403] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:21:58.511076 19786 raft_consensus.cc:359] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.511957 19786 raft_consensus.cc:740] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.512679 19786 consensus_queue.cc:260] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.514691 19786 ts_tablet_manager.cc:1434] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:21:58.515632 19786 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.517786  8925 ts_tablet_manager.cc:600] Registering tablets (31/45 complete)
I20260709 13:21:58.544095 19786 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.545414 19786 tablet_bootstrap.cc:492] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.546236 19786 ts_tablet_manager.cc:1403] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.004s
I20260709 13:21:58.549404 19786 raft_consensus.cc:359] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.550171 19786 raft_consensus.cc:740] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.550951 19786 consensus_queue.cc:260] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:58.553252 19786 ts_tablet_manager.cc:1434] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:21:58.554236 19786 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.580651 19786 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.581790 19786 tablet_bootstrap.cc:492] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.582585 19786 ts_tablet_manager.cc:1403] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.012s
I20260709 13:21:58.585610 19786 raft_consensus.cc:359] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.586344 19786 raft_consensus.cc:740] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.587154 19786 consensus_queue.cc:260] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.589432 19786 ts_tablet_manager.cc:1434] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:58.590317 19786 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.608847 19786 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.609975 19786 tablet_bootstrap.cc:492] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.610734 19786 ts_tablet_manager.cc:1403] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:21:58.613755 19786 raft_consensus.cc:359] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.614583 19786 raft_consensus.cc:740] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.615518 19786 consensus_queue.cc:260] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.617630 19786 ts_tablet_manager.cc:1434] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:21:58.618518 19786 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.633919  8925 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:21:58.634645  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 1.118s	user 0.392s	sys 0.018s
I20260709 13:21:58.649341 19786 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.650797 19786 tablet_bootstrap.cc:492] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.651919 19786 ts_tablet_manager.cc:1403] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.004s
I20260709 13:21:58.656075 19786 raft_consensus.cc:359] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.657258 19786 raft_consensus.cc:740] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.658298 19786 consensus_queue.cc:260] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.661859 19786 ts_tablet_manager.cc:1434] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:21:58.663198 19786 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.700762 19786 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.702414 19786 tablet_bootstrap.cc:492] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.703586 19786 ts_tablet_manager.cc:1403] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.008s
I20260709 13:21:58.707959 19786 raft_consensus.cc:359] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.709110 19786 raft_consensus.cc:740] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.710245 19786 consensus_queue.cc:260] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.713523 19786 ts_tablet_manager.cc:1434] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:21:58.714957 19786 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.742225 19786 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.743886 19786 tablet_bootstrap.cc:492] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.745026 19786 ts_tablet_manager.cc:1403] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.003s
I20260709 13:21:58.749452 19786 raft_consensus.cc:359] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.750650 19786 raft_consensus.cc:740] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.751888 19786 consensus_queue.cc:260] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.755301 19786 ts_tablet_manager.cc:1434] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 13:21:58.756704 19786 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:58.758569 18845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111) [suppressed 404 similar messages]
I20260709 13:21:58.804613 19786 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.806165 19786 tablet_bootstrap.cc:492] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.807056 19786 ts_tablet_manager.cc:1403] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.051s	user 0.026s	sys 0.004s
I20260709 13:21:58.812703 19786 raft_consensus.cc:359] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.813800 19786 raft_consensus.cc:740] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.815852 19786 consensus_queue.cc:260] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:58.818765 19786 ts_tablet_manager.cc:1434] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:21:58.819808 19786 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:58.830726 18993 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:58.866590 19786 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:21:58.875865 18845 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:58.876788 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:58.879534 19786 tablet_bootstrap.cc:492] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.880779 19786 ts_tablet_manager.cc:1403] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.012s
I20260709 13:21:58.884560 19786 raft_consensus.cc:359] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.885366 19786 raft_consensus.cc:740] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.886250 19786 consensus_queue.cc:260] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.889015 19786 ts_tablet_manager.cc:1434] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:21:58.889983 19786 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:58.892078 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:58.905989  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:40287
I20260709 13:21:58.906203 19876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:40287 every 8 connection(s)
I20260709 13:21:58.909323 19786 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:58.910615 19786 tablet_bootstrap.cc:492] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:58.911413 19786 ts_tablet_manager.cc:1403] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 13:21:58.914726 19786 raft_consensus.cc:359] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.915580 19786 raft_consensus.cc:740] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:58.916539 19786 consensus_queue.cc:260] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:58.919276 19786 ts_tablet_manager.cc:1434] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:58.920295 19786 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:58.995980 19786 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:21:58.997152 18919 consensus_peers.cc:597] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:58.997231 19786 tablet_bootstrap.cc:492] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.000654 19786 ts_tablet_manager.cc:1403] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.081s	user 0.018s	sys 0.037s
I20260709 13:21:59.004122 19786 raft_consensus.cc:359] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.004952 19786 raft_consensus.cc:740] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.005752 19786 consensus_queue.cc:260] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.008487 19786 ts_tablet_manager.cc:1434] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:21:59.008713 19877 heartbeater.cc:344] Connected to a master server at 127.8.183.126:45441
I20260709 13:21:59.009290 19877 heartbeater.cc:461] Registering TS with master...
I20260709 13:21:59.009639 19786 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.010459 19877 heartbeater.cc:507] Master 127.8.183.126:45441 requested a full tablet report, sending...
I20260709 13:21:59.029446 19786 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.030598 19786 tablet_bootstrap.cc:492] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.031459 19786 ts_tablet_manager.cc:1403] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.003s
I20260709 13:21:59.034176 19786 raft_consensus.cc:359] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.034909 19786 raft_consensus.cc:740] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.035300 18701 ts_manager.cc:194] Re-registered known tserver with Master: 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:21:59.035766 19786 consensus_queue.cc:260] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.038353 19786 ts_tablet_manager.cc:1434] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:59.039381 19786 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.063722 19786 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.064860 19786 tablet_bootstrap.cc:492] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.065714 19786 ts_tablet_manager.cc:1403] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.004s
I20260709 13:21:59.068240 19786 raft_consensus.cc:359] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:59.069074 19786 raft_consensus.cc:740] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.069892 19786 consensus_queue.cc:260] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:59.072613 19786 ts_tablet_manager.cc:1434] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:21:59.073731 19786 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.099484 19786 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.100752 19786 tablet_bootstrap.cc:492] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.101603 19786 ts_tablet_manager.cc:1403] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:21:59.104665 19786 raft_consensus.cc:359] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.105551 19786 raft_consensus.cc:740] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.106390 19786 consensus_queue.cc:260] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.108076 18701 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33062
I20260709 13:21:59.108745 19786 ts_tablet_manager.cc:1434] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:21:59.109884 19786 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.115607 19877 heartbeater.cc:499] Master 127.8.183.126:45441 was elected leader, sending a full tablet report...
I20260709 13:21:59.121227 19850 raft_consensus.cc:3060] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.130379 19786 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.133205 19786 tablet_bootstrap.cc:492] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.134145 19786 ts_tablet_manager.cc:1403] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.003s
I20260709 13:21:59.138353 19786 raft_consensus.cc:359] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:59.138844 18901 debug-util.cc:398] Leaking SignalData structure 0x7b080014c8c0 after lost signal to thread 8928
I20260709 13:21:59.139324 19786 raft_consensus.cc:740] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
W20260709 13:21:59.140522 18901 debug-util.cc:398] Leaking SignalData structure 0x7b0800283ae0 after lost signal to thread 18737
I20260709 13:21:59.140463 19786 consensus_queue.cc:260] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:21:59.142699 18901 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecd60 after lost signal to thread 18904
I20260709 13:21:59.143239 19786 ts_tablet_manager.cc:1434] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 13:21:59.144642 19786 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:59.146782 18901 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3f80 after lost signal to thread 18978
W20260709 13:21:59.148499 18901 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2700 after lost signal to thread 19052
I20260709 13:21:59.167547 19786 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.168867 19786 tablet_bootstrap.cc:492] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.170315 19786 ts_tablet_manager.cc:1403] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.000s
I20260709 13:21:59.173316 19786 raft_consensus.cc:359] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.173939 19786 raft_consensus.cc:740] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.174656 19786 consensus_queue.cc:260] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.176438 19786 ts_tablet_manager.cc:1434] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:21:59.177426 19786 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.185920 19850 raft_consensus.cc:3060] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.202266 19790 raft_consensus.cc:493] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.203449 19790 raft_consensus.cc:515] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.213140 19786 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.214468 19786 tablet_bootstrap.cc:492] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.215435 19786 ts_tablet_manager.cc:1403] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.015s
I20260709 13:21:59.224439 19786 raft_consensus.cc:359] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.225363 19786 raft_consensus.cc:740] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.226480 19786 consensus_queue.cc:260] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.231294 19790 leader_election.cc:290] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:21:59.248256 19847 raft_consensus.cc:3060] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:59.265506 18993 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:59.248314 19786 ts_tablet_manager.cc:1434] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:21:59.272959 19786 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:59.277351 18845 consensus_peers.cc:597] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:59.297636 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:59.301246 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.302459 19849 raft_consensus.cc:3060] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:59.345603 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.373142 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" is_pre_election: true
I20260709 13:21:59.407166 19848 raft_consensus.cc:3060] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:21:59.412074 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" is_pre_election: true
W20260709 13:21:59.416229 18919 consensus_peers.cc:597] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:59.418319 19780 leader_election.cc:304] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 878e795a8488495f978538952e7b2324; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:21:59.420225 19894 raft_consensus.cc:2749] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:21:59.425051 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:59.427834 18919 consensus_peers.cc:597] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.434700 19786 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.436597 19786 tablet_bootstrap.cc:492] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.437541 19786 ts_tablet_manager.cc:1403] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.165s	user 0.019s	sys 0.009s
I20260709 13:21:59.457647 19848 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:59.463578 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.503391 19786 raft_consensus.cc:359] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.508795 19786 raft_consensus.cc:740] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.509895 19786 consensus_queue.cc:260] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.511853 19786 ts_tablet_manager.cc:1434] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.074s	user 0.011s	sys 0.001s
I20260709 13:21:59.512941 19786 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:59.585788 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:59.585870 18993 consensus_peers.cc:597] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.597898 19786 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.610085 19786 tablet_bootstrap.cc:492] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.616694 19786 ts_tablet_manager.cc:1403] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.104s	user 0.037s	sys 0.024s
I20260709 13:21:59.623487 19786 raft_consensus.cc:359] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.624387 19786 raft_consensus.cc:740] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.625125 19786 consensus_queue.cc:260] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.632745 19786 ts_tablet_manager.cc:1434] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:21:59.634428 19786 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.639914 19845 raft_consensus.cc:3060] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:21:59.642181 18919 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:59.663528 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:59.664157 18993 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:59.696174 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:37419, user_credentials={real_user=slave}} blocked reactor thread for 47551.4us
W20260709 13:21:59.729943 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 61642.6us
W20260709 13:21:59.742678 18993 consensus_peers.cc:597] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:59.754989 18845 consensus_peers.cc:597] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:59.772359 19786 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.773813 19786 tablet_bootstrap.cc:492] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.774694 19786 ts_tablet_manager.cc:1403] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.141s	user 0.022s	sys 0.031s
I20260709 13:21:59.778708 19786 raft_consensus.cc:359] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.779560 19786 raft_consensus.cc:740] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.780377 19786 consensus_queue.cc:260] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.783187 19786 ts_tablet_manager.cc:1434] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:21:59.784250 19786 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
W20260709 13:21:59.807145 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.851975 19786 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.853366 19786 tablet_bootstrap.cc:492] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.855753 19786 ts_tablet_manager.cc:1403] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.072s	user 0.027s	sys 0.025s
I20260709 13:21:59.859421 19786 raft_consensus.cc:359] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.861035 19786 raft_consensus.cc:740] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.862262 19786 consensus_queue.cc:260] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:21:59.887504 19786 ts_tablet_manager.cc:1434] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:21:59.889215 19786 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.910148 19786 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.911835 19786 tablet_bootstrap.cc:492] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.912851 19786 ts_tablet_manager.cc:1403] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.002s
I20260709 13:21:59.923976 19786 raft_consensus.cc:359] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:59.925547 19786 raft_consensus.cc:740] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.926435 19786 consensus_queue.cc:260] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:21:59.929402 19786 ts_tablet_manager.cc:1434] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:21:59.930461 19786 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:21:59.970455 19786 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:59.971840 19786 tablet_bootstrap.cc:492] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:21:59.972806 19786 ts_tablet_manager.cc:1403] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.003s
W20260709 13:21:59.975487 18993 consensus_peers.cc:597] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:59.976583 19786 raft_consensus.cc:359] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:21:59.977571 19786 raft_consensus.cc:740] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:21:59.978324 19786 consensus_queue.cc:260] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
W20260709 13:21:59.980641 18845 consensus_peers.cc:597] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:59.983808 19786 ts_tablet_manager.cc:1434] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:21:59.985071 19786 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.007308 19786 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.008637 19786 tablet_bootstrap.cc:492] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.009655 19786 ts_tablet_manager.cc:1403] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.003s
I20260709 13:22:00.013162 19786 raft_consensus.cc:359] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.014073 19786 raft_consensus.cc:740] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.015054 19786 consensus_queue.cc:260] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.017690 19786 ts_tablet_manager.cc:1434] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:00.019346 19786 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.034297 19845 raft_consensus.cc:3060] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.051896 19786 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.071353 19786 tablet_bootstrap.cc:492] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.072319 19786 ts_tablet_manager.cc:1403] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.006s
I20260709 13:22:00.075549 19786 raft_consensus.cc:359] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.076371 19786 raft_consensus.cc:740] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.084947 19786 consensus_queue.cc:260] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.088199 19786 ts_tablet_manager.cc:1434] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:22:00.099964 19786 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.128230 19786 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.129508 19786 tablet_bootstrap.cc:492] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.130357 19786 ts_tablet_manager.cc:1403] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.003s
I20260709 13:22:00.134352 19786 raft_consensus.cc:359] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.135306 19786 raft_consensus.cc:740] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.136317 19786 consensus_queue.cc:260] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.143594 19786 ts_tablet_manager.cc:1434] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.007s
I20260709 13:22:00.144672 19786 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.305943 19786 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.307379 19786 tablet_bootstrap.cc:492] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.308267 19786 ts_tablet_manager.cc:1403] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.164s	user 0.047s	sys 0.084s
I20260709 13:22:00.314031 19786 raft_consensus.cc:359] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.315131 19786 raft_consensus.cc:740] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.316226 19786 consensus_queue.cc:260] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.319095 19786 ts_tablet_manager.cc:1434] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:00.320657 19848 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.324689 19786 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.329879 19845 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.366680 19850 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.369611 19786 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.377432 19786 tablet_bootstrap.cc:492] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.379475 19786 ts_tablet_manager.cc:1403] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.004s
I20260709 13:22:00.401840 19786 raft_consensus.cc:359] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:00.402720 19786 raft_consensus.cc:740] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.407184 19786 consensus_queue.cc:260] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:00.410270 19786 ts_tablet_manager.cc:1434] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:22:00.411424 19786 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.438405 19786 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.439605 19786 tablet_bootstrap.cc:492] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.440457 19786 ts_tablet_manager.cc:1403] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.007s
I20260709 13:22:00.443528 19786 raft_consensus.cc:359] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:00.444221 19786 raft_consensus.cc:740] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.444998 19786 consensus_queue.cc:260] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:00.446976 19786 ts_tablet_manager.cc:1434] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:00.448915 19786 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.490226 19786 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.491690 19786 tablet_bootstrap.cc:492] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.492794 19845 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.492842 19786 ts_tablet_manager.cc:1403] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.007s
I20260709 13:22:00.510555 19786 raft_consensus.cc:359] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.517982 19786 raft_consensus.cc:740] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.519604 19786 consensus_queue.cc:260] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.524890 19786 ts_tablet_manager.cc:1434] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:22:00.532226 19786 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.605262 19786 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.607302 19786 tablet_bootstrap.cc:492] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.608198 19786 ts_tablet_manager.cc:1403] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.004s
I20260709 13:22:00.611548 19786 raft_consensus.cc:359] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.612385 19786 raft_consensus.cc:740] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.613209 19786 consensus_queue.cc:260] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.621517 19786 ts_tablet_manager.cc:1434] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:00.622852 19786 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.637880 19845 raft_consensus.cc:3060] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:00.728940 19786 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.730904 19786 tablet_bootstrap.cc:492] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.732417 19786 ts_tablet_manager.cc:1403] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.110s	user 0.022s	sys 0.008s
I20260709 13:22:00.737768 19786 raft_consensus.cc:359] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.738677 19786 raft_consensus.cc:740] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.739603 19786 consensus_queue.cc:260] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.742192 19786 ts_tablet_manager.cc:1434] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:00.747095 19786 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.885564 19786 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.886968 19786 tablet_bootstrap.cc:492] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.888401 19786 ts_tablet_manager.cc:1403] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.142s	user 0.042s	sys 0.067s
I20260709 13:22:00.892688 19786 raft_consensus.cc:359] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.894254 19786 raft_consensus.cc:740] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.895278 19786 consensus_queue.cc:260] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:00.897367 19786 ts_tablet_manager.cc:1434] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:22:00.898910 19786 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:00.922544 19786 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:00.924669 19786 tablet_bootstrap.cc:492] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:00.925724 19786 ts_tablet_manager.cc:1403] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.004s
I20260709 13:22:00.930521 19786 raft_consensus.cc:359] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.932672 19786 raft_consensus.cc:740] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:00.933959 19786 consensus_queue.cc:260] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:00.976540 19786 ts_tablet_manager.cc:1434] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.050s	user 0.018s	sys 0.021s
I20260709 13:22:00.980116 19786 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.010936 19786 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.012269 19786 tablet_bootstrap.cc:492] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.013233 19786 ts_tablet_manager.cc:1403] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.008s
I20260709 13:22:01.017004 19786 raft_consensus.cc:359] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.018029 19786 raft_consensus.cc:740] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.019030 19786 consensus_queue.cc:260] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.022053 19786 ts_tablet_manager.cc:1434] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:01.023795 19786 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.080413 19786 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.081755 19786 tablet_bootstrap.cc:492] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.082672 19786 ts_tablet_manager.cc:1403] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.059s	user 0.035s	sys 0.012s
I20260709 13:22:01.086120 19786 raft_consensus.cc:359] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.087209 19786 raft_consensus.cc:740] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.090345 19786 consensus_queue.cc:260] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.093461 19786 ts_tablet_manager.cc:1434] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:01.098356 19786 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.130805 19786 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.132259 19786 tablet_bootstrap.cc:492] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.133308 19786 ts_tablet_manager.cc:1403] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.035s	user 0.029s	sys 0.004s
I20260709 13:22:01.136574 19786 raft_consensus.cc:359] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.137529 19786 raft_consensus.cc:740] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.138597 19786 consensus_queue.cc:260] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.141423 19786 ts_tablet_manager.cc:1434] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:22:01.158288 19786 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.192965 19786 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.194283 19786 tablet_bootstrap.cc:492] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.195353 19786 ts_tablet_manager.cc:1403] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.037s	user 0.022s	sys 0.004s
I20260709 13:22:01.198709 19786 raft_consensus.cc:359] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.199717 19786 raft_consensus.cc:740] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.200821 19786 consensus_queue.cc:260] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.203876 19786 ts_tablet_manager.cc:1434] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:01.205901 19786 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.221398 19846 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.231907 19786 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.233265 19786 tablet_bootstrap.cc:492] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.234166 19786 ts_tablet_manager.cc:1403] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.008s
I20260709 13:22:01.237295 19786 raft_consensus.cc:359] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.238425 19786 raft_consensus.cc:740] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.239346 19786 consensus_queue.cc:260] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.242300 19786 ts_tablet_manager.cc:1434] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:01.243945 19786 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.269680 19786 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.270980 19786 tablet_bootstrap.cc:492] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.271880 19786 ts_tablet_manager.cc:1403] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:22:01.275089 19786 raft_consensus.cc:359] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.276208 19786 raft_consensus.cc:740] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.277103 19786 consensus_queue.cc:260] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:01.280030 19786 ts_tablet_manager.cc:1434] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:01.281229 19786 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Bootstrap starting.
I20260709 13:22:01.316215 19786 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:01.317592 19786 tablet_bootstrap.cc:492] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Bootstrap complete.
I20260709 13:22:01.318518 19786 ts_tablet_manager.cc:1403] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.005s
I20260709 13:22:01.322079 19786 raft_consensus.cc:359] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.322978 19786 raft_consensus.cc:740] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878e795a8488495f978538952e7b2324, State: Initialized, Role: FOLLOWER
I20260709 13:22:01.324051 19786 consensus_queue.cc:260] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:01.327235 19786 ts_tablet_manager.cc:1434] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324: Time spent starting tablet: real 0.008s	user 0.012s	sys 0.000s
I20260709 13:22:01.331861 19845 raft_consensus.cc:3060] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.366638 19848 raft_consensus.cc:3060] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.439378 19847 raft_consensus.cc:3060] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.536201 19847 raft_consensus.cc:3060] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:01.560402 19846 raft_consensus.cc:3060] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:04.182327 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 47147.5us
W20260709 13:22:06.049813 18901 diagnostics_log.cc:230] Unable to collect stacks to diagnostics log: Incomplete: not collecting stack trace since debugger or strace is attached
W20260709 13:22:06.305070 18993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:40287, user_credentials={real_user=slave}} blocked reactor thread for 66256.4us
W20260709 13:22:06.372057 18844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.68:46235, user_credentials={real_user=slave}} blocked reactor thread for 48233.4us
I20260709 13:22:08.946740  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:09.026693 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:09.027660 18951 raft_consensus.cc:606] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:09.031064 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:09.031807 18951 raft_consensus.cc:606] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:09.052057 19026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:09.052762 19026 raft_consensus.cc:606] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:09.056263 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:09.057029 18951 raft_consensus.cc:606] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:09.059844 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:09.060480 18951 raft_consensus.cc:606] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:22:09.064230 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:09.065091 18951 raft_consensus.cc:606] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:09.067284  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:09.069977  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:09.070290 19976 raft_consensus.cc:993] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 039c551e55fb43cdb5c3880b7cb91a0b to start an election
I20260709 13:22:09.071218 19976 raft_consensus.cc:1081] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 039c551e55fb43cdb5c3880b7cb91a0b to start an election
I20260709 13:22:09.073541 18875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773"
dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
 from {username='slave'} at 127.0.0.1:46332
I20260709 13:22:09.074347 18875 raft_consensus.cc:493] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.074826 18875 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.084226 18875 raft_consensus.cc:515] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:09.087579 18875 leader_election.cc:290] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:09.089057 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:09.090088 19026 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.091560 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd0d96cdc814e8fa997150f58dda773" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.092332 18951 raft_consensus.cc:3055] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.092662 18951 raft_consensus.cc:740] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:09.093706 18951 consensus_queue.cc:260] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:09.095822 18951 raft_consensus.cc:3060] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.106804 18951 raft_consensus.cc:2468] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:22:09.109835 18846 leader_election.cc:304] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:09.112916 19981 raft_consensus.cc:2804] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.113549 19981 raft_consensus.cc:697] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:22:09.114688 19981 consensus_queue.cc:237] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:09.123378 19026 raft_consensus.cc:2468] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:22:09.135965 18699 catalog_manager.cc:5697] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.144176 19975 raft_consensus.cc:993] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.145641 19969 raft_consensus.cc:1081] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.149649 19846 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:09.150532 19846 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.151448 19846 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.168495 19846 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.171450 19846 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:09.173017 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.174224 18951 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.183743 18951 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.185494 19783 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:09.187960 19997 raft_consensus.cc:2804] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.188329 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fde9dbeb78f450581397317487e31f0" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:09.189276 19026 raft_consensus.cc:3055] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.189651 19997 raft_consensus.cc:697] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:09.189656 19026 raft_consensus.cc:740] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:09.191262 19026 consensus_queue.cc:260] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.191226 19997 consensus_queue.cc:237] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.193187 19026 raft_consensus.cc:3060] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.203032 19026 raft_consensus.cc:2468] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.208505 18699 catalog_manager.cc:5697] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.223063 19976 raft_consensus.cc:993] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.223929 19990 raft_consensus.cc:1081] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.226168 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:09.226701 19848 raft_consensus.cc:493] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.226958 19848 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.233557 19848 raft_consensus.cc:515] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.236786 19848 leader_election.cc:290] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:22:09.236687 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:09.238123 18951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6870bd6f453d45a19a939d17ee0f92ad" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.238126 18875 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.238879 18951 raft_consensus.cc:3055] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.239310 18951 raft_consensus.cc:740] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:09.240290 18951 consensus_queue.cc:260] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.241904 18951 raft_consensus.cc:3060] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.248886 18875 raft_consensus.cc:2468] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.250180 18951 raft_consensus.cc:2468] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.251538 19780 leader_election.cc:304] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:22:09.252449 19997 raft_consensus.cc:2804] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.252959 19997 raft_consensus.cc:697] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:09.253854 19997 consensus_queue.cc:237] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.265836 18701 catalog_manager.cc:5697] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.396301 19985 raft_consensus.cc:993] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.396929 19976 raft_consensus.cc:1081] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.398779 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:09.399375 19848 raft_consensus.cc:493] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.399683 19848 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.407728 19848 raft_consensus.cc:515] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.410869 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:09.411067 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577d52a3cf7346349e47f24b3e1286e1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.412071 18880 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.412181 18947 raft_consensus.cc:3055] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:09.412672 18947 raft_consensus.cc:740] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:09.413497 19848 leader_election.cc:290] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:22:09.413714 18947 consensus_queue.cc:260] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.415586 18947 raft_consensus.cc:3060] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.428320 18947 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:09.428625 18880 raft_consensus.cc:2468] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:09.430096 19783 leader_election.cc:304] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:09.431378 19997 raft_consensus.cc:2804] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:09.431916 19997 raft_consensus.cc:697] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:09.432974 19997 consensus_queue.cc:237] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.445106 18701 catalog_manager.cc:5697] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.464458 19985 raft_consensus.cc:993] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.465099 19976 raft_consensus.cc:1081] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.466862 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:09.467519 19848 raft_consensus.cc:493] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.467947 19848 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.476500 19848 raft_consensus.cc:515] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.478641 19848 leader_election.cc:290] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:22:09.479779 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:09.479974 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e83e89f3973f4275a44d76a152e43126" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.480628 18880 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.480804 18947 raft_consensus.cc:3055] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:09.481294 18947 raft_consensus.cc:740] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:09.482102 18947 consensus_queue.cc:260] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.483460 18947 raft_consensus.cc:3060] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:09.491988 18880 raft_consensus.cc:2468] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.492609 18947 raft_consensus.cc:2468] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:09.493494 19780 leader_election.cc:304] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:22:09.494524 19997 raft_consensus.cc:2804] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:09.495209 19997 raft_consensus.cc:697] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:09.496182 19997 consensus_queue.cc:237] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:09.507557 18701 catalog_manager.cc:5697] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:09.516556 19026 raft_consensus.cc:1275] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.518438 19985 raft_consensus.cc:993] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.519038 19985 raft_consensus.cc:1081] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:09.520097 19983 consensus_queue.cc:1048] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.523562 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:09.524284 19848 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:09.524683 19848 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.538790 19848 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.541316 19848 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:09.543727 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:09.545151 18947 raft_consensus.cc:3055] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:09.545581 18947 raft_consensus.cc:740] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:09.545681 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:09.546371 19026 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.546916 18947 consensus_queue.cc:260] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.553495 18951 raft_consensus.cc:1275] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.555342 18947 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:09.555850 19026 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:09.557844 19781 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:09.557919 19983 consensus_queue.cc:1048] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.559062 19997 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:09.559933 19997 raft_consensus.cc:697] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:09.560999 19997 consensus_queue.cc:237] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:09.574785 19026 raft_consensus.cc:1275] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.577472 19998 consensus_queue.cc:1048] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.579936 18947 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:09.616706 18702 catalog_manager.cc:5697] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:09.623612 18951 raft_consensus.cc:1275] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.627775 19997 consensus_queue.cc:1048] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.661110 18951 raft_consensus.cc:1275] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.663676 20013 consensus_queue.cc:1048] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.694893 18880 raft_consensus.cc:1275] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.700079 20013 consensus_queue.cc:1048] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:09.836373 18880 raft_consensus.cc:1275] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:09.838205 20018 consensus_queue.cc:1048] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:09.850446 18951 raft_consensus.cc:1275] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:09.852327 20018 consensus_queue.cc:1048] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:09.893864 18951 raft_consensus.cc:1275] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.895453 20018 consensus_queue.cc:1048] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:09.909446 18880 raft_consensus.cc:1275] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:09.912817 19998 consensus_queue.cc:1048] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.071619  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:10.112753 19028 raft_consensus.cc:1275] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:10.114635 20013 consensus_queue.cc:1048] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:10.118750  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:10.128304 18947 raft_consensus.cc:1275] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:10.130684 20013 consensus_queue.cc:1048] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:10.225714 18951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:10.226589 18951 raft_consensus.cc:606] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 039c551e55fb43cdb5c3880b7cb91a0b
I20260709 13:22:10.234318 18947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:10.235146 18947 raft_consensus.cc:606] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:10.244256 19026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afdc899ecc694b09991abe77a2195448"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:10.245007 19026 raft_consensus.cc:606] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:10.248610 19985 raft_consensus.cc:993] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.248757 18947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6"
dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:39834
I20260709 13:22:10.249274 19985 raft_consensus.cc:1081] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.249665 18947 raft_consensus.cc:606] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:10.251830 19845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:10.252540 19845 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.253018 19845 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.266309 19845 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:10.268854 19845 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707), de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419)
I20260709 13:22:10.269690 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:10.270496 18880 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.271106 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "232db8a92f07473e91159b6e4182f3ef" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:10.272015 18947 raft_consensus.cc:3055] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.272378 18947 raft_consensus.cc:740] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:10.273366 18947 consensus_queue.cc:260] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:10.274972 18947 raft_consensus.cc:3060] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.287477 18947 raft_consensus.cc:2468] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:10.288959 19783 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:10.290112 19997 raft_consensus.cc:2804] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.290696 19997 raft_consensus.cc:697] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:10.291724 19997 consensus_queue.cc:237] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:10.303710 18880 raft_consensus.cc:2468] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:10.310065 18875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9"
dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:22:10.310770 18875 raft_consensus.cc:606] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:10.312541  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:22:10.313421 18701 catalog_manager.cc:5697] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.314451  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:10.362845 19985 raft_consensus.cc:993] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.363543 20027 raft_consensus.cc:1081] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.365262 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36100
I20260709 13:22:10.365820 19848 raft_consensus.cc:493] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.366151 19848 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.375694 19985 raft_consensus.cc:993] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: : Instructing follower 039c551e55fb43cdb5c3880b7cb91a0b to start an election
I20260709 13:22:10.375579 19848 raft_consensus.cc:515] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:10.376477 19985 raft_consensus.cc:1081] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Signalling peer 039c551e55fb43cdb5c3880b7cb91a0b to start an election
I20260709 13:22:10.378937 19848 leader_election.cc:290] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:10.378880 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:10.379896 18952 raft_consensus.cc:3055] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.380218 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4c3f02bc00e460dba727510708feee6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:10.380249 18952 raft_consensus.cc:740] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:10.380972 19026 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.381433 18952 consensus_queue.cc:260] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:10.382599 18952 raft_consensus.cc:3060] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.378657 18875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c"
dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
 from {username='slave'} at 127.0.0.1:46332
I20260709 13:22:10.384081 18875 raft_consensus.cc:493] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.384510 18875 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.393049 19026 raft_consensus.cc:2468] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:10.393515 18875 raft_consensus.cc:515] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:10.394886 18952 raft_consensus.cc:2468] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:10.394937 19781 leader_election.cc:304] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:10.396224 19997 raft_consensus.cc:2804] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.396749 18875 leader_election.cc:290] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:10.396799 19997 raft_consensus.cc:697] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:10.398010 19997 consensus_queue.cc:237] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:10.399173 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:10.399904 19028 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.409514 19028 raft_consensus.cc:2468] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:22:10.411312 19026 raft_consensus.cc:1240] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Rejecting Update request from peer de62ff9486f7483e8f0e0b93dcca24bb for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:10.411367 18844 leader_election.cc:304] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:10.412387 18952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14fe966c22b14da9985eba8dd9ed1f2c" candidate_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:10.412577 19983 raft_consensus.cc:2804] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:10.413384 19983 raft_consensus.cc:697] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Leader. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:22:10.413456 20027 consensus_queue.cc:1059] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.414245 19983 consensus_queue.cc:237] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:10.415331 19985 raft_consensus.cc:3055] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:10.415719 19985 raft_consensus.cc:740] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de62ff9486f7483e8f0e0b93dcca24bb, State: Running, Role: LEADER
I20260709 13:22:10.416654 19985 consensus_queue.cc:260] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } }
I20260709 13:22:10.418170 19985 raft_consensus.cc:3060] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:10.425951 18701 catalog_manager.cc:5697] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:10.441684 18952 raft_consensus.cc:2468] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 039c551e55fb43cdb5c3880b7cb91a0b in term 2.
I20260709 13:22:10.449779 18701 catalog_manager.cc:5697] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b reported cstate change: term changed from 1 to 2, leader changed from de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67) to 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.456429 20024 raft_consensus.cc:993] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.457248 19983 raft_consensus.cc:1081] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.459414 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36094
I20260709 13:22:10.460186 20001 raft_consensus.cc:993] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.460251 19848 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.461228 19848 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.461345 19986 raft_consensus.cc:1081] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:10.464152 19845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "afdc899ecc694b09991abe77a2195448"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:10.464892 19845 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:10.465267 19845 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.469729 19848 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.473433 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:10.473577 18880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3bb98bec02448d6990ba27836e9e7a9" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:10.474238 18880 raft_consensus.cc:3055] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:10.474633 18880 raft_consensus.cc:740] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:22:10.475577 18880 consensus_queue.cc:260] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.476855 18880 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.474238 19026 raft_consensus.cc:3060] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.479012 19845 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.481585 19845 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:10.482838 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:10.483453 19848 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:10.483675 18875 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.482888 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afdc899ecc694b09991abe77a2195448" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:10.486626 19028 raft_consensus.cc:3055] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:10.487025 19028 raft_consensus.cc:740] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:10.488068 19028 consensus_queue.cc:260] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.489573 19028 raft_consensus.cc:3060] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:10.490907 19026 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:10.492327 19781 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:10.494050 19997 raft_consensus.cc:2804] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:10.494653 19997 raft_consensus.cc:697] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:10.495096 18880 raft_consensus.cc:2468] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:10.495711 19997 consensus_queue.cc:237] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.498777 19028 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:10.505683 18875 raft_consensus.cc:2468] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:10.507285 19780 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:22:10.511161 20013 raft_consensus.cc:2804] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:10.511801 20013 raft_consensus.cc:697] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:10.512861 20013 consensus_queue.cc:237] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:10.526144 18701 catalog_manager.cc:5697] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.550881 18701 catalog_manager.cc:5697] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:10.675957 18875 raft_consensus.cc:1275] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.677449 20013 consensus_queue.cc:1048] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.692042 18952 raft_consensus.cc:1275] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.694061 19997 consensus_queue.cc:1048] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.857066 18947 raft_consensus.cc:1275] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.859076 19998 consensus_queue.cc:1048] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.871079 19028 raft_consensus.cc:1275] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.875057 20013 consensus_queue.cc:1048] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.903476 18947 raft_consensus.cc:1275] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.909564 19983 consensus_queue.cc:1048] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:10.925213 19028 raft_consensus.cc:1275] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 039c551e55fb43cdb5c3880b7cb91a0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:10.928300 20002 consensus_queue.cc:1048] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:11.010464 18875 raft_consensus.cc:1275] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.012392 20018 consensus_queue.cc:1048] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:11.025995 19028 raft_consensus.cc:1275] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.028198 20018 consensus_queue.cc:1048] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:11.067605 18875 raft_consensus.cc:1275] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.070143 20018 consensus_queue.cc:1048] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:11.092617 19028 raft_consensus.cc:1275] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:11.094321 19997 consensus_queue.cc:1048] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:11.325428  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:11.386981  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:11.438874 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:11.439620 19028 raft_consensus.cc:606] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:11.442682 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:11.443399 19028 raft_consensus.cc:606] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:11.447873 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:11.448532 19028 raft_consensus.cc:606] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:11.451442 18880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1"
dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:22:11.451972 18880 raft_consensus.cc:606] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:11.454706 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf"
dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
mode: GRACEFUL
new_leader_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:33690
I20260709 13:22:11.455333 19028 raft_consensus.cc:606] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Received request to transfer leadership to TS 878e795a8488495f978538952e7b2324
I20260709 13:22:11.456730  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:22:11.458770  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:11.609948 19986 raft_consensus.cc:993] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.611176 20001 raft_consensus.cc:1081] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.613065 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:11.613708 19848 raft_consensus.cc:493] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.614054 19848 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.620887 19848 raft_consensus.cc:515] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.623982 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:11.624747 18947 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.625319 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ad65f24c73445fca035eefe94eb8d15" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:11.626142 19028 raft_consensus.cc:3055] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.626178 19848 leader_election.cc:290] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:11.626756 19028 raft_consensus.cc:740] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:11.628012 19028 consensus_queue.cc:260] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.630335 19028 raft_consensus.cc:3060] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.634193 18947 raft_consensus.cc:2468] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.635908 19783 leader_election.cc:304] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:11.636986 19998 raft_consensus.cc:2804] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.637081 19986 raft_consensus.cc:993] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.637696 19998 raft_consensus.cc:697] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:11.637807 19986 raft_consensus.cc:1081] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.638916 19998 consensus_queue.cc:237] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.640169 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:11.640936 19848 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.641054 19028 raft_consensus.cc:2468] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.642112 19848 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.650485 19848 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.652661 19848 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:11.652889 18701 catalog_manager.cc:5697] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:11.653494 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:11.654480 19028 raft_consensus.cc:3055] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:11.654942 19028 raft_consensus.cc:740] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:11.654521 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b993ab3a9c34cada69f06e7bfc4f7cf" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:11.655844 18875 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.656062 19028 consensus_queue.cc:260] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.657572 19028 raft_consensus.cc:3060] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.664924 18875 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:11.665863 19028 raft_consensus.cc:2468] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:11.666762 19780 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:22:11.667908 19998 raft_consensus.cc:2804] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:11.668522 19998 raft_consensus.cc:697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:11.669590 19998 consensus_queue.cc:237] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.686873 18701 catalog_manager.cc:5697] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.729097 20061 raft_consensus.cc:993] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.729724 20001 raft_consensus.cc:1081] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.731678 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:11.732373 19848 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.732673 19848 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.741042 19848 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.743300 19848 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235)
I20260709 13:22:11.744127 18947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de62ff9486f7483e8f0e0b93dcca24bb"
I20260709 13:22:11.744922 18947 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.745030 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb1cf5992cd4659b7ab7f2ff1307fd6" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:11.745786 19028 raft_consensus.cc:3055] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.746096 19028 raft_consensus.cc:740] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:11.747047 19028 consensus_queue.cc:260] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.748617 19028 raft_consensus.cc:3060] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.754034 18947 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.758028 19028 raft_consensus.cc:2468] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.763463 19783 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb; no voters: 
I20260709 13:22:11.764503 19998 raft_consensus.cc:2804] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.765129 19998 raft_consensus.cc:697] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:11.766098 19998 consensus_queue.cc:237] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:11.777850 18701 catalog_manager.cc:5697] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:11.788159 20001 raft_consensus.cc:993] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.788774 20001 raft_consensus.cc:1081] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.792969 19845 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36104
I20260709 13:22:11.793696 19845 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.794106 19845 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.804160 19845 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.807266 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:11.807682 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5977247dfbeb4ce29c4baf035adc2321" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:11.807967 19026 raft_consensus.cc:3055] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:11.808386 19026 raft_consensus.cc:740] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3e2fa7cf3974c27a19386f362688d29, State: Running, Role: LEADER
I20260709 13:22:11.808396 18875 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.808581 19845 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:11.809365 19026 consensus_queue.cc:260] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.810763 19026 raft_consensus.cc:3060] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:11.816407 18875 raft_consensus.cc:2468] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.818145 19780 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324; no voters: 
I20260709 13:22:11.819197 19998 raft_consensus.cc:2804] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:11.819726 19998 raft_consensus.cc:697] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:11.820685 19998 consensus_queue.cc:237] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.824664 20002 raft_consensus.cc:993] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: : Instructing follower 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.825281 20002 raft_consensus.cc:1081] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Signalling peer 878e795a8488495f978538952e7b2324 to start an election
I20260709 13:22:11.827095 19848 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1"
dest_uuid: "878e795a8488495f978538952e7b2324"
 from {username='slave'} at 127.0.0.1:36094
I20260709 13:22:11.827725 19848 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:11.828099 19848 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.836912 19848 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.840713 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:11.841293 18875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c3bd79ae04b4dfe8ebf5c82eb93cac1" candidate_uuid: "878e795a8488495f978538952e7b2324" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "039c551e55fb43cdb5c3880b7cb91a0b"
I20260709 13:22:11.840957 18701 catalog_manager.cc:5697] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 1 to 2, leader changed from f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:11.842018 18875 raft_consensus.cc:3055] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:11.842303 19848 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:11.842603 18875 raft_consensus.cc:740] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 039c551e55fb43cdb5c3880b7cb91a0b, State: Running, Role: LEADER
I20260709 13:22:11.841478 19028 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.843816 18875 consensus_queue.cc:260] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.845371 18875 raft_consensus.cc:3060] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:11.841806 19026 raft_consensus.cc:2468] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 2.
I20260709 13:22:11.855006 19028 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:11.856884 19781 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 878e795a8488495f978538952e7b2324, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:11.857194 18875 raft_consensus.cc:2468] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 878e795a8488495f978538952e7b2324 in term 3.
I20260709 13:22:11.857860 19998 raft_consensus.cc:2804] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:11.858604 19998 raft_consensus.cc:697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Becoming Leader. State: Replica: 878e795a8488495f978538952e7b2324, State: Running, Role: LEADER
I20260709 13:22:11.860598 19998 consensus_queue.cc:237] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:11.875296 18701 catalog_manager.cc:5697] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 reported cstate change: term changed from 2 to 3, leader changed from 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66) to 878e795a8488495f978538952e7b2324 (127.8.183.65). New cstate: current_term: 3 leader_uuid: "878e795a8488495f978538952e7b2324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:12.072625 19028 raft_consensus.cc:1275] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:12.074442 19998 consensus_queue.cc:1048] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:12.101222 19026 raft_consensus.cc:1275] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.103385 20018 consensus_queue.cc:1048] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.107965 18952 raft_consensus.cc:1275] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.110327 20018 consensus_queue.cc:1048] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.115682 18880 raft_consensus.cc:1275] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:12.117213 19997 consensus_queue.cc:1048] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:12.301720 18947 raft_consensus.cc:1275] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.306034 18875 raft_consensus.cc:1275] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:12.316236 20013 consensus_queue.cc:1048] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:12.318750 19998 consensus_queue.cc:1048] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.375713 19026 raft_consensus.cc:1275] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:12.384121 19026 raft_consensus.cc:1275] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.400658 19998 consensus_queue.cc:1048] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:12.403043 19998 consensus_queue.cc:1048] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.425107 19026 raft_consensus.cc:1275] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.429708 18875 raft_consensus.cc:1275] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Refusing update from remote peer 878e795a8488495f978538952e7b2324: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:12.437412 19998 consensus_queue.cc:1048] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.457113 20079 consensus_queue.cc:1048] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:12.460026  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:12.550467  8925 tablet_server.cc:179] TabletServer@127.8.183.65:40287 shutting down...
W20260709 13:22:12.560992 18845 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 20 similar messages]
W20260709 13:22:12.568536 18845 consensus_peers.cc:597] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.602653  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:12.604285  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.605446  8925 raft_consensus.cc:2243] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.606560  8925 raft_consensus.cc:2272] T e83e89f3973f4275a44d76a152e43126 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.610954  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.612052  8925 raft_consensus.cc:2243] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.612658  8925 raft_consensus.cc:2272] T 7175a5fa52a94e2d8504a19eb7aa1421 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.617002  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.618122  8925 raft_consensus.cc:2243] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.618782  8925 raft_consensus.cc:2272] T d350b38546104d82881fdcd2f9c251d3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.618695 18993 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.620844 18845 consensus_peers.cc:597] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.623016  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.624289  8925 raft_consensus.cc:2243] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.625510  8925 raft_consensus.cc:2272] T 4ad65f24c73445fca035eefe94eb8d15 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.629298 18993 consensus_peers.cc:597] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.630620  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.631774  8925 raft_consensus.cc:2243] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.632531  8925 raft_consensus.cc:2272] T 04089f925f3e44ab917020605bd8a75e P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.635848  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.636759  8925 raft_consensus.cc:2243] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.637257  8925 raft_consensus.cc:2272] T dd6bf8d58b494607937dbeb92cdb2015 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.640115  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.641149  8925 raft_consensus.cc:2243] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.642477  8925 raft_consensus.cc:2272] T e3bb98bec02448d6990ba27836e9e7a9 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.645582  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.646351  8925 raft_consensus.cc:2243] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.647064  8925 raft_consensus.cc:2272] T e3e819f8fc7440af86dd0b3190f1edf3 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.650275  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.651269  8925 raft_consensus.cc:2243] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.652415  8925 raft_consensus.cc:2272] T 232db8a92f07473e91159b6e4182f3ef P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.655572  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.656534  8925 raft_consensus.cc:2243] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.657164  8925 raft_consensus.cc:2272] T 0ed73ebff034411ab7b405ac83d2e34e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.660401  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.661269  8925 raft_consensus.cc:2243] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.661892  8925 raft_consensus.cc:2272] T d850a82d849e47ef9576b6d0b73700c0 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.665372  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.666465  8925 raft_consensus.cc:2243] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.667644  8925 raft_consensus.cc:2272] T afdc899ecc694b09991abe77a2195448 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.668007 18845 consensus_peers.cc:597] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.670286  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.671097  8925 raft_consensus.cc:2243] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.671643  8925 raft_consensus.cc:2272] T d217d9f5218640ed95be5641ff1fb098 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.674098  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.674914  8925 raft_consensus.cc:2243] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.675961  8925 raft_consensus.cc:2272] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.679673  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.680518  8925 raft_consensus.cc:2243] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.681777  8925 raft_consensus.cc:2272] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.685271  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.686034  8925 raft_consensus.cc:2243] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.687161  8925 raft_consensus.cc:2272] T 577d52a3cf7346349e47f24b3e1286e1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.690630  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.691767  8925 raft_consensus.cc:2243] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.692344  8925 raft_consensus.cc:2272] T c67b792f4ba14e55abb2a3a92d4d0b2b P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.694908 18993 consensus_peers.cc:597] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.695858  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.697072  8925 raft_consensus.cc:2243] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.698144  8925 raft_consensus.cc:2272] T b7cb8b0da1be426bad1c6d04041a06db P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.701936  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.702916  8925 raft_consensus.cc:2243] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.704105  8925 raft_consensus.cc:2272] T b4c3f02bc00e460dba727510708feee6 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.707729  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.708736  8925 raft_consensus.cc:2243] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.709523  8925 raft_consensus.cc:2272] T f302bf297c554158b7f4ebcd5c7760bc P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.709486 18993 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.714244  8925 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:12.714165 18993 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.715526  8925 raft_consensus.cc:2243] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:12.716457 18919 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.716764  8925 raft_consensus.cc:2272] T 6fde9dbeb78f450581397317487e31f0 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.716990 18993 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.720506  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.721632  8925 raft_consensus.cc:2243] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.722294  8925 raft_consensus.cc:2272] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.725780  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.726730  8925 raft_consensus.cc:2243] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.727480  8925 raft_consensus.cc:2272] T e076455b2f73491282b009a90112d07d P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.729918  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.730901  8925 raft_consensus.cc:2243] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.731680  8925 raft_consensus.cc:2272] T a01b903fffc64c949e13c3c74abe8489 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.735148  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.736171  8925 raft_consensus.cc:2243] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:12.736295 18845 consensus_peers.cc:597] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.736809  8925 raft_consensus.cc:2272] T 35fbf5e9b0e446b39953dd28e5aa71be P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.740343  8925 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:12.740412 18993 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.741494  8925 raft_consensus.cc:2243] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.742770  8925 raft_consensus.cc:2272] T 6870bd6f453d45a19a939d17ee0f92ad P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.746122  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.747071  8925 raft_consensus.cc:2243] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.747713  8925 raft_consensus.cc:2272] T 4f2be91edb9e4188a248db3e309f0e44 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.750514  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.751525  8925 raft_consensus.cc:2243] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.752990  8925 raft_consensus.cc:2272] T 4832e091fcef44ab877283bf9a254752 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.753841 18845 consensus_peers.cc:597] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.756598  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.757897  8925 raft_consensus.cc:2243] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.758630  8925 raft_consensus.cc:2272] T da6dc24978cc43a79db989f59bfed38a P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.760519 18845 consensus_peers.cc:597] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.762481  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.765165  8925 raft_consensus.cc:2243] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.765959  8925 raft_consensus.cc:2272] T b57e1d1c76904a8492670c30af464d8d P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.769483  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.770502  8925 raft_consensus.cc:2243] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.771193  8925 raft_consensus.cc:2272] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.773957  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.774681  8925 raft_consensus.cc:2243] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.775682  8925 raft_consensus.cc:2272] T 5977247dfbeb4ce29c4baf035adc2321 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.779282  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.780082  8925 raft_consensus.cc:2243] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.780755  8925 raft_consensus.cc:2272] T ab7b798c63c94bce8e69fefd480e295b P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.782882 18845 consensus_peers.cc:597] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.783866  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.784560  8925 raft_consensus.cc:2243] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.785552  8925 raft_consensus.cc:2272] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 878e795a8488495f978538952e7b2324 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.788573  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.789539  8925 raft_consensus.cc:2243] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:12.789593 18919 consensus_peers.cc:597] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.790434  8925 raft_consensus.cc:2272] T 13175817e62e49c38eafca15e3877bc6 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.793866  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.794689  8925 raft_consensus.cc:2243] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.795372  8925 raft_consensus.cc:2272] T a3b2d906586545bda1c94132ed20782a P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.796639 18919 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.798324  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.799046  8925 raft_consensus.cc:2243] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.799634  8925 raft_consensus.cc:2272] T fcb4a0d3f9924af3881fae78b771264e P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.802281  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.803202  8925 raft_consensus.cc:2243] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.803722  8925 raft_consensus.cc:2272] T 03fbc43806c04306990613e14c199ff1 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.807269  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.808220  8925 raft_consensus.cc:2243] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.808773  8925 raft_consensus.cc:2272] T 45bd3ea063de41839fe0187978e00474 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.812052  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.812862  8925 raft_consensus.cc:2243] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.813524  8925 raft_consensus.cc:2272] T bcc4d045faa6417e894366a8fb7b9f52 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.816727  8925 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:12.817158 18919 consensus_peers.cc:597] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.817495  8925 raft_consensus.cc:2243] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.818262  8925 raft_consensus.cc:2272] T 706f5712f7ac487b97054415f672886f P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.820250 18919 consensus_peers.cc:597] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.821655  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.822742  8925 raft_consensus.cc:2243] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.823458  8925 raft_consensus.cc:2272] T 09abb2d24f6748bf938539c210dc01fc P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.826807  8925 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:12.826934 18993 consensus_peers.cc:597] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.827641  8925 raft_consensus.cc:2243] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.828316  8925 raft_consensus.cc:2272] T 0f439429f2a945c5b906f0a9e2d57e41 P 878e795a8488495f978538952e7b2324 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.830894  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.831660  8925 raft_consensus.cc:2243] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.832206  8925 raft_consensus.cc:2272] T 8ac0dee3476f4361a8531c64161fee77 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.834954  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:12.835724  8925 raft_consensus.cc:2243] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:12.836347  8925 raft_consensus.cc:2272] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 878e795a8488495f978538952e7b2324 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:12.837467 18919 consensus_peers.cc:597] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.860548 18919 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.862948 18845 consensus_peers.cc:597] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.879287 18993 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.893102 18993 consensus_peers.cc:597] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.895329 18845 consensus_peers.cc:597] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:12.924234 18919 consensus_peers.cc:597] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:12.934286  8925 tablet_server.cc:196] TabletServer@127.8.183.65:40287 shutdown complete.
W20260709 13:22:12.990651 18993 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.044957 18919 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.051885 18919 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.064702  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
W20260709 13:22:13.083285 18991 consensus_peers.cc:597] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.107016 18917 consensus_peers.cc:597] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.123821 18919 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Couldn't send request to peer 878e795a8488495f978538952e7b2324. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.140111 18991 consensus_peers.cc:597] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.140530 18917 consensus_peers.cc:597] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.149838  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:13.150942  8925 tablet_replica.cc:333] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.151952  8925 raft_consensus.cc:2243] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.152626  8925 raft_consensus.cc:2272] T 9b993ab3a9c34cada69f06e7bfc4f7cf P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.156852  8925 tablet_replica.cc:333] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.157826  8925 raft_consensus.cc:2243] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.158496  8925 raft_consensus.cc:2272] T fcb4a0d3f9924af3881fae78b771264e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.161315  8925 tablet_replica.cc:333] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.162194  8925 raft_consensus.cc:2243] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.162745  8925 raft_consensus.cc:2272] T e83e89f3973f4275a44d76a152e43126 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.164937  8925 tablet_replica.cc:333] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.165684  8925 raft_consensus.cc:2243] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.166236  8925 raft_consensus.cc:2272] T 5977247dfbeb4ce29c4baf035adc2321 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.169198  8925 tablet_replica.cc:333] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.170317  8925 raft_consensus.cc:2243] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.171000  8925 raft_consensus.cc:2272] T 34f9127f17e646d7a2d0f25d4a4b0d4f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.174835  8925 tablet_replica.cc:333] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.176031  8925 raft_consensus.cc:2243] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.177040  8925 raft_consensus.cc:2272] T 0240263315db416dbece9ba6b467818d P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.181094  8925 tablet_replica.cc:333] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.182261  8925 raft_consensus.cc:2243] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.183037  8925 raft_consensus.cc:2272] T c4490b378f034620bcb7df67470bdab9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.187402  8925 tablet_replica.cc:333] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.188448  8925 raft_consensus.cc:2243] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:13.188875 18917 consensus_peers.cc:597] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.189160  8925 raft_consensus.cc:2272] T f302bf297c554158b7f4ebcd5c7760bc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.193259  8925 tablet_replica.cc:333] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.194420  8925 raft_consensus.cc:2243] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.195215  8925 raft_consensus.cc:2272] T 7175a5fa52a94e2d8504a19eb7aa1421 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.199326  8925 tablet_replica.cc:333] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.200402  8925 raft_consensus.cc:2243] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.201125  8925 raft_consensus.cc:2272] T 6808727e98af41939b051f079357a49f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.205710  8925 tablet_replica.cc:333] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.207005  8925 raft_consensus.cc:2243] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.208475  8925 raft_consensus.cc:2272] T b57e1d1c76904a8492670c30af464d8d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.217679  8925 tablet_replica.cc:333] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.218513  8925 raft_consensus.cc:2243] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.219064  8925 raft_consensus.cc:2272] T 13175817e62e49c38eafca15e3877bc6 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.222537  8925 tablet_replica.cc:333] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.223694  8925 raft_consensus.cc:2243] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.224323  8925 raft_consensus.cc:2272] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.228048  8925 tablet_replica.cc:333] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.228906  8925 raft_consensus.cc:2243] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.229631  8925 raft_consensus.cc:2272] T da6dc24978cc43a79db989f59bfed38a P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.233382  8925 tablet_replica.cc:333] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.234400  8925 raft_consensus.cc:2243] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.235764  8925 raft_consensus.cc:2272] T 211a9a1cbd3f4dca903a20b4ecb32209 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.239529  8925 tablet_replica.cc:333] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.240643  8925 raft_consensus.cc:2243] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.241346  8925 raft_consensus.cc:2272] T f54c60d833534725beae61cec6e75c4b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.245150  8925 tablet_replica.cc:333] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.246194  8925 raft_consensus.cc:2243] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.246891  8925 raft_consensus.cc:2272] T e3bb98bec02448d6990ba27836e9e7a9 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.250429  8925 tablet_replica.cc:333] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.251389  8925 raft_consensus.cc:2243] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.252101  8925 raft_consensus.cc:2272] T 706f5712f7ac487b97054415f672886f P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.255478  8925 tablet_replica.cc:333] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.256680  8925 raft_consensus.cc:2243] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.257369  8925 raft_consensus.cc:2272] T 45bd3ea063de41839fe0187978e00474 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.261181  8925 tablet_replica.cc:333] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.262341  8925 raft_consensus.cc:2243] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.263758  8925 raft_consensus.cc:2272] T bcc4d045faa6417e894366a8fb7b9f52 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.265769 18917 consensus_peers.cc:597] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.267046  8925 tablet_replica.cc:333] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.268236  8925 raft_consensus.cc:2243] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.268889  8925 raft_consensus.cc:2272] T 6870bd6f453d45a19a939d17ee0f92ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.270829 18917 consensus_peers.cc:597] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.272466  8925 tablet_replica.cc:333] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.273550  8925 raft_consensus.cc:2243] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.274263  8925 raft_consensus.cc:2272] T 04089f925f3e44ab917020605bd8a75e P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.277899  8925 tablet_replica.cc:333] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
W20260709 13:22:13.278467 18991 consensus_peers.cc:597] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.279009  8925 raft_consensus.cc:2243] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.280529  8925 raft_consensus.cc:2272] T 14fe966c22b14da9985eba8dd9ed1f2c P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.284015  8925 tablet_replica.cc:333] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.284965  8925 raft_consensus.cc:2243] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.285643  8925 raft_consensus.cc:2272] T 577d52a3cf7346349e47f24b3e1286e1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.287356 18917 consensus_peers.cc:597] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.288686  8925 tablet_replica.cc:333] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.291447  8925 raft_consensus.cc:2243] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:22:13.292008 18917 consensus_peers.cc:597] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.292902  8925 raft_consensus.cc:2272] T c67b792f4ba14e55abb2a3a92d4d0b2b P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.297163  8925 tablet_replica.cc:333] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.297993  8925 raft_consensus.cc:2243] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.298599  8925 raft_consensus.cc:2272] T a01b903fffc64c949e13c3c74abe8489 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.302045  8925 tablet_replica.cc:333] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.302870  8925 raft_consensus.cc:2243] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.304478  8925 raft_consensus.cc:2272] T e076455b2f73491282b009a90112d07d P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.308372  8925 tablet_replica.cc:333] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.309491  8925 raft_consensus.cc:2243] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.310189  8925 raft_consensus.cc:2272] T 232db8a92f07473e91159b6e4182f3ef P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.310989 18991 consensus_peers.cc:597] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.313889  8925 tablet_replica.cc:333] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.315007  8925 raft_consensus.cc:2243] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.316159  8925 raft_consensus.cc:2272] T bdd0d96cdc814e8fa997150f58dda773 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.319653  8925 tablet_replica.cc:333] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.320660  8925 raft_consensus.cc:2243] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.322023  8925 raft_consensus.cc:2272] T c692689ede48466a882155f06b0763ad P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.325369  8925 tablet_replica.cc:333] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.326319  8925 raft_consensus.cc:2243] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.327689  8925 raft_consensus.cc:2272] T 03fbc43806c04306990613e14c199ff1 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.331380  8925 tablet_replica.cc:333] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.332329  8925 raft_consensus.cc:2243] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.332978  8925 raft_consensus.cc:2272] T a4a64164017441fd8ce325d46dcbbd07 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.336624  8925 tablet_replica.cc:333] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.337720  8925 raft_consensus.cc:2243] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.338486  8925 raft_consensus.cc:2272] T a3b2d906586545bda1c94132ed20782a P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.341207 18991 consensus_peers.cc:597] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.342206  8925 tablet_replica.cc:333] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.343250  8925 raft_consensus.cc:2243] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.344365  8925 raft_consensus.cc:2272] T 3e3966983799476b9516092542b22438 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.347741  8925 tablet_replica.cc:333] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.348641  8925 raft_consensus.cc:2243] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.349328  8925 raft_consensus.cc:2272] T afdc899ecc694b09991abe77a2195448 P 039c551e55fb43cdb5c3880b7cb91a0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.352432  8925 tablet_replica.cc:333] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.353204  8925 raft_consensus.cc:2243] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.354327  8925 raft_consensus.cc:2272] T d217d9f5218640ed95be5641ff1fb098 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.357694 18917 consensus_peers.cc:597] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.357824  8925 tablet_replica.cc:333] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.359086  8925 raft_consensus.cc:2243] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.359741  8925 raft_consensus.cc:2272] T fee6b2223720435195a84c3570506ea5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.363014  8925 tablet_replica.cc:333] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.364029  8925 raft_consensus.cc:2243] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.364763  8925 raft_consensus.cc:2272] T d850a82d849e47ef9576b6d0b73700c0 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.368382  8925 tablet_replica.cc:333] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.369283  8925 raft_consensus.cc:2243] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.369961  8925 raft_consensus.cc:2272] T 0ed73ebff034411ab7b405ac83d2e34e P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.373342  8925 tablet_replica.cc:333] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.374195  8925 raft_consensus.cc:2243] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.374862  8925 raft_consensus.cc:2272] T d3f20d931bf640d09b913bc7a1e8e3a8 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.377998  8925 tablet_replica.cc:333] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.378630  8925 raft_consensus.cc:2243] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.379702  8925 raft_consensus.cc:2272] T 0346a3d5c16b4965a7492826c0da6872 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.382934  8925 tablet_replica.cc:333] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.383697  8925 raft_consensus.cc:2243] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.384851  8925 raft_consensus.cc:2272] T 0f439429f2a945c5b906f0a9e2d57e41 P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.387939  8925 tablet_replica.cc:333] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.388592  8925 raft_consensus.cc:2243] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.389578  8925 raft_consensus.cc:2272] T ccb9f9f9dbb04b459b94b15ddb6864d5 P 039c551e55fb43cdb5c3880b7cb91a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.391983  8925 tablet_replica.cc:333] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b: stopping tablet replica
I20260709 13:22:13.392553  8925 raft_consensus.cc:2243] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.393496  8925 raft_consensus.cc:2272] T 09abb2d24f6748bf938539c210dc01fc P 039c551e55fb43cdb5c3880b7cb91a0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.452312 18991 consensus_peers.cc:597] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.494078 18991 consensus_peers.cc:597] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.495585 18991 consensus_peers.cc:597] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.495538 18917 consensus_peers.cc:597] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.503347 18917 consensus_peers.cc:597] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:13.529788 18991 consensus_peers.cc:597] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.533928  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
W20260709 13:22:13.537962 18991 consensus_peers.cc:597] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 -> Peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Couldn't send request to peer 039c551e55fb43cdb5c3880b7cb91a0b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.648734  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
W20260709 13:22:13.704383 18994 consensus_peers.cc:597] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.720431  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:13.721697  8925 tablet_replica.cc:333] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.722816  8925 raft_consensus.cc:2243] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.723583  8925 raft_consensus.cc:2272] T dd6bf8d58b494607937dbeb92cdb2015 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.725494 18994 consensus_peers.cc:597] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.727242  8925 tablet_replica.cc:333] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.728076  8925 raft_consensus.cc:2243] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.729017  8925 raft_consensus.cc:2272] T fcb4a0d3f9924af3881fae78b771264e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.731474  8925 tablet_replica.cc:333] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.732193  8925 raft_consensus.cc:2243] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.732831  8925 raft_consensus.cc:2272] T 35fbf5e9b0e446b39953dd28e5aa71be P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.735838  8925 tablet_replica.cc:333] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.736819  8925 raft_consensus.cc:2243] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.737489  8925 raft_consensus.cc:2272] T e83e89f3973f4275a44d76a152e43126 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.740424  8925 tablet_replica.cc:333] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.741115  8925 raft_consensus.cc:2243] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.741976  8925 raft_consensus.cc:2272] T 34f9127f17e646d7a2d0f25d4a4b0d4f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.744429  8925 tablet_replica.cc:333] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.745253  8925 raft_consensus.cc:2243] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.745833  8925 raft_consensus.cc:2272] T 4ad65f24c73445fca035eefe94eb8d15 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.748775  8925 tablet_replica.cc:333] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.749809  8925 raft_consensus.cc:2243] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.751084  8925 raft_consensus.cc:2272] T 0240263315db416dbece9ba6b467818d P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.753867  8925 tablet_replica.cc:333] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.754630  8925 raft_consensus.cc:2243] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.755304  8925 raft_consensus.cc:2272] T c4490b378f034620bcb7df67470bdab9 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.755517 20090 raft_consensus.cc:493] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.756145 20090 raft_consensus.cc:515] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } } peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } }
I20260709 13:22:13.759222  8925 tablet_replica.cc:333] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.759037 20090 leader_election.cc:290] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:13.760484  8925 raft_consensus.cc:2243] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.761271  8925 raft_consensus.cc:2272] T b4c3f02bc00e460dba727510708feee6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.764711  8925 tablet_replica.cc:333] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
W20260709 13:22:13.765017 18917 leader_election.cc:336] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
I20260709 13:22:13.765708  8925 raft_consensus.cc:2243] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:13.765812 18919 leader_election.cc:336] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.766435 18919 leader_election.cc:304] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:13.766600  8925 raft_consensus.cc:2272] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.767323 20090 raft_consensus.cc:2749] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:13.769888  8925 tablet_replica.cc:333] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.770763  8925 raft_consensus.cc:2243] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.771415  8925 raft_consensus.cc:2272] T 211a9a1cbd3f4dca903a20b4ecb32209 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.773926  8925 tablet_replica.cc:333] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.774670  8925 raft_consensus.cc:2243] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.775744  8925 raft_consensus.cc:2272] T 8ac0dee3476f4361a8531c64161fee77 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.778141  8925 tablet_replica.cc:333] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.778831  8925 raft_consensus.cc:2243] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.779366  8925 raft_consensus.cc:2272] T 232db8a92f07473e91159b6e4182f3ef P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.781793  8925 tablet_replica.cc:333] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.782474  8925 raft_consensus.cc:2243] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.782985  8925 raft_consensus.cc:2272] T bdd0d96cdc814e8fa997150f58dda773 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.785406  8925 tablet_replica.cc:333] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.786051  8925 raft_consensus.cc:2243] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.786538  8925 raft_consensus.cc:2272] T c692689ede48466a882155f06b0763ad P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.789085  8925 tablet_replica.cc:333] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.789800  8925 raft_consensus.cc:2243] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.790759  8925 raft_consensus.cc:2272] T 706f5712f7ac487b97054415f672886f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.793721  8925 tablet_replica.cc:333] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.794469  8925 raft_consensus.cc:2243] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.795522  8925 raft_consensus.cc:2272] T 45bd3ea063de41839fe0187978e00474 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.797717  8925 tablet_replica.cc:333] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.798452  8925 raft_consensus.cc:2243] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.799068  8925 raft_consensus.cc:2272] T bcc4d045faa6417e894366a8fb7b9f52 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.801748  8925 tablet_replica.cc:333] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.802379  8925 raft_consensus.cc:2243] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.803299  8925 raft_consensus.cc:2272] T 14fe966c22b14da9985eba8dd9ed1f2c P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.806496  8925 tablet_replica.cc:333] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.807283  8925 raft_consensus.cc:2243] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.808187  8925 raft_consensus.cc:2272] T d350b38546104d82881fdcd2f9c251d3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.810539  8925 tablet_replica.cc:333] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.811329  8925 raft_consensus.cc:2243] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.811898  8925 raft_consensus.cc:2272] T 09abb2d24f6748bf938539c210dc01fc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.814070  8925 tablet_replica.cc:333] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.814821  8925 raft_consensus.cc:2243] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.815379  8925 raft_consensus.cc:2272] T 6fde9dbeb78f450581397317487e31f0 P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.817641  8925 tablet_replica.cc:333] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.818327  8925 raft_consensus.cc:2243] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.819350  8925 raft_consensus.cc:2272] T f302bf297c554158b7f4ebcd5c7760bc P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.821678  8925 tablet_replica.cc:333] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.822407  8925 raft_consensus.cc:2243] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.822937  8925 raft_consensus.cc:2272] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.826128  8925 tablet_replica.cc:333] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
W20260709 13:22:13.826403 18994 consensus_peers.cc:597] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.826959  8925 raft_consensus.cc:2243] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.827888  8925 raft_consensus.cc:2272] T 6870bd6f453d45a19a939d17ee0f92ad P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.830722  8925 tablet_replica.cc:333] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.831624  8925 raft_consensus.cc:2243] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.832175  8925 raft_consensus.cc:2272] T 4f2be91edb9e4188a248db3e309f0e44 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.834697  8925 tablet_replica.cc:333] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.835577  8925 raft_consensus.cc:2243] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.836541  8925 raft_consensus.cc:2272] T 4832e091fcef44ab877283bf9a254752 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.838987  8925 tablet_replica.cc:333] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.839777  8925 raft_consensus.cc:2243] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.840451  8925 raft_consensus.cc:2272] T f54c60d833534725beae61cec6e75c4b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.844607  8925 tablet_replica.cc:333] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.845607  8925 raft_consensus.cc:2243] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.846740  8925 raft_consensus.cc:2272] T e3e819f8fc7440af86dd0b3190f1edf3 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.849932  8925 tablet_replica.cc:333] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.850059 20054 raft_consensus.cc:493] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.850679  8925 raft_consensus.cc:2243] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.850747 20054 raft_consensus.cc:515] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.851404  8925 raft_consensus.cc:2272] T d217d9f5218640ed95be5641ff1fb098 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.852389 20093 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.853011 20093 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.853480 20054 leader_election.cc:290] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:22:13.854787  8925 tablet_replica.cc:333] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.855402 20093 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:22:13.855808  8925 raft_consensus.cc:2243] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.857172  8925 raft_consensus.cc:2272] T fee6b2223720435195a84c3570506ea5 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.860586  8925 tablet_replica.cc:333] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.861475  8925 raft_consensus.cc:2243] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:22:13.861822 18993 leader_election.cc:336] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:22:13.862445 18994 leader_election.cc:336] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:13.862500 18993 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.862797  8925 raft_consensus.cc:2272] T 0ed73ebff034411ab7b405ac83d2e34e P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.862941 20090 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.863166 18994 leader_election.cc:304] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:22:13.863978 20090 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.864364 20093 raft_consensus.cc:2749] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:22:13.865132 18994 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:13.865747 18994 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:22:13.866534 20093 raft_consensus.cc:2749] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:13.866920  8925 tablet_replica.cc:333] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.867740 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29" is_pre_election: true
I20260709 13:22:13.867986  8925 raft_consensus.cc:2243] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.868484 19028 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 3.
I20260709 13:22:13.869557  8925 raft_consensus.cc:2272] T d3f20d931bf640d09b913bc7a1e8e3a8 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.869777 20090 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:22:13.870121 18918 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 
I20260709 13:22:13.871411 20090 raft_consensus.cc:2804] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:22:13.871841 20090 raft_consensus.cc:493] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Starting leader election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.872203 20090 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:13.873178  8925 tablet_replica.cc:333] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.874058  8925 raft_consensus.cc:2243] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:13.873993 18919 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.874953  8925 raft_consensus.cc:2272] T 0346a3d5c16b4965a7492826c0da6872 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.877610  8925 tablet_replica.cc:333] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.879647 20090 raft_consensus.cc:515] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.880769  8925 raft_consensus.cc:2243] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.880947 20093 raft_consensus.cc:493] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.881645 20093 raft_consensus.cc:515] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.882056 20090 leader_election.cc:290] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 election: Requested vote from peers f3e2fa7cf3974c27a19386f362688d29 (127.8.183.68:46235), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:22:13.882932 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cb8b0da1be426bad1c6d04041a06db" candidate_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3e2fa7cf3974c27a19386f362688d29"
I20260709 13:22:13.883534 19028 raft_consensus.cc:3060] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:13.883783  8925 raft_consensus.cc:2272] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.886077 20093 leader_election.cc:290] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
I20260709 13:22:13.887389  8925 tablet_replica.cc:333] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
W20260709 13:22:13.888191 18994 leader_election.cc:336] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:13.888367  8925 raft_consensus.cc:2243] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.889205  8925 raft_consensus.cc:2272] T 577d52a3cf7346349e47f24b3e1286e1 P de62ff9486f7483e8f0e0b93dcca24bb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.892477  8925 tablet_replica.cc:333] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.893451  8925 raft_consensus.cc:2243] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.894133  8925 raft_consensus.cc:2272] T c67b792f4ba14e55abb2a3a92d4d0b2b P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.897877  8925 tablet_replica.cc:333] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
W20260709 13:22:13.897708 18919 leader_election.cc:336] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.899071  8925 raft_consensus.cc:2243] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.900806  8925 raft_consensus.cc:2272] T 13175817e62e49c38eafca15e3877bc6 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.905558  8925 tablet_replica.cc:333] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.906587  8925 raft_consensus.cc:2243] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:13.908125  8925 raft_consensus.cc:2272] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.911195 18994 consensus_peers.cc:597] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:13.912837  8925 tablet_replica.cc:333] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.914340  8925 raft_consensus.cc:2243] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.915346  8925 raft_consensus.cc:2272] T 6808727e98af41939b051f079357a49f P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.920589  8925 tablet_replica.cc:333] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.921983  8925 raft_consensus.cc:2243] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.922955  8925 raft_consensus.cc:2272] T b57e1d1c76904a8492670c30af464d8d P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.920727 19028 raft_consensus.cc:2468] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de62ff9486f7483e8f0e0b93dcca24bb in term 4.
I20260709 13:22:13.928413  8925 tablet_replica.cc:333] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.929464  8925 raft_consensus.cc:2243] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.930152  8925 raft_consensus.cc:2272] T ab7b798c63c94bce8e69fefd480e295b P de62ff9486f7483e8f0e0b93dcca24bb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.930644 18918 leader_election.cc:304] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de62ff9486f7483e8f0e0b93dcca24bb, f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324
W20260709 13:22:13.931591 18918 raft_consensus.cc:2686] T b7cb8b0da1be426bad1c6d04041a06db P de62ff9486f7483e8f0e0b93dcca24bb: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:22:13.933300 19986 raft_consensus.cc:493] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
W20260709 13:22:13.933621 18993 leader_election.cc:336] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.934268 18993 leader_election.cc:304] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:22:13.934019 19986 raft_consensus.cc:515] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:13.935254  8925 tablet_replica.cc:333] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.935379 20089 raft_consensus.cc:2749] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:13.936180  8925 raft_consensus.cc:2243] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.936932  8925 raft_consensus.cc:2272] T a4a64164017441fd8ce325d46dcbbd07 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:13.940483  8925 tablet_replica.cc:333] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb: stopping tablet replica
I20260709 13:22:13.941236  8925 raft_consensus.cc:2243] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:13.941831  8925 raft_consensus.cc:2272] T 3e3966983799476b9516092542b22438 P de62ff9486f7483e8f0e0b93dcca24bb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:13.942878 18994 leader_election.cc:336] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:13.937731 19986 leader_election.cc:290] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:22:13.943982 18993 leader_election.cc:336] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:13.944502 18993 leader_election.cc:304] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:22:13.945415 19986 raft_consensus.cc:2749] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:13.950740 19986 raft_consensus.cc:493] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:13.951462 19986 raft_consensus.cc:515] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:13.955314 19986 leader_election.cc:290] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:22:13.960835 18993 leader_election.cc:336] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:22:13.961429 18991 leader_election.cc:336] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
I20260709 13:22:13.961992 18991 leader_election.cc:304] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:13.962843 19986 raft_consensus.cc:2749] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:22:13.998373 18994 consensus_peers.cc:597] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.012271 20089 raft_consensus.cc:493] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:14.013099 20089 raft_consensus.cc:515] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
W20260709 13:22:14.016906 18994 consensus_peers.cc:597] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.018103 20089 leader_election.cc:290] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:22:14.024701 18991 leader_election.cc:336] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
W20260709 13:22:14.025476 18993 leader_election.cc:336] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:14.026041 18993 leader_election.cc:304] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:14.027065 20089 raft_consensus.cc:2749] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:22:14.034511 18994 consensus_peers.cc:597] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:37419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.042817 19986 raft_consensus.cc:493] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:14.043634 19986 raft_consensus.cc:515] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:14.046612 19986 leader_election.cc:290] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
I20260709 13:22:14.047351  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:22:14.048547 19986 raft_consensus.cc:493] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:14.049232 19986 raft_consensus.cc:515] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:14.052066 19986 leader_election.cc:290] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:22:14.054770 18991 leader_election.cc:336] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
W20260709 13:22:14.057793 18991 leader_election.cc:336] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
W20260709 13:22:14.061045 18993 leader_election.cc:336] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:14.061681 18993 leader_election.cc:304] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:14.062613 19986 raft_consensus.cc:2749] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:22:14.062848 18993 leader_election.cc:336] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:14.063520 18993 leader_election.cc:304] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:14.064325 19986 raft_consensus.cc:2749] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:22:14.083727 18994 consensus_peers.cc:597] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:37419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.084692 19986 raft_consensus.cc:493] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:14.085278 19986 raft_consensus.cc:515] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:14.089782 19986 leader_election.cc:290] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:22:14.093658 18991 leader_election.cc:336] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
W20260709 13:22:14.095706 18993 leader_election.cc:336] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:14.096246 18993 leader_election.cc:304] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:14.097008 19986 raft_consensus.cc:2749] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:22:14.169003 18994 consensus_peers.cc:597] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 -> Peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Couldn't send request to peer de62ff9486f7483e8f0e0b93dcca24bb. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.67:37419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:14.177642  8925 tablet_server.cc:179] TabletServer@127.8.183.68:0 shutting down...
I20260709 13:22:14.190762 19986 raft_consensus.cc:493] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 878e795a8488495f978538952e7b2324)
I20260709 13:22:14.191538 19986 raft_consensus.cc:515] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de62ff9486f7483e8f0e0b93dcca24bb" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 37419 } } peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } }
I20260709 13:22:14.193976 19986 leader_election.cc:290] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419), 878e795a8488495f978538952e7b2324 (127.8.183.65:40287)
W20260709 13:22:14.205018 18993 leader_election.cc:336] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
W20260709 13:22:14.205684 18994 leader_election.cc:336] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer de62ff9486f7483e8f0e0b93dcca24bb (127.8.183.67:37419): Network error: Client connection negotiation failed: client connection to 127.8.183.67:37419: connect: Connection refused (error 111)
I20260709 13:22:14.206308 18994 leader_election.cc:304] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 878e795a8488495f978538952e7b2324, de62ff9486f7483e8f0e0b93dcca24bb
I20260709 13:22:14.207408 19986 raft_consensus.cc:2749] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:14.227232 19986 raft_consensus.cc:493] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 039c551e55fb43cdb5c3880b7cb91a0b)
I20260709 13:22:14.227958 19986 raft_consensus.cc:515] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3e2fa7cf3974c27a19386f362688d29" member_type: VOTER last_known_addr { host: "127.8.183.68" port: 46235 } } peers { permanent_uuid: "878e795a8488495f978538952e7b2324" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 40287 } } peers { permanent_uuid: "039c551e55fb43cdb5c3880b7cb91a0b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 37707 } }
I20260709 13:22:14.230422 19986 leader_election.cc:290] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 878e795a8488495f978538952e7b2324 (127.8.183.65:40287), 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707)
W20260709 13:22:14.235752 18991 leader_election.cc:336] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 039c551e55fb43cdb5c3880b7cb91a0b (127.8.183.66:37707): Network error: Client connection negotiation failed: client connection to 127.8.183.66:37707: connect: Connection refused (error 111)
W20260709 13:22:14.236536 18993 leader_election.cc:336] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 878e795a8488495f978538952e7b2324 (127.8.183.65:40287): Network error: Client connection negotiation failed: client connection to 127.8.183.65:40287: connect: Connection refused (error 111)
I20260709 13:22:14.236913 18993 leader_election.cc:304] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3e2fa7cf3974c27a19386f362688d29; no voters: 039c551e55fb43cdb5c3880b7cb91a0b, 878e795a8488495f978538952e7b2324
I20260709 13:22:14.237656 19986 raft_consensus.cc:2749] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:14.245580  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:14.246948  8925 tablet_replica.cc:333] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.247803  8925 raft_consensus.cc:2243] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.248872  8925 raft_consensus.cc:2272] T dd6bf8d58b494607937dbeb92cdb2015 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.251760  8925 tablet_replica.cc:333] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.252503  8925 raft_consensus.cc:2243] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.253060  8925 raft_consensus.cc:2272] T b7cb8b0da1be426bad1c6d04041a06db P f3e2fa7cf3974c27a19386f362688d29 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.255502  8925 tablet_replica.cc:333] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.256235  8925 raft_consensus.cc:2243] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.256726  8925 raft_consensus.cc:2272] T 5977247dfbeb4ce29c4baf035adc2321 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.259234  8925 tablet_replica.cc:333] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.259838  8925 raft_consensus.cc:2243] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.260915  8925 raft_consensus.cc:2272] T 6808727e98af41939b051f079357a49f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.263310  8925 tablet_replica.cc:333] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.263985  8925 raft_consensus.cc:2243] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.264423  8925 raft_consensus.cc:2272] T 34f9127f17e646d7a2d0f25d4a4b0d4f P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.266736  8925 tablet_replica.cc:333] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.267517  8925 raft_consensus.cc:2243] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.268184  8925 raft_consensus.cc:2272] T 0240263315db416dbece9ba6b467818d P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.271222  8925 tablet_replica.cc:333] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.272223  8925 raft_consensus.cc:2243] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.273541  8925 raft_consensus.cc:2272] T c4490b378f034620bcb7df67470bdab9 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.276782  8925 tablet_replica.cc:333] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.277587  8925 raft_consensus.cc:2243] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.278128  8925 raft_consensus.cc:2272] T b4c3f02bc00e460dba727510708feee6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.280741  8925 tablet_replica.cc:333] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.281482  8925 raft_consensus.cc:2243] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.282023  8925 raft_consensus.cc:2272] T 9b993ab3a9c34cada69f06e7bfc4f7cf P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.284641  8925 tablet_replica.cc:333] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.285310  8925 raft_consensus.cc:2243] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.285791  8925 raft_consensus.cc:2272] T 2cb1cf5992cd4659b7ab7f2ff1307fd6 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.288213  8925 tablet_replica.cc:333] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.288888  8925 raft_consensus.cc:2243] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.289397  8925 raft_consensus.cc:2272] T afdc899ecc694b09991abe77a2195448 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.292356  8925 tablet_replica.cc:333] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.293336  8925 raft_consensus.cc:2243] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.294479  8925 raft_consensus.cc:2272] T 7175a5fa52a94e2d8504a19eb7aa1421 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.297734  8925 tablet_replica.cc:333] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.298763  8925 raft_consensus.cc:2243] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.299996  8925 raft_consensus.cc:2272] T da6dc24978cc43a79db989f59bfed38a P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.302951  8925 tablet_replica.cc:333] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.303676  8925 raft_consensus.cc:2243] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.304173  8925 raft_consensus.cc:2272] T 211a9a1cbd3f4dca903a20b4ecb32209 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.306401  8925 tablet_replica.cc:333] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.307157  8925 raft_consensus.cc:2243] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.307776  8925 raft_consensus.cc:2272] T 6c3bd79ae04b4dfe8ebf5c82eb93cac1 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.310458  8925 tablet_replica.cc:333] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.311616  8925 raft_consensus.cc:2243] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.312319  8925 raft_consensus.cc:2272] T 0f439429f2a945c5b906f0a9e2d57e41 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.315078  8925 tablet_replica.cc:333] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.315848  8925 raft_consensus.cc:2243] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.316395  8925 raft_consensus.cc:2272] T ccb9f9f9dbb04b459b94b15ddb6864d5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.318742  8925 tablet_replica.cc:333] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.319540  8925 raft_consensus.cc:2243] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.320127  8925 raft_consensus.cc:2272] T 8ac0dee3476f4361a8531c64161fee77 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.322288  8925 tablet_replica.cc:333] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.323016  8925 raft_consensus.cc:2243] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.324138  8925 raft_consensus.cc:2272] T f54c60d833534725beae61cec6e75c4b P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.326586  8925 tablet_replica.cc:333] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.327368  8925 raft_consensus.cc:2243] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.327936  8925 raft_consensus.cc:2272] T e3bb98bec02448d6990ba27836e9e7a9 P f3e2fa7cf3974c27a19386f362688d29 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.330236  8925 tablet_replica.cc:333] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.330937  8925 raft_consensus.cc:2243] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.331570  8925 raft_consensus.cc:2272] T 14fe966c22b14da9985eba8dd9ed1f2c P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.334285  8925 tablet_replica.cc:333] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.335296  8925 raft_consensus.cc:2243] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.335944  8925 raft_consensus.cc:2272] T d350b38546104d82881fdcd2f9c251d3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.339174  8925 tablet_replica.cc:333] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.339931  8925 raft_consensus.cc:2243] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.340889  8925 raft_consensus.cc:2272] T 4f2be91edb9e4188a248db3e309f0e44 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.343822  8925 tablet_replica.cc:333] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.344558  8925 raft_consensus.cc:2243] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.345116  8925 raft_consensus.cc:2272] T 4832e091fcef44ab877283bf9a254752 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.347628  8925 tablet_replica.cc:333] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.348326  8925 raft_consensus.cc:2243] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.348839  8925 raft_consensus.cc:2272] T 4ad65f24c73445fca035eefe94eb8d15 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.351037  8925 tablet_replica.cc:333] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.351778  8925 raft_consensus.cc:2243] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.352718  8925 raft_consensus.cc:2272] T 04089f925f3e44ab917020605bd8a75e P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.354964  8925 tablet_replica.cc:333] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.355635  8925 raft_consensus.cc:2243] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.356539  8925 raft_consensus.cc:2272] T a4a64164017441fd8ce325d46dcbbd07 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.358887  8925 tablet_replica.cc:333] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.359609  8925 raft_consensus.cc:2243] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.360522  8925 raft_consensus.cc:2272] T a3b2d906586545bda1c94132ed20782a P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.362794  8925 tablet_replica.cc:333] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.363566  8925 raft_consensus.cc:2243] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.364298  8925 raft_consensus.cc:2272] T 3e3966983799476b9516092542b22438 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.367439  8925 tablet_replica.cc:333] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.368299  8925 raft_consensus.cc:2243] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.369498  8925 raft_consensus.cc:2272] T 35fbf5e9b0e446b39953dd28e5aa71be P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.372707  8925 tablet_replica.cc:333] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.373675  8925 raft_consensus.cc:2243] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.374653  8925 raft_consensus.cc:2272] T a01b903fffc64c949e13c3c74abe8489 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.376844  8925 tablet_replica.cc:333] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.377420  8925 raft_consensus.cc:2243] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.377872  8925 raft_consensus.cc:2272] T e076455b2f73491282b009a90112d07d P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.380106  8925 tablet_replica.cc:333] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.380777  8925 raft_consensus.cc:2243] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.381204  8925 raft_consensus.cc:2272] T bdd0d96cdc814e8fa997150f58dda773 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.383378  8925 tablet_replica.cc:333] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.383963  8925 raft_consensus.cc:2243] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.384534  8925 raft_consensus.cc:2272] T c692689ede48466a882155f06b0763ad P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.386765  8925 tablet_replica.cc:333] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.387458  8925 raft_consensus.cc:2243] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.387915  8925 raft_consensus.cc:2272] T 03fbc43806c04306990613e14c199ff1 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.390134  8925 tablet_replica.cc:333] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.390745  8925 raft_consensus.cc:2243] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.391271  8925 raft_consensus.cc:2272] T 7ac00a69e4894ac79bdbbb74eaa3ad51 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.393554  8925 tablet_replica.cc:333] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.394121  8925 raft_consensus.cc:2243] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.395067  8925 raft_consensus.cc:2272] T ab7b798c63c94bce8e69fefd480e295b P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.397367  8925 tablet_replica.cc:333] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.397929  8925 raft_consensus.cc:2243] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.398363  8925 raft_consensus.cc:2272] T 6fde9dbeb78f450581397317487e31f0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.400627  8925 tablet_replica.cc:333] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.401160  8925 raft_consensus.cc:2243] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.401983  8925 raft_consensus.cc:2272] T 33445ebf2e3d4cb2bfea774ce4be1ee4 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.404297  8925 tablet_replica.cc:333] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.404884  8925 raft_consensus.cc:2243] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.405316  8925 raft_consensus.cc:2272] T e3e819f8fc7440af86dd0b3190f1edf3 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.407393  8925 tablet_replica.cc:333] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.407958  8925 raft_consensus.cc:2243] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.408564  8925 raft_consensus.cc:2272] T fee6b2223720435195a84c3570506ea5 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.410732  8925 tablet_replica.cc:333] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.411465  8925 raft_consensus.cc:2243] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.412353  8925 raft_consensus.cc:2272] T d850a82d849e47ef9576b6d0b73700c0 P f3e2fa7cf3974c27a19386f362688d29 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.414702  8925 tablet_replica.cc:333] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.415302  8925 raft_consensus.cc:2243] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.415818  8925 raft_consensus.cc:2272] T d3f20d931bf640d09b913bc7a1e8e3a8 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.417938  8925 tablet_replica.cc:333] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29: stopping tablet replica
I20260709 13:22:14.418547  8925 raft_consensus.cc:2243] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:14.419267  8925 raft_consensus.cc:2272] T 0346a3d5c16b4965a7492826c0da6872 P f3e2fa7cf3974c27a19386f362688d29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.496567  8925 tablet_server.cc:196] TabletServer@127.8.183.68:0 shutdown complete.
I20260709 13:22:14.580554  8925 master.cc:562] Master@127.8.183.126:45441 shutting down...
I20260709 13:22:14.629290  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:14.630049  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:14.630424  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P e2ca2de18d2e4c83a44246d79f233970: stopping tablet replica
I20260709 13:22:14.656553  8925 master.cc:584] Master@127.8.183.126:45441 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (44772 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:22:14.767992  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:41951
I20260709 13:22:14.769407  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:14.777699 20098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:14.778280 20097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:14.778321 20100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:14.779079  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:14.780792  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:14.781016  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:14.781141  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603334781130 us; error 0 us; skew 500 ppm
I20260709 13:22:14.781690  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:14.785241  8925 webserver.cc:533] Webserver started at http://127.8.183.126:41149/ using document root <none> and password file <none>
I20260709 13:22:14.785809  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:14.785990  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:14.786239  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:14.787705  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "99a427d28fdd433b9197e12e4b98a318"
format_stamp: "Formatted at 2026-07-09 13:22:14 on dist-test-slave-ch3x"
I20260709 13:22:14.793313  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:22:14.797881 20105 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:14.798918  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:22:14.799345  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "99a427d28fdd433b9197e12e4b98a318"
format_stamp: "Formatted at 2026-07-09 13:22:14 on dist-test-slave-ch3x"
I20260709 13:22:14.799713  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:14.817183  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:14.818662  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:14.865847  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:41951
I20260709 13:22:14.866016 20156 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:41951 every 8 connection(s)
I20260709 13:22:14.870842 20157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.884817 20157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318: Bootstrap starting.
I20260709 13:22:14.891760 20157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.897768 20157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318: No bootstrap required, opened a new log
I20260709 13:22:14.900516 20157 raft_consensus.cc:359] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER }
I20260709 13:22:14.901039 20157 raft_consensus.cc:385] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.901317 20157 raft_consensus.cc:740] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a427d28fdd433b9197e12e4b98a318, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.902171 20157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER }
I20260709 13:22:14.902783 20157 raft_consensus.cc:399] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:14.903064 20157 raft_consensus.cc:493] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:14.903468 20157 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:14.909276 20157 raft_consensus.cc:515] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER }
I20260709 13:22:14.910028 20157 leader_election.cc:304] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99a427d28fdd433b9197e12e4b98a318; no voters: 
I20260709 13:22:14.911489 20157 leader_election.cc:290] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:14.911924 20160 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:14.913743 20160 raft_consensus.cc:697] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 1 LEADER]: Becoming Leader. State: Replica: 99a427d28fdd433b9197e12e4b98a318, State: Running, Role: LEADER
I20260709 13:22:14.914620 20160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER }
I20260709 13:22:14.915627 20157 sys_catalog.cc:565] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:14.919770 20162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99a427d28fdd433b9197e12e4b98a318. Latest consensus state: current_term: 1 leader_uuid: "99a427d28fdd433b9197e12e4b98a318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER } }
I20260709 13:22:14.920044 20161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99a427d28fdd433b9197e12e4b98a318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99a427d28fdd433b9197e12e4b98a318" member_type: VOTER } }
I20260709 13:22:14.920922 20162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:14.921139 20161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:14.924093 20166 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:14.930249 20166 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:14.936877  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:14.943207 20166 catalog_manager.cc:1383] Generated new cluster ID: fc5634dae6204447a707f3865b9b6d9c
I20260709 13:22:14.943565 20166 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:14.962700 20166 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:14.964571 20166 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:14.989281 20166 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318: Generated new TSK 0
I20260709 13:22:14.990131 20166 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:15.005585  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:15.013411 20178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:15.014923 20179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:15.017910  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:15.018015 20181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:15.019850  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:15.020104  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:15.020255  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603335020243 us; error 0 us; skew 500 ppm
I20260709 13:22:15.020808  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:15.024325  8925 webserver.cc:533] Webserver started at http://127.8.183.65:46807/ using document root <none> and password file <none>
I20260709 13:22:15.024904  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:15.025100  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:15.025364  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:15.026634  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.032336  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:22:15.037106 20186 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.038288  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:22:15.038659  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.038975  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:15.052419  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:15.053857  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:15.055552  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:15.058671  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:15.058933  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.059257  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:15.059453  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.111378  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:41519
I20260709 13:22:15.111503 20248 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:41519 every 8 connection(s)
I20260709 13:22:15.117274  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:15.127651 20254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:15.127673 20253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:15.132925 20256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:15.138904  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:15.141036  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:15.141458 20249 heartbeater.cc:344] Connected to a master server at 127.8.183.126:41951
W20260709 13:22:15.141481  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:15.142019  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603335141989 us; error 0 us; skew 500 ppm
I20260709 13:22:15.142078 20249 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:15.142858  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:15.143383 20249 heartbeater.cc:507] Master 127.8.183.126:41951 requested a full tablet report, sending...
I20260709 13:22:15.146938 20122 ts_manager.cc:194] Registered new tserver with Master: c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
I20260709 13:22:15.146941  8925 webserver.cc:533] Webserver started at http://127.8.183.66:43133/ using document root <none> and password file <none>
I20260709 13:22:15.148043  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:15.148305  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:15.148640  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:15.149559 20122 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48232
I20260709 13:22:15.150324  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "011628ba1ff8482686fdab14ac8a56db"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.156435  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:22:15.160806 20261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.161911  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:15.162230  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "011628ba1ff8482686fdab14ac8a56db"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.162530  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:15.180270  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:15.181721  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:15.183545  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:15.186379  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:15.186607  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.186908  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:15.187062  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.242033  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:46687
I20260709 13:22:15.242151 20323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:46687 every 8 connection(s)
I20260709 13:22:15.248490  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:15.258853 20327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:15.261904 20328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:15.267032 20330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:15.267679  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:15.267969 20324 heartbeater.cc:344] Connected to a master server at 127.8.183.126:41951
I20260709 13:22:15.268549 20324 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:15.269035  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:15.269380  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:15.269490 20324 heartbeater.cc:507] Master 127.8.183.126:41951 requested a full tablet report, sending...
I20260709 13:22:15.269599  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603335269562 us; error 0 us; skew 500 ppm
I20260709 13:22:15.270625  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:15.272466 20122 ts_manager.cc:194] Registered new tserver with Master: 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:15.273847  8925 webserver.cc:533] Webserver started at http://127.8.183.67:33235/ using document root <none> and password file <none>
I20260709 13:22:15.274430 20122 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48240
I20260709 13:22:15.274556  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:15.275017  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:15.275696  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:15.277540  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "d8b8188a96bc4327a85238f8d21b4e20"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.283792  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:22:15.288532 20335 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.289649  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:22:15.290071  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "d8b8188a96bc4327a85238f8d21b4e20"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:15.290469  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:15.303681  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:15.305207  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:15.306823  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:15.309944  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:15.310370  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.310736  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:15.311007  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:15.365635  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:43763
I20260709 13:22:15.365818 20397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:43763 every 8 connection(s)
I20260709 13:22:15.385591 20398 heartbeater.cc:344] Connected to a master server at 127.8.183.126:41951
I20260709 13:22:15.386261 20398 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:15.387106 20398 heartbeater.cc:507] Master 127.8.183.126:41951 requested a full tablet report, sending...
I20260709 13:22:15.389846 20122 ts_manager.cc:194] Registered new tserver with Master: d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:15.390321  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020806214s
I20260709 13:22:15.391637 20122 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48242
I20260709 13:22:15.425185 20122 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48250:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:15.429129 20122 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:15.521908 20289 tablet_service.cc:1511] Processing CreateTablet for tablet 38f6496480a24495a98022256578d05f (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:15.523757 20289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f6496480a24495a98022256578d05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.528371 20288 tablet_service.cc:1511] Processing CreateTablet for tablet 0b69c46b5e834f90be7fa65c6dee4aa3 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:15.530227 20288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b69c46b5e834f90be7fa65c6dee4aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.532272 20286 tablet_service.cc:1511] Processing CreateTablet for tablet 579315f866164c9e850c8c9eaaada965 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:15.534109 20286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579315f866164c9e850c8c9eaaada965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.536381 20287 tablet_service.cc:1511] Processing CreateTablet for tablet db01cf8910de4d24a3140ef547b492ed (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:15.538224 20287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db01cf8910de4d24a3140ef547b492ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.536571 20214 tablet_service.cc:1511] Processing CreateTablet for tablet 38f6496480a24495a98022256578d05f (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:15.538456 20213 tablet_service.cc:1511] Processing CreateTablet for tablet 0b69c46b5e834f90be7fa65c6dee4aa3 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:15.541149 20212 tablet_service.cc:1511] Processing CreateTablet for tablet db01cf8910de4d24a3140ef547b492ed (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:15.541512 20285 tablet_service.cc:1511] Processing CreateTablet for tablet 1a3fe03229a74edc95ba4c0074f0ce7c (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:15.543083 20285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3fe03229a74edc95ba4c0074f0ce7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.543809 20211 tablet_service.cc:1511] Processing CreateTablet for tablet 579315f866164c9e850c8c9eaaada965 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:15.546130 20284 tablet_service.cc:1511] Processing CreateTablet for tablet 77d69a876a07439493c3b99d2d4733c1 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:15.548035 20284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d69a876a07439493c3b99d2d4733c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.547716 20209 tablet_service.cc:1511] Processing CreateTablet for tablet 77d69a876a07439493c3b99d2d4733c1 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:15.553978 20283 tablet_service.cc:1511] Processing CreateTablet for tablet 7b55fc1e5174468da1f8a7858bc90d77 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:15.546241 20210 tablet_service.cc:1511] Processing CreateTablet for tablet 1a3fe03229a74edc95ba4c0074f0ce7c (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:15.558145 20282 tablet_service.cc:1511] Processing CreateTablet for tablet e7ebf561fbbb487daa09f2db1eb9a685 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:15.559065 20214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f6496480a24495a98022256578d05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.560565 20281 tablet_service.cc:1511] Processing CreateTablet for tablet 97bf353b7e1a4b0f98275f4f6326838d (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:15.568562 20281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bf353b7e1a4b0f98275f4f6326838d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.568939 20208 tablet_service.cc:1511] Processing CreateTablet for tablet 7b55fc1e5174468da1f8a7858bc90d77 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:15.569607 20282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ebf561fbbb487daa09f2db1eb9a685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.570770 20208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b55fc1e5174468da1f8a7858bc90d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.571532 20209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d69a876a07439493c3b99d2d4733c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.572937 20210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3fe03229a74edc95ba4c0074f0ce7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.574857 20283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b55fc1e5174468da1f8a7858bc90d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.575572 20206 tablet_service.cc:1511] Processing CreateTablet for tablet 97bf353b7e1a4b0f98275f4f6326838d (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:15.579615 20205 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcca7ef5c974670bd13d8d33e23e3c0 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:15.573292 20207 tablet_service.cc:1511] Processing CreateTablet for tablet e7ebf561fbbb487daa09f2db1eb9a685 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:15.576172 20213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b69c46b5e834f90be7fa65c6dee4aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.582854 20212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db01cf8910de4d24a3140ef547b492ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.583670 20207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ebf561fbbb487daa09f2db1eb9a685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.586369 20206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bf353b7e1a4b0f98275f4f6326838d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.591917 20205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcca7ef5c974670bd13d8d33e23e3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.595624 20211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579315f866164c9e850c8c9eaaada965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.599597 20363 tablet_service.cc:1511] Processing CreateTablet for tablet 38f6496480a24495a98022256578d05f (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:15.603080 20413 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:15.603832 20363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f6496480a24495a98022256578d05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.612979 20362 tablet_service.cc:1511] Processing CreateTablet for tablet 0b69c46b5e834f90be7fa65c6dee4aa3 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:15.614802 20362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b69c46b5e834f90be7fa65c6dee4aa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.623032 20288 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcca7ef5c974670bd13d8d33e23e3c0 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:15.625398 20288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcca7ef5c974670bd13d8d33e23e3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.691090 20361 tablet_service.cc:1511] Processing CreateTablet for tablet db01cf8910de4d24a3140ef547b492ed (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:15.717392 20361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db01cf8910de4d24a3140ef547b492ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.726713 20360 tablet_service.cc:1511] Processing CreateTablet for tablet 579315f866164c9e850c8c9eaaada965 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:15.729417 20360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579315f866164c9e850c8c9eaaada965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.728780 20413 tablet_bootstrap.cc:654] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.748664 20414 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:15.748976 20359 tablet_service.cc:1511] Processing CreateTablet for tablet 1a3fe03229a74edc95ba4c0074f0ce7c (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:15.750813 20359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a3fe03229a74edc95ba4c0074f0ce7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.755486 20358 tablet_service.cc:1511] Processing CreateTablet for tablet 77d69a876a07439493c3b99d2d4733c1 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:15.757355 20358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d69a876a07439493c3b99d2d4733c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.762005 20357 tablet_service.cc:1511] Processing CreateTablet for tablet 7b55fc1e5174468da1f8a7858bc90d77 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:15.785003 20355 tablet_service.cc:1511] Processing CreateTablet for tablet 97bf353b7e1a4b0f98275f4f6326838d (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:15.785045 20356 tablet_service.cc:1511] Processing CreateTablet for tablet e7ebf561fbbb487daa09f2db1eb9a685 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:15.764930 20357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b55fc1e5174468da1f8a7858bc90d77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.818392 20356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ebf561fbbb487daa09f2db1eb9a685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.819283 20355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bf353b7e1a4b0f98275f4f6326838d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.830384 20354 tablet_service.cc:1511] Processing CreateTablet for tablet 8bcca7ef5c974670bd13d8d33e23e3c0 (DEFAULT_TABLE table=test-workload [id=6d0d1935e50e4eb3ae4a1ef978101839]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:15.832005 20414 tablet_bootstrap.cc:654] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.832561 20354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bcca7ef5c974670bd13d8d33e23e3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.839984 20413 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:15.840744 20413 ts_tablet_manager.cc:1403] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.238s	user 0.027s	sys 0.041s
I20260709 13:22:15.844918 20413 raft_consensus.cc:359] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.846022 20413 raft_consensus.cc:385] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.846359 20413 raft_consensus.cc:740] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.847419 20413 consensus_queue.cc:260] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.865742 20416 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:15.872901 20324 heartbeater.cc:499] Master 127.8.183.126:41951 was elected leader, sending a full tablet report...
I20260709 13:22:15.873940 20413 ts_tablet_manager.cc:1434] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.033s	user 0.015s	sys 0.017s
I20260709 13:22:15.874553 20414 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:15.875633 20413 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:15.882341 20416 tablet_bootstrap.cc:654] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.882843 20413 tablet_bootstrap.cc:654] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.883515 20414 ts_tablet_manager.cc:1403] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.135s	user 0.017s	sys 0.005s
I20260709 13:22:15.888047 20414 raft_consensus.cc:359] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.888875 20414 raft_consensus.cc:385] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.889206 20414 raft_consensus.cc:740] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.890237 20414 consensus_queue.cc:260] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.915563 20414 ts_tablet_manager.cc:1434] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.006s
I20260709 13:22:15.918318 20249 heartbeater.cc:499] Master 127.8.183.126:41951 was elected leader, sending a full tablet report...
I20260709 13:22:15.919234 20414 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:15.926964 20413 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:15.927834 20413 ts_tablet_manager.cc:1403] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.004s
W20260709 13:22:15.930563 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:15.931175 20414 tablet_bootstrap.cc:654] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.931670 20413 raft_consensus.cc:359] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.932665 20413 raft_consensus.cc:385] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.933040 20413 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.934273 20413 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.935784 20416 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:15.936571 20416 ts_tablet_manager.cc:1403] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.012s
I20260709 13:22:15.938611 20413 ts_tablet_manager.cc:1434] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:22:15.940600 20413 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:15.946842 20416 raft_consensus.cc:359] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.947966 20416 raft_consensus.cc:385] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.948311 20416 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.949426 20416 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.950722 20413 tablet_bootstrap.cc:654] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.955775 20398 heartbeater.cc:499] Master 127.8.183.126:41951 was elected leader, sending a full tablet report...
I20260709 13:22:15.957553 20416 ts_tablet_manager.cc:1434] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.006s
I20260709 13:22:15.959762 20416 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:15.965052 20414 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:15.966029 20414 ts_tablet_manager.cc:1403] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.047s	user 0.025s	sys 0.009s
I20260709 13:22:15.970331 20414 raft_consensus.cc:359] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.971508 20414 raft_consensus.cc:385] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.970927 20416 tablet_bootstrap.cc:654] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.972028 20414 raft_consensus.cc:740] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.973542 20414 consensus_queue.cc:260] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.976650 20414 ts_tablet_manager.cc:1434] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:22:15.978328 20414 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:15.980430 20413 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:15.981487 20413 ts_tablet_manager.cc:1403] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.014s
I20260709 13:22:15.987073 20413 raft_consensus.cc:359] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.988375 20413 raft_consensus.cc:385] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.988849 20413 raft_consensus.cc:740] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.990231 20413 consensus_queue.cc:260] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:15.993475 20413 ts_tablet_manager.cc:1434] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:22:15.994777 20413 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:15.995530 20414 tablet_bootstrap.cc:654] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:15.998066 20325 tablet.cc:2406] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:15.998159 20416 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:15.999400 20416 ts_tablet_manager.cc:1403] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.019s
I20260709 13:22:16.002620 20413 tablet_bootstrap.cc:654] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.003357 20416 raft_consensus.cc:359] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.004333 20416 raft_consensus.cc:385] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.004725 20416 raft_consensus.cc:740] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.005712 20416 consensus_queue.cc:260] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.011426 20416 ts_tablet_manager.cc:1434] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.006s
I20260709 13:22:16.013279 20416 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:16.014259 20414 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:16.015090 20414 ts_tablet_manager.cc:1403] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.012s
I20260709 13:22:16.019347 20414 raft_consensus.cc:359] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.020895 20414 raft_consensus.cc:385] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.021332 20414 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.022666 20416 tablet_bootstrap.cc:654] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.022249 20414 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.023945 20413 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:16.024695 20413 ts_tablet_manager.cc:1403] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.030s	user 0.002s	sys 0.021s
I20260709 13:22:16.025430 20414 ts_tablet_manager.cc:1434] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:16.027099 20414 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:16.029701 20413 raft_consensus.cc:359] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.032455 20413 raft_consensus.cc:385] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.032977 20413 raft_consensus.cc:740] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.034163 20413 consensus_queue.cc:260] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.040266 20413 ts_tablet_manager.cc:1434] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 13:22:16.041486 20414 tablet_bootstrap.cc:654] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.043901 20413 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:16.043712 20423 raft_consensus.cc:493] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.044762 20423 raft_consensus.cc:515] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.052635 20413 tablet_bootstrap.cc:654] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.056710 20423 leader_election.cc:290] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:16.060922 20416 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:16.062131 20416 ts_tablet_manager.cc:1403] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.009s
I20260709 13:22:16.066008 20416 raft_consensus.cc:359] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.066735 20416 raft_consensus.cc:385] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.067296 20416 raft_consensus.cc:740] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.068856 20414 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:16.069700 20414 ts_tablet_manager.cc:1403] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.019s
I20260709 13:22:16.071069 20413 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:16.071846 20413 ts_tablet_manager.cc:1403] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.001s
I20260709 13:22:16.068352 20416 consensus_queue.cc:260] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.073624 20414 raft_consensus.cc:359] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.074612 20414 raft_consensus.cc:385] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.075008 20414 raft_consensus.cc:740] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.075320 20413 raft_consensus.cc:359] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.076236 20413 raft_consensus.cc:385] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.076752 20413 raft_consensus.cc:740] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.076241 20414 consensus_queue.cc:260] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.077905 20413 consensus_queue.cc:260] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.080856 20413 ts_tablet_manager.cc:1434] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:22:16.082276 20413 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:16.083593 20416 ts_tablet_manager.cc:1434] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.006s
I20260709 13:22:16.083781 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a3fe03229a74edc95ba4c0074f0ce7c" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:16.084748 20224 raft_consensus.cc:2468] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:16.085204 20416 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:16.087765 20418 raft_consensus.cc:493] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.088490 20418 raft_consensus.cc:515] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.090508 20413 tablet_bootstrap.cc:654] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.095841 20336 leader_election.cc:304] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:16.097088 20423 raft_consensus.cc:2804] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.097599 20423 raft_consensus.cc:493] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.099650 20418 leader_election.cc:290] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.100298 20423 raft_consensus.cc:3060] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.095336 20414 ts_tablet_manager.cc:1434] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.004s
I20260709 13:22:16.109356 20414 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:16.132323 20416 tablet_bootstrap.cc:654] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.136505 20421 raft_consensus.cc:493] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.139016 20414 tablet_bootstrap.cc:654] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.138823 20421 raft_consensus.cc:515] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.142566 20421 leader_election.cc:290] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.141239 20423 raft_consensus.cc:515] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.159543 20423 leader_election.cc:290] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:16.160699 20416 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:16.161437 20416 ts_tablet_manager.cc:1403] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.005s
I20260709 13:22:16.161312 20413 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:16.163709 20423 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.164590 20413 ts_tablet_manager.cc:1403] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.012s
I20260709 13:22:16.164443 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a3fe03229a74edc95ba4c0074f0ce7c" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:16.165442 20298 raft_consensus.cc:3060] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.168373 20416 raft_consensus.cc:359] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.168620 20413 raft_consensus.cc:359] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.169528 20416 raft_consensus.cc:385] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.169624 20413 raft_consensus.cc:385] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.169982 20416 raft_consensus.cc:740] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.170150 20413 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.163496 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a3fe03229a74edc95ba4c0074f0ce7c" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.164582 20423 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.173463 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a3fe03229a74edc95ba4c0074f0ce7c" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:16.174575 20224 raft_consensus.cc:3060] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.174988 20423 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:16.174162 20413 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.177840 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.178754 20299 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:16.180634 20338 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:16.182168 20423 raft_consensus.cc:2804] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.182684 20423 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.183133 20423 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.184589 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:16.183192 20298 raft_consensus.cc:2468] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:16.176086 20416 consensus_queue.cc:260] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.191828 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
W20260709 13:22:16.192152 20336 leader_election.cc:343] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.192756 20223 raft_consensus.cc:2468] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 0.
I20260709 13:22:16.197214 20414 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:16.197932 20414 ts_tablet_manager.cc:1403] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.007s
I20260709 13:22:16.202095 20413 ts_tablet_manager.cc:1434] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.001s
I20260709 13:22:16.203641 20413 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:16.204659 20262 leader_election.cc:304] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:16.206038 20418 raft_consensus.cc:2804] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.206549 20418 raft_consensus.cc:493] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.206940 20418 raft_consensus.cc:3060] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.204800 20414 raft_consensus.cc:359] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.207767 20414 raft_consensus.cc:385] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.208750 20414 raft_consensus.cc:740] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.210482 20224 raft_consensus.cc:2468] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:16.191762 20338 leader_election.cc:304] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:16.214334 20416 ts_tablet_manager.cc:1434] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.052s	user 0.016s	sys 0.003s
I20260709 13:22:16.214498 20413 tablet_bootstrap.cc:654] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.219322 20443 raft_consensus.cc:2804] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.220151 20443 raft_consensus.cc:697] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:16.221211 20443 consensus_queue.cc:237] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.228109 20416 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:16.210430 20414 consensus_queue.cc:260] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.209448 20423 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.236204 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
W20260709 13:22:16.243683 20263 leader_election.cc:343] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.256866 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.243666 20414 ts_tablet_manager.cc:1434] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.002s
I20260709 13:22:16.254590 20418 raft_consensus.cc:515] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.263064 20298 raft_consensus.cc:2393] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c653c4d65e9a4850abfe3c2c9dff4d26 in current term 1: Already voted for candidate 011628ba1ff8482686fdab14ac8a56db in this term.
I20260709 13:22:16.261845 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:16.265821 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:16.266052 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:16.267261 20223 raft_consensus.cc:3060] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:16.269140 20336 leader_election.cc:343] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.261925 20414 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
W20260709 13:22:16.267442 20188 leader_election.cc:343] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.273770 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
W20260709 13:22:16.276075 20263 leader_election.cc:343] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.282606 20446 raft_consensus.cc:493] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.283399 20446 raft_consensus.cc:515] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.284529 20423 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:16.277663 20418 leader_election.cc:290] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.286234 20446 leader_election.cc:290] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.338639 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:16.344061 20421 raft_consensus.cc:493] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.344874 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:16.347079 20373 raft_consensus.cc:2468] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 0.
I20260709 13:22:16.348534 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:16.350425 20299 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.350695 20263 leader_election.cc:304] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
W20260709 13:22:16.353861 20262 leader_election.cc:343] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.355612 20423 raft_consensus.cc:493] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.359052 20423 raft_consensus.cc:515] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.361275 20414 tablet_bootstrap.cc:654] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.346535 20421 raft_consensus.cc:515] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.364975 20189 leader_election.cc:304] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:16.372961 20299 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:16.375058 20446 raft_consensus.cc:2804] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.376049 20446 raft_consensus.cc:493] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.376627 20446 raft_consensus.cc:3060] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.383553 20223 raft_consensus.cc:2468] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 1.
I20260709 13:22:16.390972 20416 tablet_bootstrap.cc:654] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.400695 20338 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:16.403275 20413 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:16.404032 20443 raft_consensus.cc:2804] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.397372 20446 raft_consensus.cc:515] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.405040 20443 raft_consensus.cc:697] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:16.404870 20413 ts_tablet_manager.cc:1403] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.201s	user 0.022s	sys 0.014s
I20260709 13:22:16.411455 20443 consensus_queue.cc:237] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.417122 20421 leader_election.cc:290] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.418088 20421 raft_consensus.cc:2749] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.419225 20423 leader_election.cc:290] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
I20260709 13:22:16.419144 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
W20260709 13:22:16.420419 20189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:46687, user_credentials={real_user=slave}} blocked reactor thread for 56049.7us
W20260709 13:22:16.421623 20188 leader_election.cc:343] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.423930 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.425079 20299 raft_consensus.cc:2468] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 0.
I20260709 13:22:16.425642 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.426203 20416 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:16.427680 20262 leader_election.cc:304] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:16.428697 20446 leader_election.cc:290] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.430264 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:16.430989 20373 raft_consensus.cc:3060] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.432811 20416 ts_tablet_manager.cc:1403] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.205s	user 0.027s	sys 0.012s
I20260709 13:22:16.437829 20449 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.437572 20121 catalog_manager.cc:5697] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:16.438920 20449 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.439879 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:16.441004 20223 raft_consensus.cc:2468] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:16.440575 20413 raft_consensus.cc:359] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.445916 20413 raft_consensus.cc:385] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.446504 20413 raft_consensus.cc:740] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.438843 20416 raft_consensus.cc:359] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.447659 20336 leader_election.cc:304] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:16.448196 20416 raft_consensus.cc:385] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.447713 20446 raft_consensus.cc:2804] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.448911 20416 raft_consensus.cc:740] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.449261 20446 raft_consensus.cc:697] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:16.450371 20372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:16.450556 20416 consensus_queue.cc:260] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.457829 20413 consensus_queue.cc:260] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.459435 20416 ts_tablet_manager.cc:1434] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.003s
I20260709 13:22:16.461629 20416 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
W20260709 13:22:16.463642 20338 leader_election.cc:343] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:16.464036 20188 leader_election.cc:343] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.450906 20446 consensus_queue.cc:237] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.471237 20413 ts_tablet_manager.cc:1434] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.004s
I20260709 13:22:16.472863 20416 tablet_bootstrap.cc:654] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.473707 20413 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:16.455427 20423 raft_consensus.cc:2804] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.446542 20449 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.481226 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.482409 20298 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 0.
I20260709 13:22:16.478564 20423 raft_consensus.cc:493] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.485502 20423 raft_consensus.cc:3060] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.486436 20414 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:16.487686 20414 ts_tablet_manager.cc:1403] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.226s	user 0.025s	sys 0.019s
I20260709 13:22:16.504961 20414 raft_consensus.cc:359] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.506114 20414 raft_consensus.cc:385] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.506548 20414 raft_consensus.cc:740] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.507813 20414 consensus_queue.cc:260] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.508504 20373 raft_consensus.cc:2468] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 1.
I20260709 13:22:16.510725 20263 leader_election.cc:304] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:16.511574 20414 ts_tablet_manager.cc:1434] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.004s
I20260709 13:22:16.512372 20446 raft_consensus.cc:2804] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.512584 20189 leader_election.cc:304] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:16.513199 20446 raft_consensus.cc:697] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:16.513681 20414 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:16.514456 20189 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:16.515714 20443 raft_consensus.cc:493] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.514631 20446 consensus_queue.cc:237] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.515166 20122 catalog_manager.cc:5697] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 0 to 1, leader changed from <none> to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 1 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.515755 20449 raft_consensus.cc:2804] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.516392 20443 raft_consensus.cc:515] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.518162 20449 raft_consensus.cc:493] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.526475 20449 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.519210 20421 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.527364 20421 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.527868 20421 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.532464 20416 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:16.533533 20418 raft_consensus.cc:493] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.535677 20414 tablet_bootstrap.cc:654] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.535223 20418 raft_consensus.cc:515] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.535341 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:16.541265 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:16.548070 20418 leader_election.cc:290] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763), c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
I20260709 13:22:16.552556 20416 ts_tablet_manager.cc:1403] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.091s	user 0.021s	sys 0.011s
W20260709 13:22:16.564308 20262 leader_election.cc:343] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.571154 20416 raft_consensus.cc:359] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.572395 20416 raft_consensus.cc:385] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.572747 20416 raft_consensus.cc:740] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.580031 20421 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.588924 20421 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Requested vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:16.588145 20413 tablet_bootstrap.cc:654] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.590070 20443 leader_election.cc:290] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:16.593122 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:16.610332 20414 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:16.588034 20416 consensus_queue.cc:260] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.612675 20423 raft_consensus.cc:515] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:16.614152 20414 ts_tablet_manager.cc:1403] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.101s	user 0.027s	sys 0.029s
I20260709 13:22:16.618285 20423 leader_election.cc:290] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Requested vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
I20260709 13:22:16.620604 20414 raft_consensus.cc:359] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:16.621685 20414 raft_consensus.cc:385] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.622015 20414 raft_consensus.cc:740] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.630125 20416 ts_tablet_manager.cc:1434] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.001s
I20260709 13:22:16.921798 20416 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:16.622923 20414 consensus_queue.cc:260] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.026703 20449 raft_consensus.cc:515] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.284132 20413 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:17.285203 20413 ts_tablet_manager.cc:1403] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.812s	user 0.030s	sys 0.037s
I20260709 13:22:17.334160 20449 leader_election.cc:290] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Requested vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.378484 20414 ts_tablet_manager.cc:1434] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.761s	user 0.008s	sys 0.004s
W20260709 13:22:17.379279 20414 ts_tablet_manager.cc:1467] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Tablet startup took 762ms
W20260709 13:22:17.379740 20414 ts_tablet_manager.cc:1469] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Trace:
0709 13:22:15.534568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.546376 (+ 11808us) service_pool.cc:224] Handling call
0709 13:22:15.561029 (+ 14653us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:22:15.561180 (+   151us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:22:15.707461 (+146281us) tablet_metadata.cc:665] Metadata flushed
0709 13:22:15.836055 (+128594us) tablet_replica.cc:206] Creating consensus instance
0709 13:22:15.836722 (+   667us) inbound_call.cc:177] Queueing success response
0709 13:22:16.512914 (+676192us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:22:16.616498 (+103584us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:22:16.617279 (+   781us) tablet_replica.cc:261] Starting instrumentation
0709 13:22:16.618087 (+   808us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":32948,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":15341,"tablet-open.queue_time_us":676262}
I20260709 13:22:17.406903 20414 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:17.407752 20416 tablet_bootstrap.cc:654] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.408993 20413 raft_consensus.cc:359] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.410024 20413 raft_consensus.cc:385] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.410372 20413 raft_consensus.cc:740] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.411283 20413 consensus_queue.cc:260] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.415607 20414 tablet_bootstrap.cc:654] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.432580 20463 raft_consensus.cc:493] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.444585 20414 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:17.445659 20414 ts_tablet_manager.cc:1403] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.012s
I20260709 13:22:17.454566 20463 raft_consensus.cc:515] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.464943 20416 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:17.465684 20463 leader_election.cc:290] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:17.465257 20414 raft_consensus.cc:359] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.465940 20416 ts_tablet_manager.cc:1403] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.545s	user 0.016s	sys 0.023s
I20260709 13:22:17.466408 20414 raft_consensus.cc:385] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.466843 20414 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.467937 20414 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.470672 20414 ts_tablet_manager.cc:1434] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:17.470230 20416 raft_consensus.cc:359] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.471304 20416 raft_consensus.cc:385] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.471692 20416 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.472318 20413 ts_tablet_manager.cc:1434] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.183s	user 0.027s	sys 0.014s
I20260709 13:22:17.472831 20414 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:17.472780 20416 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.474016 20413 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:17.477818 20416 ts_tablet_manager.cc:1434] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:22:17.479900 20416 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:17.485101 20413 tablet_bootstrap.cc:654] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.486124 20414 tablet_bootstrap.cc:654] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.489140 20416 tablet_bootstrap.cc:654] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.508657 20413 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:17.509486 20413 ts_tablet_manager.cc:1403] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.003s
I20260709 13:22:17.510710 20414 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:17.511407 20414 ts_tablet_manager.cc:1403] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.011s
I20260709 13:22:17.513119 20413 raft_consensus.cc:359] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.514277 20413 raft_consensus.cc:385] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.514640 20413 raft_consensus.cc:740] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.515638 20413 consensus_queue.cc:260] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.515506 20414 raft_consensus.cc:359] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.517184 20414 raft_consensus.cc:385] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.517648 20414 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.518627 20414 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.519174 20413 ts_tablet_manager.cc:1434] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.002s
I20260709 13:22:17.519752 20416 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:17.520910 20416 ts_tablet_manager.cc:1403] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.014s
I20260709 13:22:17.521548 20414 ts_tablet_manager.cc:1434] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:17.521622 20413 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: Bootstrap starting.
I20260709 13:22:17.523401 20414 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:17.524874 20416 raft_consensus.cc:359] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.525882 20416 raft_consensus.cc:385] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.526253 20416 raft_consensus.cc:740] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.526491 20449 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.527148 20449 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.527426 20416 consensus_queue.cc:260] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.530197 20449 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.533955 20416 ts_tablet_manager.cc:1434] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:22:17.535575 20413 tablet_bootstrap.cc:654] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.535925 20416 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:17.542404 20413 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: No bootstrap required, opened a new log
I20260709 13:22:17.543016 20413 ts_tablet_manager.cc:1403] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:22:17.544899 20416 tablet_bootstrap.cc:654] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.545637 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:17.546566 20299 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.547159 20413 raft_consensus.cc:359] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.548156 20413 raft_consensus.cc:385] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.548506 20413 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.549518 20373 raft_consensus.cc:2393] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 011628ba1ff8482686fdab14ac8a56db in current term 1: Already voted for candidate d8b8188a96bc4327a85238f8d21b4e20 in this term.
I20260709 13:22:17.549973 20462 raft_consensus.cc:493] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.544798 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:17.550436 20414 tablet_bootstrap.cc:654] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.550848 20462 raft_consensus.cc:515] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.551098 20223 raft_consensus.cc:2468] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 0.
I20260709 13:22:17.555756 20372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:17.556986 20372 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 0.
I20260709 13:22:17.557380 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:17.558702 20297 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.559911 20263 leader_election.cc:304] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:17.561707 20464 raft_consensus.cc:2804] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.562326 20464 raft_consensus.cc:493] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.562772 20464 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.571316 20462 leader_election.cc:290] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.571766 20449 raft_consensus.cc:493] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.572813 20449 raft_consensus.cc:515] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.576866 20296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:17.573020 20413 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.581435 20413 ts_tablet_manager.cc:1434] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:22:17.601730 20464 raft_consensus.cc:515] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.587767 20296 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 0.
I20260709 13:22:17.612344 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:17.613715 20223 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:17.616814 20464 leader_election.cc:290] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.618549 20372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:17.618893 20189 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:17.619499 20372 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:17.622051 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:17.618606 20449 leader_election.cc:290] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.629271 20449 raft_consensus.cc:2804] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.629902 20449 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.630366 20449 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.650368 20296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:17.651508 20296 raft_consensus.cc:2393] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c653c4d65e9a4850abfe3c2c9dff4d26 in current term 1: Already voted for candidate 011628ba1ff8482686fdab14ac8a56db in this term.
I20260709 13:22:17.652393 20120 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48232 (request call id 12) took 1101 ms. Trace:
I20260709 13:22:17.653005 20120 rpcz_store.cc:276] 0709 13:22:16.550790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:17.349209 (+798419us) service_pool.cc:224] Handling call
0709 13:22:17.652245 (+303036us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:22:17.657220 20372 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 1.
I20260709 13:22:17.660070 20263 leader_election.cc:304] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:17.661736 20464 raft_consensus.cc:2804] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.662506 20464 raft_consensus.cc:697] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:17.663861 20464 consensus_queue.cc:237] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.665134 20297 raft_consensus.cc:2468] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 1.
I20260709 13:22:17.662427 20299 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 1.
I20260709 13:22:17.668171 20189 leader_election.cc:304] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:17.668352 20373 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57186 (request call id 4) took 1128 ms. Trace:
I20260709 13:22:17.668926 20373 rpcz_store.cc:276] 0709 13:22:16.540393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:16.540680 (+   287us) service_pool.cc:224] Handling call
0709 13:22:17.668087 (+1127407us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400123754. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6d571 00007fa2780082fa 00007fa27cfc9b17 00007fa2765446d4 00007fa276544662 00007fa276544591 00007fa27654450c 00007fa2765437c2 00007fa27633f48c 00007fa27633e8c6 00007fa27633e26f
0709 13:22:17.668265 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310219648}
I20260709 13:22:17.670972 20473 raft_consensus.cc:2804] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.671361 20189 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:17.674396 20473 raft_consensus.cc:697] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Becoming Leader. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Running, Role: LEADER
I20260709 13:22:17.675331 20478 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.676128 20478 raft_consensus.cc:697] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Becoming Leader. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Running, Role: LEADER
I20260709 13:22:17.676265 20473 consensus_queue.cc:237] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.679529 20478 consensus_queue.cc:237] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.682756 20416 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:17.685915 20416 ts_tablet_manager.cc:1403] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.150s	user 0.057s	sys 0.049s
I20260709 13:22:17.689877 20336 leader_election.cc:304] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:17.690946 20416 raft_consensus.cc:359] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.692169 20416 raft_consensus.cc:385] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.692589 20466 raft_consensus.cc:2764] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:17.692619 20416 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.694229 20416 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.696797 20223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:17.707603 20219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:17.707939 20223 raft_consensus.cc:3060] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.708424 20219 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.709399 20416 ts_tablet_manager.cc:1434] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.003s
I20260709 13:22:17.709741 20371 raft_consensus.cc:3060] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.710919 20416 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: Bootstrap starting.
I20260709 13:22:17.697845 20222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:17.711994 20222 raft_consensus.cc:2393] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8b8188a96bc4327a85238f8d21b4e20 in current term 1: Already voted for candidate 011628ba1ff8482686fdab14ac8a56db in this term.
I20260709 13:22:17.689980 20122 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48240 (request call id 13) took 1190 ms. Trace:
I20260709 13:22:17.699345 20121 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48242 (request call id 9) took 1275 ms. Trace:
I20260709 13:22:17.713830 20121 rpcz_store.cc:276] 0709 13:22:16.423651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:16.423910 (+   259us) service_pool.cc:224] Handling call
0709 13:22:17.699251 (+1275341us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:16.480637 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:16.481535 (+   898us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:16.481602 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:22:16.481648 (+    46us) tablet.cc:662] Decoding operations
0709 13:22:16.482263 (+   615us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:16.482394 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:16.482436 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:22:16.482833 (+   397us) tablet.cc:701] Row locks acquired
0709 13:22:16.482876 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:22:16.483274 (+   398us) write_op.cc:270] Start()
0709 13:22:16.483461 (+   187us) write_op.cc:276] Timestamp: P: 1783603336483176 usec, L: 0
0709 13:22:16.483506 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:22:16.484868 (+  1362us) log.cc:844] Serialized 537 byte log entry
0709 13:22:16.547830 (+ 62962us) op_driver.cc:464] REPLICATION: finished
0709 13:22:16.628526 (+ 80696us) write_op.cc:301] APPLY: starting
0709 13:22:16.628710 (+   184us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:16.628957 (+   247us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:16.629254 (+   297us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:16.630088 (+   834us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:16.650108 (+ 20020us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:22:16.650260 (+   152us) write_op.cc:312] APPLY: finished
0709 13:22:16.650830 (+   570us) log.cc:844] Serialized 22 byte log entry
0709 13:22:16.651231 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:16.651386 (+   155us) write_op.cc:454] Released schema lock
0709 13:22:16.651491 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:16.652266 (+   775us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:16.652314 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":578,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":80141,"apply.run_cpu_time_us":4301,"apply.run_wall_time_us":23720,"num_ops":1,"prepare.queue_time_us":34510,"prepare.run_cpu_time_us":4610,"prepare.run_wall_time_us":31452,"replication_time_us":64073,"thread_start_us":68202,"threads_started":2}]]}
I20260709 13:22:17.700779 20373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:17.718730 20373 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.719359 20416 tablet_bootstrap.cc:654] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.701121 20372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:17.721666 20372 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.722864 20223 raft_consensus.cc:2468] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:17.724534 20224 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.703306 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:17.703979 20369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:17.728191 20368 raft_consensus.cc:2393] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c653c4d65e9a4850abfe3c2c9dff4d26 in current term 1: Already voted for candidate 011628ba1ff8482686fdab14ac8a56db in this term.
I20260709 13:22:17.713258 20122 rpcz_store.cc:276] 0709 13:22:16.499881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:16.500214 (+   333us) service_pool.cc:224] Handling call
0709 13:22:17.689848 (+1189634us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:17.663786 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:17.664483 (+   697us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:17.664525 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:22:17.664546 (+    21us) tablet.cc:662] Decoding operations
0709 13:22:17.664959 (+   413us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:17.665044 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:17.665082 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:22:17.665333 (+   251us) tablet.cc:701] Row locks acquired
0709 13:22:17.665354 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:22:17.665515 (+   161us) write_op.cc:270] Start()
0709 13:22:17.665658 (+   143us) write_op.cc:276] Timestamp: P: 1783603337665461 usec, L: 0
0709 13:22:17.665684 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:22:17.667630 (+  1946us) log.cc:844] Serialized 537 byte log entry
0709 13:22:17.671095 (+  3465us) op_driver.cc:464] REPLICATION: finished
0709 13:22:17.681709 (+ 10614us) write_op.cc:301] APPLY: starting
0709 13:22:17.681839 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:17.681967 (+   128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:17.682148 (+   181us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:17.682488 (+   340us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:17.682749 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:22:17.682815 (+    66us) write_op.cc:312] APPLY: finished
0709 13:22:17.683074 (+   259us) log.cc:844] Serialized 22 byte log entry
0709 13:22:17.687869 (+  4795us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:17.687984 (+   115us) write_op.cc:454] Released schema lock
0709 13:22:17.688045 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:17.690338 (+  2293us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:17.690380 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":100631,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":10179,"apply.run_cpu_time_us":2392,"apply.run_wall_time_us":8600,"num_ops":1,"prepare.queue_time_us":37624,"prepare.run_cpu_time_us":3148,"prepare.run_wall_time_us":4187,"replication_time_us":5206,"thread_start_us":44574,"threads_started":2}]]}
I20260709 13:22:17.704406 20262 leader_election.cc:304] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: d8b8188a96bc4327a85238f8d21b4e20
W20260709 13:22:17.705580 20338 leader_election.cc:343] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.735399 20464 raft_consensus.cc:2804] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.735935 20464 raft_consensus.cc:493] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.736299 20464 raft_consensus.cc:3060] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.706297 20220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:17.706727 20462 consensus_queue.cc:1048] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.743232 20414 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: No bootstrap required, opened a new log
I20260709 13:22:17.744215 20414 ts_tablet_manager.cc:1403] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.221s	user 0.035s	sys 0.042s
I20260709 13:22:17.747787 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579315f866164c9e850c8c9eaaada965" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:17.749148 20298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:17.749969 20462 consensus_queue.cc:1048] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.731036 20449 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.761631 20462 consensus_queue.cc:1048] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.763522 20449 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Requested vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.764027 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:17.764920 20297 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.764762 20414 raft_consensus.cc:359] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.765796 20414 raft_consensus.cc:385] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.766191 20414 raft_consensus.cc:740] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.767370 20414 consensus_queue.cc:260] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
W20260709 13:22:17.750886 20262 consensus_peers.cc:597] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:17.772567 20188 leader_election.cc:343] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.772692 20466 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.774549 20188 leader_election.cc:304] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:17.774866 20336 leader_election.cc:304] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:17.775285 20463 raft_consensus.cc:493] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.776193 20463 raft_consensus.cc:515] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.774940 20466 consensus_queue.cc:1048] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:22:17.779178 20463 leader_election.cc:290] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:17.780001 20463 raft_consensus.cc:2804] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.781427 20371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:17.782930 20416 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: No bootstrap required, opened a new log
I20260709 13:22:17.783505 20338 leader_election.cc:304] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8b8188a96bc4327a85238f8d21b4e20; no voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:17.783578 20416 ts_tablet_manager.cc:1403] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.033s
I20260709 13:22:17.787737 20478 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.788043 20416 raft_consensus.cc:359] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.788528 20478 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.792963 20478 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:17.793771 20416 raft_consensus.cc:385] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.794162 20416 raft_consensus.cc:740] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.795374 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:17.796650 20299 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:17.796483 20416 consensus_queue.cc:260] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.802512 20416 ts_tablet_manager.cc:1434] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:22:17.806355 20295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:17.807420 20466 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:22:17.806730 20338 leader_election.cc:304] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:17.757719 20464 raft_consensus.cc:515] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
W20260709 13:22:17.813442 20188 leader_election.cc:343] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.820286 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:17.821067 20298 raft_consensus.cc:2393] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8b8188a96bc4327a85238f8d21b4e20 in current term 1: Already voted for candidate 011628ba1ff8482686fdab14ac8a56db in this term.
I20260709 13:22:17.821385 20224 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 0.
I20260709 13:22:17.828490 20466 raft_consensus.cc:2804] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.831504 20464 leader_election.cc:290] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Requested vote from peers d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763), c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
I20260709 13:22:17.831730 20486 raft_consensus.cc:2749] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.840184 20466 raft_consensus.cc:493] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.832701 20371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:17.836594 20414 ts_tablet_manager.cc:1434] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 13:22:17.840726 20466 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.851336 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38f6496480a24495a98022256578d05f" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:17.852878 20224 raft_consensus.cc:2393] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 011628ba1ff8482686fdab14ac8a56db in current term 1: Already voted for candidate d8b8188a96bc4327a85238f8d21b4e20 in this term.
I20260709 13:22:17.870687 20219 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 1.
I20260709 13:22:17.872083 20449 raft_consensus.cc:2749] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.859897 20372 raft_consensus.cc:2468] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 1.
I20260709 13:22:17.867851 20463 raft_consensus.cc:697] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:17.874960 20463 consensus_queue.cc:237] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:17.890413 20297 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 1.
I20260709 13:22:17.893847 20189 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:17.899962 20449 raft_consensus.cc:2804] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.905179 20449 raft_consensus.cc:697] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Becoming Leader. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Running, Role: LEADER
I20260709 13:22:17.917303 20449 consensus_queue.cc:237] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.924311 20373 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in term 1.
I20260709 13:22:17.938731 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:17.903066 20494 consensus_queue.cc:1048] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.058s
W20260709 13:22:17.963440 20262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:41519, user_credentials={real_user=slave}} blocked reactor thread for 57641.5us
I20260709 13:22:17.956708 20466 raft_consensus.cc:515] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:17.953197 20121 catalog_manager.cc:5697] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 0 to 1, leader changed from <none> to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 1 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.995175 20188 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:18.000200 20449 raft_consensus.cc:2749] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:17.999435 20121 catalog_manager.cc:5697] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 0 to 1, leader changed from <none> to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 1 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.013386 20263 leader_election.cc:304] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db; no voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:18.035666 20219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:18.037114 20219 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.037444 20466 leader_election.cc:290] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:18.049700 20498 raft_consensus.cc:2749] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.049929 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:18.051728 20297 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:18.049593 20263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.67:43763, user_credentials={real_user=slave}} blocked reactor thread for 44833.2us
I20260709 13:22:18.061578 20122 catalog_manager.cc:5697] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.064251 20122 catalog_manager.cc:5697] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.084894 20219 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:18.086591 20336 leader_election.cc:304] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:18.087752 20487 raft_consensus.cc:2804] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.088325 20487 raft_consensus.cc:697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:18.089423 20487 consensus_queue.cc:237] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:18.093371 20297 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:18.104923 20464 consensus_queue.cc:1048] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.105238 20120 catalog_manager.cc:5697] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 0 to 1, leader changed from <none> to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } }
I20260709 13:22:18.120946 20498 consensus_queue.cc:1048] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.128026 20121 catalog_manager.cc:5697] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.142716 20449 raft_consensus.cc:493] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.143530 20449 raft_consensus.cc:515] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:18.144011 20122 catalog_manager.cc:5697] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.146256 20449 leader_election.cc:290] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:18.146872 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:18.149380 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b69c46b5e834f90be7fa65c6dee4aa3" candidate_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:18.151398 20188 leader_election.cc:304] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:18.152412 20449 raft_consensus.cc:2749] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.162636 20449 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.164192 20121 catalog_manager.cc:5697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.180326 20479 consensus_queue.cc:1048] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.180451 20449 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.197443 20487 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.198041 20487 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:18.199719 20449 consensus_queue.cc:1048] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.202023 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:18.203042 20297 raft_consensus.cc:2393] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8b8188a96bc4327a85238f8d21b4e20 in current term 1: Already voted for candidate c653c4d65e9a4850abfe3c2c9dff4d26 in this term.
I20260709 13:22:18.202950 20224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" is_pre_election: true
I20260709 13:22:18.205082 20336 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8b8188a96bc4327a85238f8d21b4e20; no voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:18.206077 20487 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:18.206238 20512 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.217872 20512 raft_consensus.cc:2749] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.234150 20224 raft_consensus.cc:3060] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.252195  8925 ts_manager.cc:295] Set tserver state for c653c4d65e9a4850abfe3c2c9dff4d26 to MAINTENANCE_MODE
I20260709 13:22:18.253415  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
I20260709 13:22:18.272949 20512 consensus_queue.cc:1048] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:22:18.291721 20336 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 159 similar messages]
W20260709 13:22:18.309686 20336 consensus_peers.cc:597] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:18.313529  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:18.315037  8925 tablet_replica.cc:333] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.316380  8925 raft_consensus.cc:2243] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.316977  8925 raft_consensus.cc:2272] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.319991  8925 tablet_replica.cc:333] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.320711  8925 raft_consensus.cc:2243] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.321201  8925 raft_consensus.cc:2272] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.323973  8925 tablet_replica.cc:333] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.324632  8925 raft_consensus.cc:2243] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:18.325326  8925 pending_rounds.cc:70] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Trying to abort 1 pending ops.
I20260709 13:22:18.325510  8925 pending_rounds.cc:77] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639272322998272 op_type: NO_OP noop_request { }
I20260709 13:22:18.325798  8925 raft_consensus.cc:2889] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:18.326076  8925 raft_consensus.cc:2272] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.328996  8925 tablet_replica.cc:333] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.329730  8925 raft_consensus.cc:2243] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.330160  8925 raft_consensus.cc:2272] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.332194  8925 tablet_replica.cc:333] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.332774  8925 raft_consensus.cc:2243] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.333225  8925 raft_consensus.cc:2272] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.335884  8925 tablet_replica.cc:333] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.336609  8925 raft_consensus.cc:2243] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.337064  8925 raft_consensus.cc:2272] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.339879  8925 tablet_replica.cc:333] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.340514  8925 raft_consensus.cc:2243] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:18.341697  8925 raft_consensus.cc:2272] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.344902  8925 tablet_replica.cc:333] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.345639  8925 raft_consensus.cc:2243] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.346216  8925 raft_consensus.cc:2272] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.348780  8925 tablet_replica.cc:333] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.349476  8925 raft_consensus.cc:2243] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:18.350505  8925 raft_consensus.cc:2272] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.353909  8925 tablet_replica.cc:333] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: stopping tablet replica
I20260709 13:22:18.354609  8925 raft_consensus.cc:2243] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:18.355221  8925 raft_consensus.cc:2272] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:18.393195  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:22:18.409837 20336 consensus_peers.cc:597] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:18.428402 20336 consensus_peers.cc:597] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:18.452193 20262 consensus_peers.cc:597] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:18.599705 20512 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:22:18.603854 20262 consensus_peers.cc:597] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:18.616144 20336 consensus_peers.cc:597] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:18.623432 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:18.652403 20394 debug-util.cc:398] Leaking SignalData structure 0x7b080001e460 after lost signal to thread 8928
W20260709 13:22:18.653564 20394 debug-util.cc:398] Leaking SignalData structure 0x7b080001ec60 after lost signal to thread 20156
W20260709 13:22:18.656124 20394 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc2e0 after lost signal to thread 20323
W20260709 13:22:18.657341 20394 debug-util.cc:398] Leaking SignalData structure 0x7b0800270280 after lost signal to thread 20397
W20260709 13:22:18.660506 20262 consensus_peers.cc:597] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:19.624683 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:19.676613 20464 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
I20260709 13:22:19.677446 20464 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.679639 20464 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:19.681442 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:19.685403 20512 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
W20260709 13:22:19.685509 20262 leader_election.cc:336] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.686302 20262 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db; no voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:19.686242 20512 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.687652 20464 raft_consensus.cc:2749] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:19.689916 20512 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:19.691506 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:19.692552 20297 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:19.693897 20512 raft_consensus.cc:493] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
I20260709 13:22:19.694303 20338 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:19.694744 20512 raft_consensus.cc:515] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
W20260709 13:22:19.696177 20336 leader_election.cc:336] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.697978 20530 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:19.698621 20530 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
I20260709 13:22:19.699179 20530 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.702015 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:19.704821 20464 raft_consensus.cc:493] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
I20260709 13:22:19.699942 20512 leader_election.cc:290] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:19.705711 20464 raft_consensus.cc:515] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
W20260709 13:22:19.705699 20336 leader_election.cc:336] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.707010 20336 leader_election.cc:304] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8b8188a96bc4327a85238f8d21b4e20; no voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:19.708884 20512 raft_consensus.cc:2749] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:19.710217 20464 leader_election.cc:290] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:19.710551 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20" is_pre_election: true
I20260709 13:22:19.711576 20368 raft_consensus.cc:2468] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 1.
I20260709 13:22:19.711776 20530 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.713302 20263 leader_election.cc:304] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:19.714718 20530 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:19.714912 20464 raft_consensus.cc:2804] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:19.715531 20464 raft_consensus.cc:493] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting leader election (detected failure of leader c653c4d65e9a4850abfe3c2c9dff4d26)
I20260709 13:22:19.716013 20464 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:19.721462 20336 leader_election.cc:336] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.726684 20464 raft_consensus.cc:515] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.728178 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
W20260709 13:22:19.728626 20262 leader_election.cc:336] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.729002 20297 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.732959 20464 leader_election.cc:290] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:19.734393 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b55fc1e5174468da1f8a7858bc90d77" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:19.735368 20368 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:19.741137 20297 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 2.
I20260709 13:22:19.745162 20338 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: c653c4d65e9a4850abfe3c2c9dff4d26
W20260709 13:22:19.745990 20262 leader_election.cc:336] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:19.747483 20530 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:19.750198 20530 raft_consensus.cc:697] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:19.751608 20530 consensus_queue.cc:237] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.752571 20368 raft_consensus.cc:2468] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 2.
I20260709 13:22:19.756906 20263 leader_election.cc:304] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:19.759034 20464 raft_consensus.cc:2804] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:19.765897 20464 raft_consensus.cc:697] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:19.767419 20464 consensus_queue.cc:237] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:19.782650 20120 catalog_manager.cc:5697] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 1 to 2, leader changed from c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.791857 20122 catalog_manager.cc:5697] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 1 to 2, leader changed from c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 2 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:20.073688 20530 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.074496 20530 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:20.077382 20530 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:20.078841 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db" is_pre_election: true
I20260709 13:22:20.079790 20297 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 1.
I20260709 13:22:20.081024 20338 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:20.082099 20530 raft_consensus.cc:2804] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:20.082625 20530 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:20.082901 20336 leader_election.cc:336] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:20.083192 20530 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:20.095429 20530 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:20.098332 20530 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:20.100363 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:20.101265 20297 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:20.103080 20336 leader_election.cc:336] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111)
I20260709 13:22:20.112036 20297 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 2.
I20260709 13:22:20.113931 20338 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, d8b8188a96bc4327a85238f8d21b4e20; no voters: c653c4d65e9a4850abfe3c2c9dff4d26
I20260709 13:22:20.115195 20530 raft_consensus.cc:2804] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:20.115732 20530 raft_consensus.cc:697] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:20.116782 20530 consensus_queue.cc:237] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:20.127266 20122 catalog_manager.cc:5697] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 1 to 2, leader changed from c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:20.140110 20336 consensus_peers.cc:597] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:20.236413 20262 consensus_peers.cc:597] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:20.240340 20297 raft_consensus.cc:1275] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:20.242360 20530 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:20.337570 20368 raft_consensus.cc:1275] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:20.339399 20464 consensus_queue.cc:1048] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:22:20.614120 20336 consensus_peers.cc:597] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:20.626029 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:20.630582 20297 raft_consensus.cc:1275] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:22:20.632630 20512 consensus_queue.cc:1048] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:20.731042 20336 consensus_peers.cc:597] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:20.791373 20336 consensus_peers.cc:597] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:20.938638 20336 consensus_peers.cc:597] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:21.066128 20262 consensus_peers.cc:597] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:21.141554 20336 consensus_peers.cc:597] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:21.180298 20262 consensus_peers.cc:597] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:21.262538 20262 consensus_peers.cc:597] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:21.627421 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:22.536875 20336 consensus_peers.cc:597] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:22.628656 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:22.748333 20262 consensus_peers.cc:597] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:23.025923 20336 consensus_peers.cc:597] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:23.130854 20336 consensus_peers.cc:597] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:23.339779 20336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111) [suppressed 97 similar messages]
W20260709 13:22:23.343479 20336 consensus_peers.cc:597] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:23.393975 20336 consensus_peers.cc:597] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:23.430383  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:23.441953 20552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:23.445253 20551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:23.450232  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:23.450359 20554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:23.452351  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:23.452713  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:23.452970  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603343452939 us; error 0 us; skew 500 ppm
I20260709 13:22:23.453879  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:23.458608  8925 webserver.cc:533] Webserver started at http://127.8.183.65:46807/ using document root <none> and password file <none>
I20260709 13:22:23.459427  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:23.459753  8925 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:22:23.464908  8925 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:22:23.470113 20559 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:23.471282  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:23.471798  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
format_stamp: "Formatted at 2026-07-09 13:22:15 on dist-test-slave-ch3x"
I20260709 13:22:23.472287  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:23.510998  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:23.512691  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:23.514916  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:23.520337 20566 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:22:23.579176 20336 consensus_peers.cc:597] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:23.592780  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:22:23.593429  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.075s	user 0.000s	sys 0.004s
I20260709 13:22:23.593904  8925 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:22:23.604375 20566 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.626048 20566 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:23.627285 20566 tablet_bootstrap.cc:492] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.628062 20566 ts_tablet_manager.cc:1403] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.024s	user 0.021s	sys 0.000s
W20260709 13:22:23.629982 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:23.631407 20566 raft_consensus.cc:359] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.632263 20566 raft_consensus.cc:740] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.633112 20566 consensus_queue.cc:260] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.636181 20566 ts_tablet_manager.cc:1434] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:23.637167 20566 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.654460 20566 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.655668 20566 tablet_bootstrap.cc:492] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.656433 20566 ts_tablet_manager.cc:1403] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:22:23.659381 20566 raft_consensus.cc:359] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.660106 20566 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.661048 20566 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.663156 20566 ts_tablet_manager.cc:1434] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:23.664095 20566 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.670105  8925 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:22:23.670672  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.077s	user 0.066s	sys 0.008s
I20260709 13:22:23.684648 20566 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.686122 20566 tablet_bootstrap.cc:492] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.686952 20566 ts_tablet_manager.cc:1403] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.012s
I20260709 13:22:23.690140 20566 raft_consensus.cc:359] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.690852 20566 raft_consensus.cc:740] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.691779 20566 consensus_queue.cc:260] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.693849 20566 ts_tablet_manager.cc:1434] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:23.694818 20566 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.717338 20566 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.718757 20566 tablet_bootstrap.cc:492] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.719910 20566 ts_tablet_manager.cc:1403] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.007s
I20260709 13:22:23.723855 20566 raft_consensus.cc:359] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.724880 20566 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.726069 20566 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.729059 20566 ts_tablet_manager.cc:1434] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:22:23.730428 20566 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.750447  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:41519
I20260709 13:22:23.750555 20628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:41519 every 8 connection(s)
I20260709 13:22:23.752002 20566 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.753328 20566 tablet_bootstrap.cc:492] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.754151 20566 ts_tablet_manager.cc:1403] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:22:23.757543  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:23.757894 20566 raft_consensus.cc:359] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.758919 20566 raft_consensus.cc:740] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.763983 20566 consensus_queue.cc:260] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.767071 20566 ts_tablet_manager.cc:1434] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.004s
I20260709 13:22:23.768509 20566 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.804904 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:23.805940 20368 raft_consensus.cc:606] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:23.807161 20566 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.808452 20566 tablet_bootstrap.cc:492] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.809327 20629 heartbeater.cc:344] Connected to a master server at 127.8.183.126:41951
I20260709 13:22:23.809643 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:23.809937 20629 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:23.809401 20566 ts_tablet_manager.cc:1403] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.010s
I20260709 13:22:23.810417 20368 raft_consensus.cc:606] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:23.811204 20629 heartbeater.cc:507] Master 127.8.183.126:41951 requested a full tablet report, sending...
I20260709 13:22:23.812362  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:23.813203  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:23.813962  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:23.813835 20566 raft_consensus.cc:359] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.814908 20566 raft_consensus.cc:740] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.815856 20566 consensus_queue.cc:260] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.818398 20566 ts_tablet_manager.cc:1434] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:23.819550 20566 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.821242 20120 ts_manager.cc:194] Re-registered known tserver with Master: c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519)
W20260709 13:22:23.828266 20262 consensus_peers.cc:597] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:23.836812 20120 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35846
I20260709 13:22:23.842083 20629 heartbeater.cc:499] Master 127.8.183.126:41951 was elected leader, sending a full tablet report...
I20260709 13:22:23.842684 20603 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:23.844858 20566 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.845909 20566 tablet_bootstrap.cc:492] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.846611 20566 ts_tablet_manager.cc:1403] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.007s
I20260709 13:22:23.849820 20566 raft_consensus.cc:359] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.850994 20566 raft_consensus.cc:740] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.852178 20566 consensus_queue.cc:260] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.855875 20566 ts_tablet_manager.cc:1434] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 13:22:23.857069 20603 pending_rounds.cc:85] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Aborting all ops after (but not including) 0
I20260709 13:22:23.858639 20603 pending_rounds.cc:107] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:22:23.858712 20550 raft_consensus.cc:993] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:23.857535 20566 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.859283 20603 raft_consensus.cc:2889] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:23.859681 20550 raft_consensus.cc:1081] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:23.864809 20297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:23.865495 20297 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:23.865872 20297 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:23.877600 20297 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.884240 20602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:23.884388 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:23.885095 20602 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:23.885222 20368 raft_consensus.cc:3055] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:23.885519 20566 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.885774 20368 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:23.886839 20368 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.887827 20566 tablet_bootstrap.cc:492] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.888240 20368 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:23.888880 20566 ts_tablet_manager.cc:1403] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.032s	user 0.022s	sys 0.004s
I20260709 13:22:23.891987 20566 raft_consensus.cc:359] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.892807 20566 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.893712 20566 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.895267 20297 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:23.896123 20602 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 2.
I20260709 13:22:23.896678 20566 ts_tablet_manager.cc:1434] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.001s
I20260709 13:22:23.898387 20262 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:23.898615 20566 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
I20260709 13:22:23.899495 20464 raft_consensus.cc:2804] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:23.902621 20368 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 2.
I20260709 13:22:23.905521 20464 raft_consensus.cc:697] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:23.907603 20464 consensus_queue.cc:237] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.919363 20120 catalog_manager.cc:5697] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 1 to 2, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 2 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:23.920351 20566 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.921334 20566 tablet_bootstrap.cc:492] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.922039 20566 ts_tablet_manager.cc:1403] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:22:23.924481 20566 raft_consensus.cc:359] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:23.925099 20566 raft_consensus.cc:740] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.925779 20566 consensus_queue.cc:260] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } } peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } }
I20260709 13:22:23.928081 20566 ts_tablet_manager.cc:1434] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:22:23.929311 20566 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap starting.
W20260709 13:22:23.947714 20262 consensus_peers.cc:597] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:23.951648 20566 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:23.952719 20566 tablet_bootstrap.cc:492] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Bootstrap complete.
I20260709 13:22:23.953435 20566 ts_tablet_manager.cc:1403] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 13:22:23.955818 20566 raft_consensus.cc:359] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.956458 20566 raft_consensus.cc:740] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c653c4d65e9a4850abfe3c2c9dff4d26, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.957120 20566 consensus_queue.cc:260] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:23.958781 20566 ts_tablet_manager.cc:1434] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:24.071568 20550 raft_consensus.cc:993] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:24.072281 20638 raft_consensus.cc:1081] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:24.074376 20297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:24.075304 20297 raft_consensus.cc:493] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:24.075803 20297 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:24.084097 20297 raft_consensus.cc:515] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:24.086342 20297 leader_election.cc:290] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:24.087436 20602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:24.088060 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:24.088347 20602 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:24.088881 20368 raft_consensus.cc:3055] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:24.089339 20368 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:24.090266 20368 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:24.091627 20368 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:24.097350 20602 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 2.
I20260709 13:22:24.098935 20262 leader_election.cc:304] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:24.100029 20464 raft_consensus.cc:2804] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:24.100615 20464 raft_consensus.cc:697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:24.101078 20368 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 2.
I20260709 13:22:24.101634 20464 consensus_queue.cc:237] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:24.111676 20122 catalog_manager.cc:5697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 1 to 2, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 2 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:24.234782 20602 raft_consensus.cc:3060] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:24.298646 20603 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:24.317674 20603 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:24.319399 20464 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:24.334826 20368 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:24.336930 20464 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:22:24.631441 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:24.649417 20603 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:24.651361 20464 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:24.661203 20368 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:24.663909 20464 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:24.828014  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:24.850637 20297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
mode: GRACEFUL
new_leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:34180
I20260709 13:22:24.851547 20297 raft_consensus.cc:606] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Received request to transfer leadership to TS d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:24.855276 20297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
mode: GRACEFUL
new_leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:34180
I20260709 13:22:24.856123 20297 raft_consensus.cc:606] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Received request to transfer leadership to TS d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:24.857823  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:24.858505  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:24.859355  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:25.099504 20643 raft_consensus.cc:993] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: : Instructing follower d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:25.100183 20464 raft_consensus.cc:1081] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Signalling peer d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:25.102156 20368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:22:25.102939 20368 raft_consensus.cc:493] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:25.103444 20368 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.110895 20368 raft_consensus.cc:515] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.112818 20368 leader_election.cc:290] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:25.114144 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:25.114176 20602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d69a876a07439493c3b99d2d4733c1" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:25.114969 20297 raft_consensus.cc:3055] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:25.115170 20602 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.115569 20297 raft_consensus.cc:740] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:25.116573 20297 consensus_queue.cc:260] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.118002 20297 raft_consensus.cc:3060] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.123500 20602 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 2.
I20260709 13:22:25.125309 20336 leader_election.cc:304] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:25.125658 20297 raft_consensus.cc:2468] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 2.
I20260709 13:22:25.126412 20638 raft_consensus.cc:2804] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:25.127067 20638 raft_consensus.cc:697] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:25.128139 20638 consensus_queue.cc:237] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.137387 20122 catalog_manager.cc:5697] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 1 to 2, leader changed from 011628ba1ff8482686fdab14ac8a56db (127.8.183.66) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:25.151317 20655 raft_consensus.cc:993] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: : Instructing follower d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:25.152062 20464 raft_consensus.cc:1081] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Signalling peer d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:25.153863 20368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:22:25.154543 20368 raft_consensus.cc:493] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:25.155010 20368 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.162935 20368 raft_consensus.cc:515] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.165123 20368 leader_election.cc:290] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 3 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:25.166412 20602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:25.166412 20297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:25.167294 20602 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.167320 20297 raft_consensus.cc:3055] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:25.167906 20297 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:25.168830 20297 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.170248 20297 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:25.177117 20602 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 3.
I20260709 13:22:25.179083 20336 leader_election.cc:304] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:25.179749 20297 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 3.
I20260709 13:22:25.180066 20550 raft_consensus.cc:2804] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:25.180645 20550 raft_consensus.cc:697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:25.181587 20550 consensus_queue.cc:237] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:25.190397 20120 catalog_manager.cc:5697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 2 to 3, leader changed from 011628ba1ff8482686fdab14ac8a56db (127.8.183.66) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:25.632766 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:25.651546 20297 raft_consensus.cc:1275] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:25.653401 20638 consensus_queue.cc:1048] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:25.666402 20602 raft_consensus.cc:1275] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:25.669024 20638 consensus_queue.cc:1048] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:25.712231 20603 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:25.713969 20550 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:25.724777 20297 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:25.727146 20550 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:25.867067  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:25.876057 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:25.876747 20368 raft_consensus.cc:606] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:25.880437 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:25.881139 20368 raft_consensus.cc:606] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:25.882539  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:25.883262  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:25.883985  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:26.040755 20550 raft_consensus.cc:993] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:26.041400 20550 raft_consensus.cc:1081] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:26.043680 20297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:26.044361 20297 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:26.044670 20297 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.051990 20297 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.053850 20297 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 3 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:26.054955 20603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:26.055158 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:26.055882 20368 raft_consensus.cc:3055] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:26.055855 20603 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.056516 20368 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:26.057530 20368 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.058702 20368 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.065448 20603 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 3.
I20260709 13:22:26.065455 20368 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 3.
I20260709 13:22:26.067178 20262 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:26.068092 20672 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:26.068706 20672 raft_consensus.cc:697] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:26.069612 20672 consensus_queue.cc:237] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.078387 20121 catalog_manager.cc:5697] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 2 to 3, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 3 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:26.278862 20550 raft_consensus.cc:993] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:26.279644 20550 raft_consensus.cc:1081] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:26.281638 20299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:26.282398 20299 raft_consensus.cc:493] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:26.282815 20299 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.289980 20299 raft_consensus.cc:515] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.292136 20299 leader_election.cc:290] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 3 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:26.293387 20603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:26.293375 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7ebf561fbbb487daa09f2db1eb9a685" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:26.294519 20603 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.294703 20368 raft_consensus.cc:3055] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:26.295298 20368 raft_consensus.cc:740] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:26.296185 20368 consensus_queue.cc:260] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.297299 20368 raft_consensus.cc:3060] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:26.302570 20603 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 3.
I20260709 13:22:26.304342 20262 leader_election.cc:304] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:26.304858 20368 raft_consensus.cc:2468] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 3.
I20260709 13:22:26.305428 20672 raft_consensus.cc:2804] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:26.306870 20672 raft_consensus.cc:697] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:26.308046 20672 consensus_queue.cc:237] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:26.317209 20121 catalog_manager.cc:5697] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 2 to 3, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 3 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:26.509963 20368 raft_consensus.cc:1275] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.512230 20672 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:26.537564 20602 raft_consensus.cc:1275] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:26.540975 20672 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:22:26.634287 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:26.701467 20602 raft_consensus.cc:1275] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:26.703449 20464 consensus_queue.cc:1048] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:26.719571 20368 raft_consensus.cc:1275] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:26.721953 20685 consensus_queue.cc:1048] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:26.891999  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:26.904062 20299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
mode: GRACEFUL
new_leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:34180
I20260709 13:22:26.904965 20299 raft_consensus.cc:606] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Received request to transfer leadership to TS d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:26.909469 20299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
mode: GRACEFUL
new_leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:34180
I20260709 13:22:26.910211 20299 raft_consensus.cc:606] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Received request to transfer leadership to TS d8b8188a96bc4327a85238f8d21b4e20
I20260709 13:22:26.911751  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:26.912446  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:26.913545  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:27.087214 20691 raft_consensus.cc:993] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: : Instructing follower d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:27.088153 20685 raft_consensus.cc:1081] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Signalling peer d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:27.090430 20368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:22:27.091224 20368 raft_consensus.cc:493] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:27.091596 20368 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:27.100497 20368 raft_consensus.cc:515] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.102633 20368 leader_election.cc:290] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 4 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:27.104244 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:27.104456 20602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bf353b7e1a4b0f98275f4f6326838d" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:27.105363 20299 raft_consensus.cc:3055] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:27.105824 20299 raft_consensus.cc:740] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:27.106241 20602 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:27.107334 20299 consensus_queue.cc:260] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.108709 20299 raft_consensus.cc:3060] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:27.117957 20602 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 4.
I20260709 13:22:27.119283 20603 raft_consensus.cc:1240] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Rejecting Update request from peer 011628ba1ff8482686fdab14ac8a56db for earlier term 3. Current term is 4. Ops: []
I20260709 13:22:27.119410 20299 raft_consensus.cc:2468] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 4.
I20260709 13:22:27.119735 20336 leader_election.cc:304] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:27.121233 20684 raft_consensus.cc:2804] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:27.121850 20684 raft_consensus.cc:697] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 4 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:27.123588 20684 consensus_queue.cc:237] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.133184 20121 catalog_manager.cc:5697] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 3 to 4, leader changed from 011628ba1ff8482686fdab14ac8a56db (127.8.183.66) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 4 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.250383 20691 raft_consensus.cc:993] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: : Instructing follower d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:27.251096 20685 raft_consensus.cc:1081] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Signalling peer d8b8188a96bc4327a85238f8d21b4e20 to start an election
I20260709 13:22:27.253304 20368 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
 from {username='slave'} at 127.0.0.1:57186
I20260709 13:22:27.253981 20368 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:27.254333 20368 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.261373 20368 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.263222 20368 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 3 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687)
I20260709 13:22:27.264353 20299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
I20260709 13:22:27.264353 20603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "d8b8188a96bc4327a85238f8d21b4e20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:27.265405 20299 raft_consensus.cc:3055] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:27.265509 20603 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.265929 20299 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:27.267396 20299 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.269351 20299 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:27.277936 20603 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 3.
I20260709 13:22:27.279604 20299 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8b8188a96bc4327a85238f8d21b4e20 in term 3.
I20260709 13:22:27.280099 20336 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c653c4d65e9a4850abfe3c2c9dff4d26, d8b8188a96bc4327a85238f8d21b4e20; no voters: 
I20260709 13:22:27.281410 20683 raft_consensus.cc:2804] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:27.283417 20683 raft_consensus.cc:697] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Becoming Leader. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:27.284744 20683 consensus_queue.cc:237] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:27.295224 20121 catalog_manager.cc:5697] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 reported cstate change: term changed from 2 to 3, leader changed from 011628ba1ff8482686fdab14ac8a56db (127.8.183.66) to d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "d8b8188a96bc4327a85238f8d21b4e20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:27.507427 20603 raft_consensus.cc:1275] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:27.509346 20683 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:27.521044 20299 raft_consensus.cc:1275] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:27.523175 20683 consensus_queue.cc:1048] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:22:27.635385 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:27.794822 20299 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:27.796682 20683 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:27.809916 20603 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Refusing update from remote peer d8b8188a96bc4327a85238f8d21b4e20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:27.812701 20683 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:27.922497  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:27.932137 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:27.932840 20368 raft_consensus.cc:606] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:27.936543 20368 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
mode: GRACEFUL
new_leader_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:57986
I20260709 13:22:27.937304 20368 raft_consensus.cc:606] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Received request to transfer leadership to TS 011628ba1ff8482686fdab14ac8a56db
I20260709 13:22:27.938647  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:27.939430  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:27.940189  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:28.359325 20684 raft_consensus.cc:993] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:28.359935 20683 raft_consensus.cc:1081] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:28.361776 20299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:28.362325 20299 raft_consensus.cc:493] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.362619 20299 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.369690 20299 raft_consensus.cc:515] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.372126 20299 leader_election.cc:290] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 4 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:28.373245 20603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:28.373528 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bcca7ef5c974670bd13d8d33e23e3c0" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:28.374174 20603 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.374277 20368 raft_consensus.cc:3055] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:28.374738 20368 raft_consensus.cc:740] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:28.375762 20368 consensus_queue.cc:260] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.376967 20368 raft_consensus.cc:3060] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.382884 20603 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 4.
I20260709 13:22:28.384693 20368 raft_consensus.cc:2468] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 4.
I20260709 13:22:28.384914 20262 leader_election.cc:304] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:28.386052 20685 raft_consensus.cc:2804] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:28.387270 20684 raft_consensus.cc:993] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: : Instructing follower 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:28.387596 20685 raft_consensus.cc:697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 4 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:28.387949 20683 raft_consensus.cc:1081] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Signalling peer 011628ba1ff8482686fdab14ac8a56db to start an election
I20260709 13:22:28.389732 20685 consensus_queue.cc:237] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.390226 20299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed"
dest_uuid: "011628ba1ff8482686fdab14ac8a56db"
 from {username='slave'} at 127.0.0.1:34984
I20260709 13:22:28.391047 20299 raft_consensus.cc:493] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:28.391592 20299 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.399631 20121 catalog_manager.cc:5697] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 3 to 4, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 4 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.401511 20299 raft_consensus.cc:515] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.404610 20299 leader_election.cc:290] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 4 election: Requested vote from peers c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519), d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67:43763)
I20260709 13:22:28.405642 20603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26"
I20260709 13:22:28.405928 20368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db01cf8910de4d24a3140ef547b492ed" candidate_uuid: "011628ba1ff8482686fdab14ac8a56db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d8b8188a96bc4327a85238f8d21b4e20"
I20260709 13:22:28.406602 20603 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.406752 20368 raft_consensus.cc:3055] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:28.407308 20368 raft_consensus.cc:740] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d8b8188a96bc4327a85238f8d21b4e20, State: Running, Role: LEADER
I20260709 13:22:28.408128 20368 consensus_queue.cc:260] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.409511 20368 raft_consensus.cc:3060] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.417650 20603 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 4.
I20260709 13:22:28.419530 20368 raft_consensus.cc:2468] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011628ba1ff8482686fdab14ac8a56db in term 4.
I20260709 13:22:28.419550 20262 leader_election.cc:304] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011628ba1ff8482686fdab14ac8a56db, c653c4d65e9a4850abfe3c2c9dff4d26; no voters: 
I20260709 13:22:28.421030 20685 raft_consensus.cc:2804] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:28.421658 20685 raft_consensus.cc:697] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 4 LEADER]: Becoming Leader. State: Replica: 011628ba1ff8482686fdab14ac8a56db, State: Running, Role: LEADER
I20260709 13:22:28.422493 20685 consensus_queue.cc:237] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } }
I20260709 13:22:28.430723 20121 catalog_manager.cc:5697] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db reported cstate change: term changed from 3 to 4, leader changed from d8b8188a96bc4327a85238f8d21b4e20 (127.8.183.67) to 011628ba1ff8482686fdab14ac8a56db (127.8.183.66). New cstate: current_term: 4 leader_uuid: "011628ba1ff8482686fdab14ac8a56db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011628ba1ff8482686fdab14ac8a56db" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:28.636580 20170 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:28.901475 20603 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:28.903693 20709 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:28.919194 20368 raft_consensus.cc:1275] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:28.920956 20709 consensus_queue.cc:1048] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:22:28.949733  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:28.976593 20603 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:28.976727 20368 raft_consensus.cc:1275] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Refusing update from remote peer 011628ba1ff8482686fdab14ac8a56db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:28.978648 20685 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c653c4d65e9a4850abfe3c2c9dff4d26" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:22:28.979655 20718 consensus_queue.cc:1048] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b8188a96bc4327a85238f8d21b4e20" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:28.996393  8925 tablet_server.cc:179] TabletServer@127.8.183.65:41519 shutting down...
W20260709 13:22:29.006093 20262 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 13:22:29.022891 20262 consensus_peers.cc:597] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.042819 20336 consensus_peers.cc:597] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.044080 20336 consensus_peers.cc:597] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.053424 20262 consensus_peers.cc:597] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.069185  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:29.070359  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.071398  8925 raft_consensus.cc:2243] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.072019  8925 raft_consensus.cc:2272] T 38f6496480a24495a98022256578d05f P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.075013  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.075935  8925 raft_consensus.cc:2243] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.076532  8925 raft_consensus.cc:2272] T 97bf353b7e1a4b0f98275f4f6326838d P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.079685  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.080765  8925 raft_consensus.cc:2243] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.081342  8925 raft_consensus.cc:2272] T 1a3fe03229a74edc95ba4c0074f0ce7c P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.085121  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.086277  8925 raft_consensus.cc:2243] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.087002  8925 raft_consensus.cc:2272] T 77d69a876a07439493c3b99d2d4733c1 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.090045  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.091209  8925 raft_consensus.cc:2243] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.091958  8925 raft_consensus.cc:2272] T 8bcca7ef5c974670bd13d8d33e23e3c0 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.096015  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.096953  8925 raft_consensus.cc:2243] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.097510  8925 raft_consensus.cc:2272] T 579315f866164c9e850c8c9eaaada965 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.100570  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.101349  8925 raft_consensus.cc:2243] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.102109  8925 raft_consensus.cc:2272] T 7b55fc1e5174468da1f8a7858bc90d77 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.105489  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.106185  8925 raft_consensus.cc:2243] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.106748  8925 raft_consensus.cc:2272] T 0b69c46b5e834f90be7fa65c6dee4aa3 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.110445  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.111258  8925 raft_consensus.cc:2243] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.111816  8925 pending_rounds.cc:70] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Trying to abort 1 pending ops.
I20260709 13:22:29.112201  8925 pending_rounds.cc:77] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305639315147534336 op_type: NO_OP noop_request { }
I20260709 13:22:29.112707  8925 raft_consensus.cc:2889] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:29.113077  8925 raft_consensus.cc:2272] T db01cf8910de4d24a3140ef547b492ed P c653c4d65e9a4850abfe3c2c9dff4d26 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.117286  8925 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:29.118165  8925 raft_consensus.cc:2243] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.118814  8925 raft_consensus.cc:2272] T e7ebf561fbbb487daa09f2db1eb9a685 P c653c4d65e9a4850abfe3c2c9dff4d26 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:29.149379 20262 consensus_peers.cc:597] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.157971  8925 tablet_server.cc:196] TabletServer@127.8.183.65:41519 shutdown complete.
I20260709 13:22:29.193871  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
W20260709 13:22:29.221228 20262 consensus_peers.cc:597] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.238207  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:29.239831  8925 tablet_replica.cc:333] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.241540  8925 raft_consensus.cc:2243] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.243038  8925 raft_consensus.cc:2272] T e7ebf561fbbb487daa09f2db1eb9a685 P 011628ba1ff8482686fdab14ac8a56db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.246126  8925 tablet_replica.cc:333] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.246841  8925 raft_consensus.cc:2243] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.247377  8925 raft_consensus.cc:2272] T 38f6496480a24495a98022256578d05f P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.250171  8925 tablet_replica.cc:333] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.250814  8925 raft_consensus.cc:2243] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.251345  8925 raft_consensus.cc:2272] T 97bf353b7e1a4b0f98275f4f6326838d P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.253840  8925 tablet_replica.cc:333] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.254390  8925 raft_consensus.cc:2243] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.255384  8925 raft_consensus.cc:2272] T 8bcca7ef5c974670bd13d8d33e23e3c0 P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.258708  8925 tablet_replica.cc:333] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.259505  8925 raft_consensus.cc:2243] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.260042  8925 raft_consensus.cc:2272] T 77d69a876a07439493c3b99d2d4733c1 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.262621  8925 tablet_replica.cc:333] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.263314  8925 raft_consensus.cc:2243] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.264400  8925 raft_consensus.cc:2272] T 579315f866164c9e850c8c9eaaada965 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.267499  8925 tablet_replica.cc:333] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.268292  8925 raft_consensus.cc:2243] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.269029  8925 raft_consensus.cc:2272] T 1a3fe03229a74edc95ba4c0074f0ce7c P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:29.272713 20338 consensus_peers.cc:597] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 -> Peer 011628ba1ff8482686fdab14ac8a56db (127.8.183.66:46687): Couldn't send request to peer 011628ba1ff8482686fdab14ac8a56db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.273059  8925 tablet_replica.cc:333] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.273864  8925 raft_consensus.cc:2243] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.275002  8925 raft_consensus.cc:2272] T 7b55fc1e5174468da1f8a7858bc90d77 P 011628ba1ff8482686fdab14ac8a56db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.277942  8925 tablet_replica.cc:333] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.278522  8925 raft_consensus.cc:2243] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.279579  8925 raft_consensus.cc:2272] T db01cf8910de4d24a3140ef547b492ed P 011628ba1ff8482686fdab14ac8a56db [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.283104  8925 tablet_replica.cc:333] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db: stopping tablet replica
I20260709 13:22:29.283758  8925 raft_consensus.cc:2243] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.284812  8925 raft_consensus.cc:2272] T 0b69c46b5e834f90be7fa65c6dee4aa3 P 011628ba1ff8482686fdab14ac8a56db [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:29.301124 20336 consensus_peers.cc:597] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.329546  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
W20260709 13:22:29.339946 20336 consensus_peers.cc:597] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 -> Peer c653c4d65e9a4850abfe3c2c9dff4d26 (127.8.183.65:41519): Couldn't send request to peer c653c4d65e9a4850abfe3c2c9dff4d26. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:41519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.366564  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
I20260709 13:22:29.405540  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:29.406836  8925 tablet_replica.cc:333] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.408123  8925 raft_consensus.cc:2243] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.408777  8925 raft_consensus.cc:2272] T e7ebf561fbbb487daa09f2db1eb9a685 P d8b8188a96bc4327a85238f8d21b4e20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.412467  8925 tablet_replica.cc:333] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.413375  8925 raft_consensus.cc:2243] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.414194  8925 raft_consensus.cc:2272] T 7b55fc1e5174468da1f8a7858bc90d77 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.416956  8925 tablet_replica.cc:333] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.417673  8925 raft_consensus.cc:2243] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.418243  8925 raft_consensus.cc:2272] T db01cf8910de4d24a3140ef547b492ed P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.421943  8925 tablet_replica.cc:333] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.422721  8925 raft_consensus.cc:2243] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.423461  8925 raft_consensus.cc:2272] T 0b69c46b5e834f90be7fa65c6dee4aa3 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.425992  8925 tablet_replica.cc:333] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.426671  8925 raft_consensus.cc:2243] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.427703  8925 raft_consensus.cc:2272] T 97bf353b7e1a4b0f98275f4f6326838d P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.430296  8925 tablet_replica.cc:333] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.430953  8925 raft_consensus.cc:2243] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.432039  8925 raft_consensus.cc:2272] T 38f6496480a24495a98022256578d05f P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.434620  8925 tablet_replica.cc:333] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.435425  8925 raft_consensus.cc:2243] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.436539  8925 raft_consensus.cc:2272] T 1a3fe03229a74edc95ba4c0074f0ce7c P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.439283  8925 tablet_replica.cc:333] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.440090  8925 raft_consensus.cc:2243] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.440667  8925 raft_consensus.cc:2272] T 8bcca7ef5c974670bd13d8d33e23e3c0 P d8b8188a96bc4327a85238f8d21b4e20 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.443907  8925 tablet_replica.cc:333] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.444612  8925 raft_consensus.cc:2243] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:29.445283  8925 raft_consensus.cc:2272] T 579315f866164c9e850c8c9eaaada965 P d8b8188a96bc4327a85238f8d21b4e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.448246  8925 tablet_replica.cc:333] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20: stopping tablet replica
I20260709 13:22:29.448886  8925 raft_consensus.cc:2243] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.450152  8925 raft_consensus.cc:2272] T 77d69a876a07439493c3b99d2d4733c1 P d8b8188a96bc4327a85238f8d21b4e20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.514689  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:22:29.551023  8925 master.cc:562] Master@127.8.183.126:41951 shutting down...
I20260709 13:22:29.593580  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:29.594420  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:29.594911  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P 99a427d28fdd433b9197e12e4b98a318: stopping tablet replica
I20260709 13:22:29.618501  8925 master.cc:584] Master@127.8.183.126:41951 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14909 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:22:29.677445  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:39343
I20260709 13:22:29.678745  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:29.688638 20734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:29.688678 20732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:29.689311  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:29.688942 20731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:29.691035  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:29.691408  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:29.691536  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603349691523 us; error 0 us; skew 500 ppm
I20260709 13:22:29.692179  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:29.695690  8925 webserver.cc:533] Webserver started at http://127.8.183.126:42679/ using document root <none> and password file <none>
I20260709 13:22:29.696367  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:29.696565  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:29.696957  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:29.698385  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "57378e13f67f45dbb7a500ad0221d04c"
format_stamp: "Formatted at 2026-07-09 13:22:29 on dist-test-slave-ch3x"
I20260709 13:22:29.703595  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:22:29.708240 20739 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:29.709259  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:22:29.709744  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "57378e13f67f45dbb7a500ad0221d04c"
format_stamp: "Formatted at 2026-07-09 13:22:29 on dist-test-slave-ch3x"
I20260709 13:22:29.710122  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:29.725627  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:29.727053  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:29.774914  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:39343
I20260709 13:22:29.775036 20790 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:39343 every 8 connection(s)
I20260709 13:22:29.780289 20791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:29.795599 20791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c: Bootstrap starting.
I20260709 13:22:29.801863 20791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:29.808264 20791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c: No bootstrap required, opened a new log
I20260709 13:22:29.811496 20791 raft_consensus.cc:359] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER }
I20260709 13:22:29.812247 20791 raft_consensus.cc:385] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:29.812593 20791 raft_consensus.cc:740] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57378e13f67f45dbb7a500ad0221d04c, State: Initialized, Role: FOLLOWER
I20260709 13:22:29.813428 20791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER }
I20260709 13:22:29.814041 20791 raft_consensus.cc:399] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:29.814329 20791 raft_consensus.cc:493] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:29.814671 20791 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:29.822089 20791 raft_consensus.cc:515] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER }
I20260709 13:22:29.822968 20791 leader_election.cc:304] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57378e13f67f45dbb7a500ad0221d04c; no voters: 
I20260709 13:22:29.824710 20791 leader_election.cc:290] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:29.825276 20794 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:29.827337 20794 raft_consensus.cc:697] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 1 LEADER]: Becoming Leader. State: Replica: 57378e13f67f45dbb7a500ad0221d04c, State: Running, Role: LEADER
I20260709 13:22:29.828241 20791 sys_catalog.cc:565] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:29.828181 20794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER }
I20260709 13:22:29.833966 20796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57378e13f67f45dbb7a500ad0221d04c. Latest consensus state: current_term: 1 leader_uuid: "57378e13f67f45dbb7a500ad0221d04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER } }
I20260709 13:22:29.834710 20796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:29.833839 20795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57378e13f67f45dbb7a500ad0221d04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57378e13f67f45dbb7a500ad0221d04c" member_type: VOTER } }
I20260709 13:22:29.835731 20795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:29.840214 20800 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:29.850636 20800 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:29.852421  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:29.862866 20800 catalog_manager.cc:1383] Generated new cluster ID: 40f00f5d774e4834a40a65f99bc1b17e
I20260709 13:22:29.863394 20800 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:29.906526 20800 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:29.908212 20800 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:29.926577 20800 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c: Generated new TSK 0
I20260709 13:22:29.927618 20800 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:29.984840  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:29.992676 20812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:29.994318 20813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:29.996846 20815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:29.996954  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:29.998618  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:29.998920  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:29.999063  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603349999050 us; error 0 us; skew 500 ppm
I20260709 13:22:29.999790  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:30.002803  8925 webserver.cc:533] Webserver started at http://127.8.183.65:44989/ using document root <none> and password file <none>
I20260709 13:22:30.003516  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:30.003721  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:30.003978  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:30.005388  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "90fa747f7aa94285a794108cb19c210c"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.011047  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:22:30.016098 20820 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.017225  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:22:30.017609  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "90fa747f7aa94285a794108cb19c210c"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.017947  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:30.034530  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:30.036052  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:30.037899  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:30.041141  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:30.041388  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.041728  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:30.041967  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.097482  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:35863
I20260709 13:22:30.097581 20882 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:35863 every 8 connection(s)
I20260709 13:22:30.104601  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:30.113714 20887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:30.114363 20888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:30.121601  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:30.121307 20890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:30.123800 20883 heartbeater.cc:344] Connected to a master server at 127.8.183.126:39343
I20260709 13:22:30.124881 20883 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:30.125998  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:30.126101 20883 heartbeater.cc:507] Master 127.8.183.126:39343 requested a full tablet report, sending...
W20260709 13:22:30.126505  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:30.126812  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603350126794 us; error 0 us; skew 500 ppm
I20260709 13:22:30.127727  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:30.129400 20756 ts_manager.cc:194] Registered new tserver with Master: 90fa747f7aa94285a794108cb19c210c (127.8.183.65:35863)
I20260709 13:22:30.131029  8925 webserver.cc:533] Webserver started at http://127.8.183.66:43041/ using document root <none> and password file <none>
I20260709 13:22:30.131778  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:30.132004  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:30.132073 20756 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50854
I20260709 13:22:30.132328  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:30.134187  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "b277a3d9f11341a498aaef20eac45359"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.140172  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.002s
I20260709 13:22:30.144996 20895 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.146104  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:30.146476  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "b277a3d9f11341a498aaef20eac45359"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.146775  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:30.160938  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:30.162369  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:30.164418  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:30.167690  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:30.167938  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.168313  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:30.168467  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.221341  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:36241
I20260709 13:22:30.221498 20957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:36241 every 8 connection(s)
I20260709 13:22:30.227231  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:30.236517 20961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:30.237519 20962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:30.241971 20964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:30.242012  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:30.243880  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:30.244278  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:30.244361 20958 heartbeater.cc:344] Connected to a master server at 127.8.183.126:39343
I20260709 13:22:30.244535  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603350244484 us; error 0 us; skew 500 ppm
I20260709 13:22:30.245174 20958 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:30.245703  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:30.246446 20958 heartbeater.cc:507] Master 127.8.183.126:39343 requested a full tablet report, sending...
I20260709 13:22:30.250311  8925 webserver.cc:533] Webserver started at http://127.8.183.67:43113/ using document root <none> and password file <none>
I20260709 13:22:30.250300 20756 ts_manager.cc:194] Registered new tserver with Master: b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241)
I20260709 13:22:30.251345  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:30.251680  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:30.252024  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:30.252911 20756 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50866
I20260709 13:22:30.253520  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "dcb445edccb446048f8e07d9dddc87a2"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.259872  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 13:22:30.264791 20969 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.266062  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:22:30.266569  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "dcb445edccb446048f8e07d9dddc87a2"
format_stamp: "Formatted at 2026-07-09 13:22:30 on dist-test-slave-ch3x"
I20260709 13:22:30.266947  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:30.286157  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:30.287712  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:30.289507  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:30.292503  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:30.292855  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.293159  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:30.293377  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:30.347994  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:43509
I20260709 13:22:30.348192 21031 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:43509 every 8 connection(s)
I20260709 13:22:30.369302 21032 heartbeater.cc:344] Connected to a master server at 127.8.183.126:39343
I20260709 13:22:30.369968 21032 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:30.370957 21032 heartbeater.cc:507] Master 127.8.183.126:39343 requested a full tablet report, sending...
I20260709 13:22:30.373893 20756 ts_manager.cc:194] Registered new tserver with Master: dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:30.374655  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.022466383s
I20260709 13:22:30.375891 20756 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50876
I20260709 13:22:30.407250 20756 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50882:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:30.409785 20756 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:30.483168 20848 tablet_service.cc:1511] Processing CreateTablet for tablet 153cca36fdad4e8586941ed88b22e912 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.483414 20847 tablet_service.cc:1511] Processing CreateTablet for tablet 6829cf8d2aca49aa9d02e75b120315c1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.485023 20848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153cca36fdad4e8586941ed88b22e912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.485793 20847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829cf8d2aca49aa9d02e75b120315c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.496578 20922 tablet_service.cc:1511] Processing CreateTablet for tablet 6829cf8d2aca49aa9d02e75b120315c1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.498435 20922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829cf8d2aca49aa9d02e75b120315c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.494243 20923 tablet_service.cc:1511] Processing CreateTablet for tablet 153cca36fdad4e8586941ed88b22e912 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.504524 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153cca36fdad4e8586941ed88b22e912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.506174 20997 tablet_service.cc:1511] Processing CreateTablet for tablet 153cca36fdad4e8586941ed88b22e912 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.508132 20997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 153cca36fdad4e8586941ed88b22e912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.513266 20996 tablet_service.cc:1511] Processing CreateTablet for tablet 6829cf8d2aca49aa9d02e75b120315c1 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=4c7a27918da94907b13976206d52dbbe]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.515138 20996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829cf8d2aca49aa9d02e75b120315c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.532105 21048 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: Bootstrap starting.
I20260709 13:22:30.542788 21048 tablet_bootstrap.cc:654] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.557639 21048 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: No bootstrap required, opened a new log
I20260709 13:22:30.558333 21048 ts_tablet_manager.cc:1403] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.007s
I20260709 13:22:30.562156 21048 raft_consensus.cc:359] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.563308 21048 raft_consensus.cc:385] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.563696 21048 raft_consensus.cc:740] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.564743 21048 consensus_queue.cc:260] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.568883 21047 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: Bootstrap starting.
I20260709 13:22:30.571856 20958 heartbeater.cc:499] Master 127.8.183.126:39343 was elected leader, sending a full tablet report...
I20260709 13:22:30.573006 21048 ts_tablet_manager.cc:1434] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.005s
I20260709 13:22:30.577435 21051 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: Bootstrap starting.
I20260709 13:22:30.580412 21048 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: Bootstrap starting.
I20260709 13:22:30.580760 21047 tablet_bootstrap.cc:654] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.589855 21051 tablet_bootstrap.cc:654] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.593140 21048 tablet_bootstrap.cc:654] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.602265 21047 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: No bootstrap required, opened a new log
I20260709 13:22:30.602350 21051 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: No bootstrap required, opened a new log
I20260709 13:22:30.602741 21048 tablet_bootstrap.cc:492] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: No bootstrap required, opened a new log
I20260709 13:22:30.603106 21051 ts_tablet_manager.cc:1403] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.009s
I20260709 13:22:30.603302 21047 ts_tablet_manager.cc:1403] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.012s
I20260709 13:22:30.603430 21048 ts_tablet_manager.cc:1403] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.008s
I20260709 13:22:30.606285 21051 raft_consensus.cc:359] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.607161 21051 raft_consensus.cc:385] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.607180 21047 raft_consensus.cc:359] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.607563 21051 raft_consensus.cc:740] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb445edccb446048f8e07d9dddc87a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.607533 21048 raft_consensus.cc:359] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.608328 21047 raft_consensus.cc:385] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.608618 21048 raft_consensus.cc:385] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.608768 21047 raft_consensus.cc:740] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.609143 21048 raft_consensus.cc:740] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.609136 21051 consensus_queue.cc:260] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.609867 21047 consensus_queue.cc:260] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.610476 21048 consensus_queue.cc:260] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.612680 21032 heartbeater.cc:499] Master 127.8.183.126:39343 was elected leader, sending a full tablet report...
I20260709 13:22:30.613592 21048 ts_tablet_manager.cc:1434] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:30.613880 20883 heartbeater.cc:499] Master 127.8.183.126:39343 was elected leader, sending a full tablet report...
I20260709 13:22:30.615602 21051 ts_tablet_manager.cc:1434] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:22:30.616308 21047 ts_tablet_manager.cc:1434] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:22:30.617054 21051 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: Bootstrap starting.
I20260709 13:22:30.617412 21047 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: Bootstrap starting.
I20260709 13:22:30.624560 21047 tablet_bootstrap.cc:654] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.625825 21051 tablet_bootstrap.cc:654] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.633893 21047 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: No bootstrap required, opened a new log
I20260709 13:22:30.634688 21047 ts_tablet_manager.cc:1403] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:22:30.638425 21047 raft_consensus.cc:359] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.639395 21047 raft_consensus.cc:385] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.639777 21047 raft_consensus.cc:740] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.640136 21051 tablet_bootstrap.cc:492] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: No bootstrap required, opened a new log
I20260709 13:22:30.640959 21051 ts_tablet_manager.cc:1403] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 13:22:30.640982 21047 consensus_queue.cc:260] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.646179 21047 ts_tablet_manager.cc:1434] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:22:30.646147 21051 raft_consensus.cc:359] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.647291 21051 raft_consensus.cc:385] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.647544 21051 raft_consensus.cc:740] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb445edccb446048f8e07d9dddc87a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.648428 21051 consensus_queue.cc:260] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.650573 21051 ts_tablet_manager.cc:1434] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:22:30.678748 21056 raft_consensus.cc:493] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.679713 21056 raft_consensus.cc:515] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.682478 21056 leader_election.cc:290] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509), b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241)
I20260709 13:22:30.694945 21056 raft_consensus.cc:493] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.695629 21056 raft_consensus.cc:515] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.698212 21056 leader_election.cc:290] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:30.698668 21007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153cca36fdad4e8586941ed88b22e912" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2" is_pre_election: true
I20260709 13:22:30.699658 21007 raft_consensus.cc:2468] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:30.699962 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153cca36fdad4e8586941ed88b22e912" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359" is_pre_election: true
I20260709 13:22:30.700335 20932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359" is_pre_election: true
I20260709 13:22:30.700791 21006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2" is_pre_election: true
I20260709 13:22:30.701254 20932 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:30.701807 21006 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:30.701920 20822 leader_election.cc:304] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:30.700810 20933 raft_consensus.cc:2468] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:30.703011 20824 leader_election.cc:304] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, b277a3d9f11341a498aaef20eac45359; no voters: 
I20260709 13:22:30.703385 21056 raft_consensus.cc:2804] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:30.704224 21056 raft_consensus.cc:493] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.704664 21056 raft_consensus.cc:3060] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.705929 21061 raft_consensus.cc:2804] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:30.706583 21061 raft_consensus.cc:493] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.706990 21061 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.714771 21056 raft_consensus.cc:515] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.716723 21061 raft_consensus.cc:515] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.717474 21056 leader_election.cc:290] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Requested vote from peers dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509), b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241)
I20260709 13:22:30.718511 21006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153cca36fdad4e8586941ed88b22e912" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:30.718940 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "153cca36fdad4e8586941ed88b22e912" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359"
I20260709 13:22:30.719331 21006 raft_consensus.cc:3060] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.719516 21061 leader_election.cc:290] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Requested vote from peers b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:30.719635 20933 raft_consensus.cc:3060] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.720377 20932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359"
I20260709 13:22:30.721156 20932 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.721451 21007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:30.722236 21007 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:30.730341 21006 raft_consensus.cc:2468] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:30.731981 20932 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:30.732141 20822 leader_election.cc:304] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:30.732288 20933 raft_consensus.cc:2468] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:30.733191 21007 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:30.733429 21061 raft_consensus.cc:2804] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:30.734164 21061 raft_consensus.cc:697] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Becoming Leader. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:30.734118 20824 leader_election.cc:304] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, b277a3d9f11341a498aaef20eac45359; no voters: 
I20260709 13:22:30.735833 21056 raft_consensus.cc:2804] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:30.735688 21061 consensus_queue.cc:237] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } }
I20260709 13:22:30.737357 21056 raft_consensus.cc:697] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Becoming Leader. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:30.738448 21056 consensus_queue.cc:237] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.748260 20755 catalog_manager.cc:5697] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c reported cstate change: term changed from 0 to 1, leader changed from <none> to 90fa747f7aa94285a794108cb19c210c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "90fa747f7aa94285a794108cb19c210c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.765038 20755 catalog_manager.cc:5697] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c reported cstate change: term changed from 0 to 1, leader changed from <none> to 90fa747f7aa94285a794108cb19c210c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "90fa747f7aa94285a794108cb19c210c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.840907 20755 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50894:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:30.843390 20755 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:30.873953 20848 tablet_service.cc:1511] Processing CreateTablet for tablet 63e3ab4603b9491faf0766ddd358da34 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.875494 20923 tablet_service.cc:1511] Processing CreateTablet for tablet 63e3ab4603b9491faf0766ddd358da34 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.875569 20848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e3ab4603b9491faf0766ddd358da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.877156 20923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e3ab4603b9491faf0766ddd358da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.880160 20847 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d438765174fdf8da74b213e4d5332 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.882100 20922 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d438765174fdf8da74b213e4d5332 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.883807 20922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d438765174fdf8da74b213e4d5332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.884213 20996 tablet_service.cc:1511] Processing CreateTablet for tablet 5a9d438765174fdf8da74b213e4d5332 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:22:30.885663 20996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d438765174fdf8da74b213e4d5332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.882248 20847 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a9d438765174fdf8da74b213e4d5332. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.876940 20997 tablet_service.cc:1511] Processing CreateTablet for tablet 63e3ab4603b9491faf0766ddd358da34 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1d4f28c154004e19aab62d37fa3b00da]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:22:30.890777 20997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e3ab4603b9491faf0766ddd358da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:30.898012 21047 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: Bootstrap starting.
I20260709 13:22:30.904578 21051 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: Bootstrap starting.
I20260709 13:22:30.904577 21048 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: Bootstrap starting.
I20260709 13:22:30.905352 21047 tablet_bootstrap.cc:654] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.913970 21051 tablet_bootstrap.cc:654] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.915493 21048 tablet_bootstrap.cc:654] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.924018 21048 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: No bootstrap required, opened a new log
I20260709 13:22:30.924732 21048 ts_tablet_manager.cc:1403] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 13:22:30.931609 21051 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: No bootstrap required, opened a new log
I20260709 13:22:30.932044 21047 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: No bootstrap required, opened a new log
I20260709 13:22:30.932425 21051 ts_tablet_manager.cc:1403] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: Time spent bootstrapping tablet: real 0.028s	user 0.002s	sys 0.016s
I20260709 13:22:30.932624 21047 ts_tablet_manager.cc:1403] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.007s
I20260709 13:22:30.929225 21048 raft_consensus.cc:359] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.935678 21048 raft_consensus.cc:385] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.936136 21048 raft_consensus.cc:740] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.936404 21051 raft_consensus.cc:359] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.937337 21051 raft_consensus.cc:385] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.937693 21051 raft_consensus.cc:740] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb445edccb446048f8e07d9dddc87a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.938212 21047 raft_consensus.cc:359] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.939205 21047 raft_consensus.cc:385] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.938951 21051 consensus_queue.cc:260] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.939574 21047 raft_consensus.cc:740] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.940689 21048 consensus_queue.cc:260] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.941807 21051 ts_tablet_manager.cc:1434] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:22:30.944363 21047 consensus_queue.cc:260] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.946516 21047 ts_tablet_manager.cc:1434] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:22:30.948637 21048 ts_tablet_manager.cc:1434] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.005s
I20260709 13:22:30.950363 21048 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: Bootstrap starting.
I20260709 13:22:30.956182 21051 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Bootstrap starting.
I20260709 13:22:30.957181 21047 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: Bootstrap starting.
I20260709 13:22:30.964468 21047 tablet_bootstrap.cc:654] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.966037 21048 tablet_bootstrap.cc:654] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.968479 21051 tablet_bootstrap.cc:654] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:30.972577 21047 tablet_bootstrap.cc:492] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: No bootstrap required, opened a new log
I20260709 13:22:30.973330 21047 ts_tablet_manager.cc:1403] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:22:30.975188 21051 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: No bootstrap required, opened a new log
I20260709 13:22:30.975344 21048 tablet_bootstrap.cc:492] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: No bootstrap required, opened a new log
I20260709 13:22:30.975970 21051 ts_tablet_manager.cc:1403] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 13:22:30.976146 21048 ts_tablet_manager.cc:1403] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:22:30.976728 21047 raft_consensus.cc:359] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.977638 21047 raft_consensus.cc:385] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.977960 21047 raft_consensus.cc:740] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.979014 21047 consensus_queue.cc:260] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.979662 21051 raft_consensus.cc:359] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.981132 21051 raft_consensus.cc:385] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.981446 21051 raft_consensus.cc:740] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb445edccb446048f8e07d9dddc87a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.982388 21051 consensus_queue.cc:260] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.983064 21047 ts_tablet_manager.cc:1434] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:22:30.982378 21048 raft_consensus.cc:359] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.983750 21048 raft_consensus.cc:385] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:30.984138 21048 raft_consensus.cc:740] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.986371 21051 ts_tablet_manager.cc:1434] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:22:30.985128 21048 consensus_queue.cc:260] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:30.989017 21048 ts_tablet_manager.cc:1434] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:22:31.138614 21050 raft_consensus.cc:493] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.139467 21050 raft_consensus.cc:515] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.142129 21050 leader_election.cc:290] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90fa747f7aa94285a794108cb19c210c (127.8.183.65:35863), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.157258 21056 raft_consensus.cc:493] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.158234 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90fa747f7aa94285a794108cb19c210c" is_pre_election: true
I20260709 13:22:31.158041 21056 raft_consensus.cc:515] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.158414 21007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2" is_pre_election: true
I20260709 13:22:31.159108 20858 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 0.
I20260709 13:22:31.159312 21007 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 0.
I20260709 13:22:31.160825 21056 leader_election.cc:290] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.160734 20897 leader_election.cc:304] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b277a3d9f11341a498aaef20eac45359, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.162053 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359" is_pre_election: true
I20260709 13:22:31.162117 21050 raft_consensus.cc:2804] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.162279 21007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2" is_pre_election: true
I20260709 13:22:31.163050 21050 raft_consensus.cc:493] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.163118 20933 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:31.163367 21007 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 0.
I20260709 13:22:31.163661 21050 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.164824 20822 leader_election.cc:304] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.165666 21056 raft_consensus.cc:2804] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:31.166124 21056 raft_consensus.cc:493] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.166406 21056 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.171948 21050 raft_consensus.cc:515] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.173241 21056 raft_consensus.cc:515] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.174589 21050 leader_election.cc:290] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 1 election: Requested vote from peers 90fa747f7aa94285a794108cb19c210c (127.8.183.65:35863), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.176075 21007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:31.176187 20858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90fa747f7aa94285a794108cb19c210c"
I20260709 13:22:31.177141 21007 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.176474 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b277a3d9f11341a498aaef20eac45359"
I20260709 13:22:31.177215 21056 leader_election.cc:290] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Requested vote from peers b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.177287 20858 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.177925 20933 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.177462 21006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:31.179206 21006 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:31.187907 21007 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 1.
I20260709 13:22:31.189893 20897 leader_election.cc:304] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b277a3d9f11341a498aaef20eac45359, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.190299 21006 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:31.191038 21050 raft_consensus.cc:2804] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.190299 20858 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 1.
I20260709 13:22:31.191206 20933 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 1.
I20260709 13:22:31.192004 20822 leader_election.cc:304] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.197234 21050 raft_consensus.cc:697] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 LEADER]: Becoming Leader. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Running, Role: LEADER
I20260709 13:22:31.198836 21056 raft_consensus.cc:2804] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:31.198433 21050 consensus_queue.cc:237] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.199666 21056 raft_consensus.cc:697] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Becoming Leader. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:31.200831 21056 consensus_queue.cc:237] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.210304 20753 catalog_manager.cc:5697] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c reported cstate change: term changed from 0 to 1, leader changed from <none> to 90fa747f7aa94285a794108cb19c210c (127.8.183.65). New cstate: current_term: 1 leader_uuid: "90fa747f7aa94285a794108cb19c210c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.214214 20754 catalog_manager.cc:5697] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 reported cstate change: term changed from 0 to 1, leader changed from <none> to b277a3d9f11341a498aaef20eac45359 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "b277a3d9f11341a498aaef20eac45359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.217058 21056 consensus_queue.cc:1048] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.239533 21061 consensus_queue.cc:1048] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:31.269659 20858 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1"
dest_uuid: "90fa747f7aa94285a794108cb19c210c"
mode: GRACEFUL
new_leader_uuid: "b277a3d9f11341a498aaef20eac45359"
 from {username='slave'} at 127.0.0.1:49904
I20260709 13:22:31.270383 20858 raft_consensus.cc:606] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Received request to transfer leadership to TS b277a3d9f11341a498aaef20eac45359
I20260709 13:22:31.271183 21061 consensus_queue.cc:1048] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.280891 21061 consensus_queue.cc:1048] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.306435 20933 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332"
dest_uuid: "b277a3d9f11341a498aaef20eac45359"
mode: GRACEFUL
new_leader_uuid: "90fa747f7aa94285a794108cb19c210c"
 from {username='slave'} at 127.0.0.1:54838
I20260709 13:22:31.307173 20933 raft_consensus.cc:606] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 LEADER]: Received request to transfer leadership to TS 90fa747f7aa94285a794108cb19c210c
I20260709 13:22:31.628443 21050 consensus_queue.cc:1048] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.630069 21080 consensus_queue.cc:1048] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.645453 21080 raft_consensus.cc:993] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: : Instructing follower 90fa747f7aa94285a794108cb19c210c to start an election
I20260709 13:22:31.646015 21080 raft_consensus.cc:1081] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 LEADER]: Signalling peer 90fa747f7aa94285a794108cb19c210c to start an election
I20260709 13:22:31.648478 20857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332"
dest_uuid: "90fa747f7aa94285a794108cb19c210c"
 from {username='slave'} at 127.0.0.1:49894
I20260709 13:22:31.649577 20857 raft_consensus.cc:493] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:31.650172 20857 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.666115 20857 raft_consensus.cc:515] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.667913 20858 raft_consensus.cc:1240] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Rejecting Update request from peer b277a3d9f11341a498aaef20eac45359 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:31.671073 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b277a3d9f11341a498aaef20eac45359"
I20260709 13:22:31.671520 21006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a9d438765174fdf8da74b213e4d5332" candidate_uuid: "90fa747f7aa94285a794108cb19c210c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:31.672170 20933 raft_consensus.cc:3055] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:31.672328 21006 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.672634 20933 raft_consensus.cc:740] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Running, Role: LEADER
I20260709 13:22:31.673970 20933 consensus_queue.cc:260] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.675735 20933 raft_consensus.cc:3060] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.670146 20857 leader_election.cc:290] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 2 election: Requested vote from peers b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.682112 21006 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 2.
I20260709 13:22:31.683905 20822 leader_election.cc:304] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fa747f7aa94285a794108cb19c210c, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.684158 20933 raft_consensus.cc:2468] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90fa747f7aa94285a794108cb19c210c in term 2.
I20260709 13:22:31.684916 21062 raft_consensus.cc:2804] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:31.685541 21062 raft_consensus.cc:697] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 LEADER]: Becoming Leader. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:31.686576 21062 consensus_queue.cc:237] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.696483 20756 catalog_manager.cc:5697] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c reported cstate change: term changed from 1 to 2, leader changed from b277a3d9f11341a498aaef20eac45359 (127.8.183.66) to 90fa747f7aa94285a794108cb19c210c (127.8.183.65). New cstate: current_term: 2 leader_uuid: "90fa747f7aa94285a794108cb19c210c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:31.704130 21062 consensus_queue.cc:1048] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.718910 21062 consensus_queue.cc:1048] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:31.804242 21056 raft_consensus.cc:993] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: : Instructing follower b277a3d9f11341a498aaef20eac45359 to start an election
I20260709 13:22:31.804932 21062 raft_consensus.cc:1081] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Signalling peer b277a3d9f11341a498aaef20eac45359 to start an election
I20260709 13:22:31.807480 20933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1"
dest_uuid: "b277a3d9f11341a498aaef20eac45359"
 from {username='slave'} at 127.0.0.1:54832
I20260709 13:22:31.808215 20933 raft_consensus.cc:493] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:31.808530 20933 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.814954 20933 raft_consensus.cc:515] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.816946 20933 leader_election.cc:290] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 2 election: Requested vote from peers 90fa747f7aa94285a794108cb19c210c (127.8.183.65:35863), dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67:43509)
I20260709 13:22:31.818090 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90fa747f7aa94285a794108cb19c210c"
I20260709 13:22:31.818301 21006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829cf8d2aca49aa9d02e75b120315c1" candidate_uuid: "b277a3d9f11341a498aaef20eac45359" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
I20260709 13:22:31.818889 20857 raft_consensus.cc:3055] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:31.819074 21006 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.819391 20857 raft_consensus.cc:740] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:31.820542 20857 consensus_queue.cc:260] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.821969 20857 raft_consensus.cc:3060] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.827787 21006 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 2.
I20260709 13:22:31.828572 20857 raft_consensus.cc:2468] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b277a3d9f11341a498aaef20eac45359 in term 2.
I20260709 13:22:31.829851 20897 leader_election.cc:304] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b277a3d9f11341a498aaef20eac45359, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:31.831156 21094 raft_consensus.cc:2804] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:31.831816 21094 raft_consensus.cc:697] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 2 LEADER]: Becoming Leader. State: Replica: b277a3d9f11341a498aaef20eac45359, State: Running, Role: LEADER
I20260709 13:22:31.832846 21094 consensus_queue.cc:237] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:31.840777 20753 catalog_manager.cc:5697] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 reported cstate change: term changed from 1 to 2, leader changed from 90fa747f7aa94285a794108cb19c210c (127.8.183.65) to b277a3d9f11341a498aaef20eac45359 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "b277a3d9f11341a498aaef20eac45359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:32.141891  8925 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:22:32.144090  8925 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:22:32.144671  8925 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:22:32.164925 20857 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34"
dest_uuid: "90fa747f7aa94285a794108cb19c210c"
mode: GRACEFUL
new_leader_uuid: "dcb445edccb446048f8e07d9dddc87a2"
 from {username='slave'} at 127.0.0.1:49910
I20260709 13:22:32.165684 20857 raft_consensus.cc:606] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Received request to transfer leadership to TS dcb445edccb446048f8e07d9dddc87a2
I20260709 13:22:32.166412 21006 raft_consensus.cc:1275] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Refusing update from remote peer 90fa747f7aa94285a794108cb19c210c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.167341  8925 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:22:32.168285 21062 consensus_queue.cc:1048] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:32.179827 20933 raft_consensus.cc:1275] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Refusing update from remote peer 90fa747f7aa94285a794108cb19c210c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.183435 21100 consensus_queue.cc:1048] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:32.224128 21006 raft_consensus.cc:1275] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Refusing update from remote peer b277a3d9f11341a498aaef20eac45359: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.226152 21094 consensus_queue.cc:1048] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:32.239310 20857 raft_consensus.cc:1275] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Refusing update from remote peer b277a3d9f11341a498aaef20eac45359: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:32.242058 21094 consensus_queue.cc:1048] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:32.698040 21100 raft_consensus.cc:993] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: : Instructing follower dcb445edccb446048f8e07d9dddc87a2 to start an election
I20260709 13:22:32.698740 21062 raft_consensus.cc:1081] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Signalling peer dcb445edccb446048f8e07d9dddc87a2 to start an election
I20260709 13:22:32.701038 21006 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34"
dest_uuid: "dcb445edccb446048f8e07d9dddc87a2"
 from {username='slave'} at 127.0.0.1:37738
I20260709 13:22:32.701692 21006 raft_consensus.cc:493] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:32.702056 21006 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.708997 21006 raft_consensus.cc:515] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:32.711705 21006 leader_election.cc:290] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [CANDIDATE]: Term 2 election: Requested vote from peers 90fa747f7aa94285a794108cb19c210c (127.8.183.65:35863), b277a3d9f11341a498aaef20eac45359 (127.8.183.66:36241)
I20260709 13:22:32.730441 20857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "dcb445edccb446048f8e07d9dddc87a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90fa747f7aa94285a794108cb19c210c"
I20260709 13:22:32.731287 20857 raft_consensus.cc:3055] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:32.731689 20857 raft_consensus.cc:740] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90fa747f7aa94285a794108cb19c210c, State: Running, Role: LEADER
I20260709 13:22:32.731595 20933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e3ab4603b9491faf0766ddd358da34" candidate_uuid: "dcb445edccb446048f8e07d9dddc87a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b277a3d9f11341a498aaef20eac45359"
I20260709 13:22:32.732534 20933 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.732921 20857 consensus_queue.cc:260] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:32.734598 20857 raft_consensus.cc:3060] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.742105 20933 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb445edccb446048f8e07d9dddc87a2 in term 2.
I20260709 13:22:32.744172 20857 raft_consensus.cc:2468] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb445edccb446048f8e07d9dddc87a2 in term 2.
I20260709 13:22:32.744496 20973 leader_election.cc:304] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b277a3d9f11341a498aaef20eac45359, dcb445edccb446048f8e07d9dddc87a2; no voters: 
I20260709 13:22:32.746845 21107 raft_consensus.cc:2804] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:32.748118 21107 raft_consensus.cc:697] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 LEADER]: Becoming Leader. State: Replica: dcb445edccb446048f8e07d9dddc87a2, State: Running, Role: LEADER
I20260709 13:22:32.749150 21107 consensus_queue.cc:237] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } }
I20260709 13:22:32.758080 20756 catalog_manager.cc:5697] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 reported cstate change: term changed from 1 to 2, leader changed from 90fa747f7aa94285a794108cb19c210c (127.8.183.65) to dcb445edccb446048f8e07d9dddc87a2 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "dcb445edccb446048f8e07d9dddc87a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fa747f7aa94285a794108cb19c210c" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 35863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b277a3d9f11341a498aaef20eac45359" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 36241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb445edccb446048f8e07d9dddc87a2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 43509 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:32.855468  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
I20260709 13:22:32.891022  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:32.891877  8925 tablet_replica.cc:333] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c: stopping tablet replica
I20260709 13:22:32.892519  8925 raft_consensus.cc:2243] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:32.893441  8925 raft_consensus.cc:2272] T 5a9d438765174fdf8da74b213e4d5332 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.896263  8925 tablet_replica.cc:333] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c: stopping tablet replica
I20260709 13:22:32.896951  8925 raft_consensus.cc:2243] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:32.897464  8925 raft_consensus.cc:2272] T 6829cf8d2aca49aa9d02e75b120315c1 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.900291  8925 tablet_replica.cc:333] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c: stopping tablet replica
I20260709 13:22:32.900931  8925 raft_consensus.cc:2243] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:32.901427  8925 raft_consensus.cc:2272] T 63e3ab4603b9491faf0766ddd358da34 P 90fa747f7aa94285a794108cb19c210c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.903775  8925 tablet_replica.cc:333] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c: stopping tablet replica
I20260709 13:22:32.904378  8925 raft_consensus.cc:2243] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:32.905422  8925 raft_consensus.cc:2272] T 153cca36fdad4e8586941ed88b22e912 P 90fa747f7aa94285a794108cb19c210c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.931751  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
I20260709 13:22:32.949542  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
I20260709 13:22:32.975666  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:32.976521  8925 tablet_replica.cc:333] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359: stopping tablet replica
I20260709 13:22:32.977224  8925 raft_consensus.cc:2243] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:32.977772  8925 raft_consensus.cc:2272] T 5a9d438765174fdf8da74b213e4d5332 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.980391  8925 tablet_replica.cc:333] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359: stopping tablet replica
I20260709 13:22:32.981037  8925 raft_consensus.cc:2243] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:32.981554  8925 raft_consensus.cc:2272] T 63e3ab4603b9491faf0766ddd358da34 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.983716  8925 tablet_replica.cc:333] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359: stopping tablet replica
I20260709 13:22:32.984280  8925 raft_consensus.cc:2243] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:32.984751  8925 raft_consensus.cc:2272] T 153cca36fdad4e8586941ed88b22e912 P b277a3d9f11341a498aaef20eac45359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:32.986727  8925 tablet_replica.cc:333] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359: stopping tablet replica
I20260709 13:22:32.987403  8925 raft_consensus.cc:2243] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:32.988328  8925 raft_consensus.cc:2272] T 6829cf8d2aca49aa9d02e75b120315c1 P b277a3d9f11341a498aaef20eac45359 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.014526  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
I20260709 13:22:33.030174  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
I20260709 13:22:33.056766  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:33.057598  8925 tablet_replica.cc:333] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2: stopping tablet replica
I20260709 13:22:33.058270  8925 raft_consensus.cc:2243] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.058851  8925 raft_consensus.cc:2272] T 5a9d438765174fdf8da74b213e4d5332 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.061620  8925 tablet_replica.cc:333] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2: stopping tablet replica
I20260709 13:22:33.062335  8925 raft_consensus.cc:2243] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.062876  8925 raft_consensus.cc:2272] T 6829cf8d2aca49aa9d02e75b120315c1 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.065495  8925 tablet_replica.cc:333] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: stopping tablet replica
I20260709 13:22:33.066195  8925 raft_consensus.cc:2243] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.066969  8925 pending_rounds.cc:70] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Trying to abort 1 pending ops.
I20260709 13:22:33.067245  8925 pending_rounds.cc:77] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305639332870836224 op_type: NO_OP noop_request { }
I20260709 13:22:33.067591  8925 raft_consensus.cc:2889] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:33.067942  8925 raft_consensus.cc:2272] T 63e3ab4603b9491faf0766ddd358da34 P dcb445edccb446048f8e07d9dddc87a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.070802  8925 tablet_replica.cc:333] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2: stopping tablet replica
I20260709 13:22:33.071517  8925 raft_consensus.cc:2243] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:33.072059  8925 raft_consensus.cc:2272] T 153cca36fdad4e8586941ed88b22e912 P dcb445edccb446048f8e07d9dddc87a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.097537  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:22:33.115737  8925 master.cc:562] Master@127.8.183.126:39343 shutting down...
I20260709 13:22:33.139643  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:33.140437  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:33.140852  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P 57378e13f67f45dbb7a500ad0221d04c: stopping tablet replica
I20260709 13:22:33.164018  8925 master.cc:584] Master@127.8.183.126:39343 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3522 ms)
[----------] 6 tests from LeaderRebalancerTest (423392 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:22:33.199605  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:36851
I20260709 13:22:33.200770  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:33.206946 21113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.208031 21114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.208869 21116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:33.210045  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:33.210906  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:33.211552  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:33.211717  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603353211697 us; error 0 us; skew 500 ppm
I20260709 13:22:33.212261  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:33.215227  8925 webserver.cc:533] Webserver started at http://127.8.183.126:36057/ using document root <none> and password file <none>
I20260709 13:22:33.215799  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:33.215965  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:33.216187  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:33.217387  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "e22de12c734741e8b1f9d7e0cc3cf03e"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.222806  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:22:33.227807 21121 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.228917  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:33.229326  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "e22de12c734741e8b1f9d7e0cc3cf03e"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.229642  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:33.241245  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:33.242601  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:33.290215  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:36851
I20260709 13:22:33.290350 21172 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:36851 every 8 connection(s)
I20260709 13:22:33.295050 21173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.308036 21173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: Bootstrap starting.
I20260709 13:22:33.313175 21173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:33.320052 21173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: No bootstrap required, opened a new log
I20260709 13:22:33.322742 21173 raft_consensus.cc:359] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER }
I20260709 13:22:33.323364 21173 raft_consensus.cc:385] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:33.323657 21173 raft_consensus.cc:740] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e22de12c734741e8b1f9d7e0cc3cf03e, State: Initialized, Role: FOLLOWER
I20260709 13:22:33.324496 21173 consensus_queue.cc:260] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER }
I20260709 13:22:33.325143 21173 raft_consensus.cc:399] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:33.325398 21173 raft_consensus.cc:493] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:33.325709 21173 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:33.332060 21173 raft_consensus.cc:515] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER }
I20260709 13:22:33.332789 21173 leader_election.cc:304] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e22de12c734741e8b1f9d7e0cc3cf03e; no voters: 
I20260709 13:22:33.334162 21173 leader_election.cc:290] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:33.334641 21176 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:33.336409 21176 raft_consensus.cc:697] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 1 LEADER]: Becoming Leader. State: Replica: e22de12c734741e8b1f9d7e0cc3cf03e, State: Running, Role: LEADER
I20260709 13:22:33.337217 21176 consensus_queue.cc:237] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER }
I20260709 13:22:33.337826 21173 sys_catalog.cc:565] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:33.341615 21178 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e22de12c734741e8b1f9d7e0cc3cf03e. Latest consensus state: current_term: 1 leader_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER } }
I20260709 13:22:33.342873 21178 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:33.341615 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22de12c734741e8b1f9d7e0cc3cf03e" member_type: VOTER } }
I20260709 13:22:33.343667 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:33.347667 21182 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:33.357098 21182 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:33.358493  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:33.368960 21182 catalog_manager.cc:1383] Generated new cluster ID: eb424f3de84244d0929e8d7c8f4496b7
I20260709 13:22:33.369451 21182 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:33.397166 21182 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:33.398944 21182 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:33.416296 21182 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: Generated new TSK 0
I20260709 13:22:33.417076 21182 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:33.426503  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:33.434080 21194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.436089 21195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.437970 21197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:33.438031  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:33.439754  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:33.440054  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:33.440258  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603353440217 us; error 0 us; skew 500 ppm
I20260709 13:22:33.440939  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:33.444100  8925 webserver.cc:533] Webserver started at http://127.8.183.65:43251/ using document root <none> and password file <none>
I20260709 13:22:33.444656  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:33.444836  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:33.445091  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:33.446488  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "9e4db2af5b894498a39e7996de5f4420"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.452574  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:22:33.457098 21202 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.458299  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:33.458782  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "9e4db2af5b894498a39e7996de5f4420"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.459224  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:33.478276  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:33.479839  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:33.481590  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:33.484750  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:33.485085  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.485414  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:33.485612  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.539296  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:43775
I20260709 13:22:33.539517 21264 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:43775 every 8 connection(s)
I20260709 13:22:33.545851  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:33.555051 21269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.556656 21270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.561475 21272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:33.561503  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:33.562547 21265 heartbeater.cc:344] Connected to a master server at 127.8.183.126:36851
I20260709 13:22:33.563150 21265 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:33.564306 21265 heartbeater.cc:507] Master 127.8.183.126:36851 requested a full tablet report, sending...
I20260709 13:22:33.564558  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:33.564937  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:33.565172  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603353565150 us; error 0 us; skew 500 ppm
I20260709 13:22:33.566092  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:33.567716 21138 ts_manager.cc:194] Registered new tserver with Master: 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775)
I20260709 13:22:33.569625  8925 webserver.cc:533] Webserver started at http://127.8.183.66:36813/ using document root <none> and password file <none>
I20260709 13:22:33.570423  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:33.570601 21138 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38836
I20260709 13:22:33.570683  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:33.571447  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:33.573223  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "c1674e5c97eb461c98fb5d863246c0f5"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.579020  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:22:33.583101 21277 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.584232  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:33.584642  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "c1674e5c97eb461c98fb5d863246c0f5"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.585022  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:33.630542  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:33.632103  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:33.633776  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:33.636615  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:33.636943  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.637264  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:33.637472  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.691660  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:46747
I20260709 13:22:33.691813 21339 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:46747 every 8 connection(s)
I20260709 13:22:33.698060  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:33.708184 21343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.708488 21344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:33.712249 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:33.712836  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:33.714403  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:33.714751  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:33.714989  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603353714962 us; error 0 us; skew 500 ppm
I20260709 13:22:33.715154 21340 heartbeater.cc:344] Connected to a master server at 127.8.183.126:36851
I20260709 13:22:33.715842 21340 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:33.716117  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:33.717267 21340 heartbeater.cc:507] Master 127.8.183.126:36851 requested a full tablet report, sending...
I20260709 13:22:33.720095  8925 webserver.cc:533] Webserver started at http://127.8.183.67:38849/ using document root <none> and password file <none>
I20260709 13:22:33.720743 21138 ts_manager.cc:194] Registered new tserver with Master: c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:33.720957  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:33.721306  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:33.721668  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:33.722967 21138 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38838
I20260709 13:22:33.723546  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "85475c7436484601a066fc66945e140d"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.730576  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:22:33.735080 21351 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.736174  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:22:33.736585  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "85475c7436484601a066fc66945e140d"
format_stamp: "Formatted at 2026-07-09 13:22:33 on dist-test-slave-ch3x"
I20260709 13:22:33.736934  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:33.751996  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:33.753585  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:33.755712  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:33.758658  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:33.758993  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.759320  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:33.759526  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:33.809058  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:40029
I20260709 13:22:33.809352 21413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:40029 every 8 connection(s)
I20260709 13:22:33.826967 21414 heartbeater.cc:344] Connected to a master server at 127.8.183.126:36851
I20260709 13:22:33.827700 21414 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:33.828866 21414 heartbeater.cc:507] Master 127.8.183.126:36851 requested a full tablet report, sending...
I20260709 13:22:33.832372 21138 ts_manager.cc:194] Registered new tserver with Master: 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:33.833256  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019809502s
I20260709 13:22:33.834450 21138 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38854
I20260709 13:22:33.870994 21138 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38866:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:33.874034 21138 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:33.969647 21305 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc044526df46e9bcb8a007ec2d0409 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:33.970420 21229 tablet_service.cc:1511] Processing CreateTablet for tablet 42e12fa7411c40339b9b70a8ad124b46 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:33.970882 21227 tablet_service.cc:1511] Processing CreateTablet for tablet f26560550b3b4f1d9cd5dfbcab383c09 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:33.971819 21305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc044526df46e9bcb8a007ec2d0409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.972432 21229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e12fa7411c40339b9b70a8ad124b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.973618 21226 tablet_service.cc:1511] Processing CreateTablet for tablet 82394bbf04c1483a9975840ec2e3d165 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:33.975250 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82394bbf04c1483a9975840ec2e3d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.976408 21228 tablet_service.cc:1511] Processing CreateTablet for tablet 89a0777da2644a02bda497c1c128979e (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:33.978188 21228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a0777da2644a02bda497c1c128979e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.978338 21225 tablet_service.cc:1511] Processing CreateTablet for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:33.969647 21304 tablet_service.cc:1511] Processing CreateTablet for tablet 42e12fa7411c40339b9b70a8ad124b46 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:33.979903 21230 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc044526df46e9bcb8a007ec2d0409 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:33.981410 21304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e12fa7411c40339b9b70a8ad124b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.985889 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:33.987398 21230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc044526df46e9bcb8a007ec2d0409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.988812 21225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.992504 21227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26560550b3b4f1d9cd5dfbcab383c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:33.994771 21223 tablet_service.cc:1511] Processing CreateTablet for tablet 53cb077d6586458784a8fa52fa64333a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:33.996930 21223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cb077d6586458784a8fa52fa64333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.003091 21222 tablet_service.cc:1511] Processing CreateTablet for tablet 12bfdc79f28641df8a7d71ab55d0cc59 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.005942 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12bfdc79f28641df8a7d71ab55d0cc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.007550 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.017197 21303 tablet_service.cc:1511] Processing CreateTablet for tablet 89a0777da2644a02bda497c1c128979e (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:34.025669 21301 tablet_service.cc:1511] Processing CreateTablet for tablet 82394bbf04c1483a9975840ec2e3d165 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:34.022202 21302 tablet_service.cc:1511] Processing CreateTablet for tablet f26560550b3b4f1d9cd5dfbcab383c09 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:34.029732 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26560550b3b4f1d9cd5dfbcab383c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.031754 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82394bbf04c1483a9975840ec2e3d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.036502 21378 tablet_service.cc:1511] Processing CreateTablet for tablet 42e12fa7411c40339b9b70a8ad124b46 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:34.037638 21303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a0777da2644a02bda497c1c128979e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.038473 21378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42e12fa7411c40339b9b70a8ad124b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.038956 21379 tablet_service.cc:1511] Processing CreateTablet for tablet 5ecc044526df46e9bcb8a007ec2d0409 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:34.040374 21377 tablet_service.cc:1511] Processing CreateTablet for tablet 89a0777da2644a02bda497c1c128979e (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:34.041435 21379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ecc044526df46e9bcb8a007ec2d0409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.043013 21376 tablet_service.cc:1511] Processing CreateTablet for tablet f26560550b3b4f1d9cd5dfbcab383c09 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:34.044628 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f26560550b3b4f1d9cd5dfbcab383c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.045059 21372 tablet_service.cc:1511] Processing CreateTablet for tablet 53cb077d6586458784a8fa52fa64333a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:34.047437 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cb077d6586458784a8fa52fa64333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.048173 21375 tablet_service.cc:1511] Processing CreateTablet for tablet 82394bbf04c1483a9975840ec2e3d165 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:34.050268 21375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82394bbf04c1483a9975840ec2e3d165. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.051424 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 12bfdc79f28641df8a7d71ab55d0cc59 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.052531 21374 tablet_service.cc:1511] Processing CreateTablet for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:34.053911 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12bfdc79f28641df8a7d71ab55d0cc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.055306 21373 tablet_service.cc:1511] Processing CreateTablet for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:34.057072 21373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.036502 21300 tablet_service.cc:1511] Processing CreateTablet for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:34.063994 21377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a0777da2644a02bda497c1c128979e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.080787 21298 tablet_service.cc:1511] Processing CreateTablet for tablet 53cb077d6586458784a8fa52fa64333a (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:34.091678 21297 tablet_service.cc:1511] Processing CreateTablet for tablet 12bfdc79f28641df8a7d71ab55d0cc59 (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:34.134678 21298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53cb077d6586458784a8fa52fa64333a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.081503 21299 tablet_service.cc:1511] Processing CreateTablet for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc (DEFAULT_TABLE table=test-workload [id=ea2d1122ed624178912561df502491b0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:34.140321 21299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ce7fffd50a45bebc3ba745a2ec9fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.141987 21300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.159185 21374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffeb6d2b8c944c2ba0bd0e7a06fe6a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.159528 21297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12bfdc79f28641df8a7d71ab55d0cc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:34.187057 21430 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.205096 21431 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.206660 21429 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.225384 21429 tablet_bootstrap.cc:654] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.257685 21431 tablet_bootstrap.cc:654] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.275684 21430 tablet_bootstrap.cc:654] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.281255 21431 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.285162 21431 ts_tablet_manager.cc:1403] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.017s
I20260709 13:22:34.288739 21431 raft_consensus.cc:359] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.289734 21431 raft_consensus.cc:385] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.290126 21431 raft_consensus.cc:740] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.291307 21431 consensus_queue.cc:260] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.294726 21414 heartbeater.cc:499] Master 127.8.183.126:36851 was elected leader, sending a full tablet report...
I20260709 13:22:34.297627 21431 ts_tablet_manager.cc:1434] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:22:34.298758 21429 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.299068 21431 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.299631 21429 ts_tablet_manager.cc:1403] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.006s
I20260709 13:22:34.307494 21431 tablet_bootstrap.cc:654] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.307493 21429 raft_consensus.cc:359] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.312498 21429 raft_consensus.cc:385] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.312987 21429 raft_consensus.cc:740] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.314030 21429 consensus_queue.cc:260] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.320007 21340 heartbeater.cc:499] Master 127.8.183.126:36851 was elected leader, sending a full tablet report...
I20260709 13:22:34.320886 21429 ts_tablet_manager.cc:1434] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:22:34.322211 21429 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.322618 21430 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.323318 21430 ts_tablet_manager.cc:1403] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.137s	user 0.006s	sys 0.015s
I20260709 13:22:34.327034 21430 raft_consensus.cc:359] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.330245 21429 tablet_bootstrap.cc:654] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.330518 21430 raft_consensus.cc:385] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.332825 21430 raft_consensus.cc:740] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.334197 21431 tablet_bootstrap.cc:492] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.334100 21430 consensus_queue.cc:260] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.334955 21431 ts_tablet_manager.cc:1403] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.004s
I20260709 13:22:34.339947 21431 raft_consensus.cc:359] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.343878 21431 raft_consensus.cc:385] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.344350 21431 raft_consensus.cc:740] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.345580 21431 consensus_queue.cc:260] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.348493 21431 ts_tablet_manager.cc:1434] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:34.349972 21431 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.352339 21265 heartbeater.cc:499] Master 127.8.183.126:36851 was elected leader, sending a full tablet report...
I20260709 13:22:34.360486 21429 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.361194 21429 ts_tablet_manager.cc:1403] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
W20260709 13:22:34.361904 21186 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:34.364506 21429 raft_consensus.cc:359] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.365572 21429 raft_consensus.cc:385] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.366019 21429 raft_consensus.cc:740] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.350131 21430 ts_tablet_manager.cc:1434] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.026s	user 0.014s	sys 0.000s
I20260709 13:22:34.367982 21430 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.368901 21431 tablet_bootstrap.cc:654] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.375630 21429 consensus_queue.cc:260] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.376772 21430 tablet_bootstrap.cc:654] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.381579 21429 ts_tablet_manager.cc:1434] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.006s
I20260709 13:22:34.383234 21429 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.390810 21429 tablet_bootstrap.cc:654] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.395076 21431 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.395895 21431 ts_tablet_manager.cc:1403] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.009s
I20260709 13:22:34.397353 21430 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.398398 21430 ts_tablet_manager.cc:1403] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.010s
I20260709 13:22:34.399168 21438 raft_consensus.cc:493] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.399564 21431 raft_consensus.cc:359] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.399956 21438 raft_consensus.cc:515] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.400417 21431 raft_consensus.cc:385] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.400970 21431 raft_consensus.cc:740] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.402179 21431 consensus_queue.cc:260] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.403478 21430 raft_consensus.cc:359] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.404744 21430 raft_consensus.cc:385] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.405201 21430 raft_consensus.cc:740] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.406527 21430 consensus_queue.cc:260] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.414955 21431 ts_tablet_manager.cc:1434] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:22:34.417037 21438 leader_election.cc:290] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.425195 21431 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.430007 21429 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.431180 21429 ts_tablet_manager.cc:1403] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.048s	user 0.028s	sys 0.019s
I20260709 13:22:34.436610 21429 raft_consensus.cc:359] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.437920 21431 tablet_bootstrap.cc:654] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.437965 21429 raft_consensus.cc:385] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.443356 21429 raft_consensus.cc:740] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.445165 21430 ts_tablet_manager.cc:1434] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.046s	user 0.024s	sys 0.017s
I20260709 13:22:34.447175 21430 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.444747 21429 consensus_queue.cc:260] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.456841 21434 raft_consensus.cc:493] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.457908 21434 raft_consensus.cc:515] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.459342 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:34.460244 21389 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:34.469477 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.470427 21315 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:34.470844 21431 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.471055 21429 ts_tablet_manager.cc:1434] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.008s
I20260709 13:22:34.471784 21431 ts_tablet_manager.cc:1403] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.018s
I20260709 13:22:34.472491 21429 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.479151 21430 tablet_bootstrap.cc:654] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.482275 21436 raft_consensus.cc:493] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.483175 21436 raft_consensus.cc:515] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.486653 21436 leader_election.cc:290] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.482033 21431 raft_consensus.cc:359] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.489359 21434 leader_election.cc:290] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.489764 21431 raft_consensus.cc:385] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.490199 21431 raft_consensus.cc:740] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.491585 21431 consensus_queue.cc:260] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.494582 21431 ts_tablet_manager.cc:1434] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:22:34.495975 21429 tablet_bootstrap.cc:654] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.499357 21431 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.504518 21205 leader_election.cc:304] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.511945 21444 raft_consensus.cc:2804] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.512735 21444 raft_consensus.cc:493] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.513208 21444 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.516000 21431 tablet_bootstrap.cc:654] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.534439 21431 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.536005 21431 ts_tablet_manager.cc:1403] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.000s
I20260709 13:22:34.542659 21429 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.536213 21444 raft_consensus.cc:515] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.546883 21429 ts_tablet_manager.cc:1403] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.075s	user 0.019s	sys 0.000s
I20260709 13:22:34.550225 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:34.543058 21430 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.551239 21315 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.551762 21430 ts_tablet_manager.cc:1403] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.105s	user 0.034s	sys 0.035s
I20260709 13:22:34.555884 21240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.556115 21430 raft_consensus.cc:359] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.557152 21430 raft_consensus.cc:385] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.557524 21430 raft_consensus.cc:740] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.557168 21240 raft_consensus.cc:2468] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:34.545274 21431 raft_consensus.cc:359] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.559324 21431 raft_consensus.cc:385] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.559643 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.560609 21281 leader_election.cc:304] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.561034 21314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.561372 21429 raft_consensus.cc:359] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.558420 21430 consensus_queue.cc:260] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.562650 21436 raft_consensus.cc:2804] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:22:34.562187 21355 leader_election.cc:343] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.563161 21436 raft_consensus.cc:493] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.563570 21436 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:34.563818 21354 leader_election.cc:343] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.565385 21354 leader_election.cc:304] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85475c7436484601a066fc66945e140d; no voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:34.559732 21431 raft_consensus.cc:740] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.567397 21434 raft_consensus.cc:2749] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:34.568336 21431 consensus_queue.cc:260] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.571175 21431 ts_tablet_manager.cc:1434] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:22:34.562213 21429 raft_consensus.cc:385] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.572180 21429 raft_consensus.cc:740] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.572549 21431 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.573371 21429 consensus_queue.cc:260] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.574503 21436 raft_consensus.cc:515] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.575873 21444 leader_election.cc:290] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.576813 21430 ts_tablet_manager.cc:1434] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.004s
I20260709 13:22:34.578158 21436 leader_election.cc:290] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.580628 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:34.581259 21430 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.581481 21239 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.581889 21315 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:34.583570 21205 leader_election.cc:304] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.584285 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:34.585023 21389 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.587932 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:34.590783 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
W20260709 13:22:34.593653 21281 leader_election.cc:343] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85475c7436484601a066fc66945e140d (127.8.183.67:40029): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:34.594852 21281 leader_election.cc:343] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 85475c7436484601a066fc66945e140d (127.8.183.67:40029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.595826 21444 raft_consensus.cc:2804] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.596647 21436 raft_consensus.cc:493] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.596808 21444 raft_consensus.cc:697] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:34.597249 21436 raft_consensus.cc:515] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.598042 21444 consensus_queue.cc:237] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.600144 21436 leader_election.cc:290] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.601748 21240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.602432 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:34.603039 21430 tablet_bootstrap.cc:654] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:34.603989 21281 leader_election.cc:343] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:34.605024 21281 leader_election.cc:343] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85475c7436484601a066fc66945e140d (127.8.183.67:40029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.605525 21281 leader_election.cc:304] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:34.606947 21436 raft_consensus.cc:2749] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:34.610222 21389 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:34.608812 21239 raft_consensus.cc:2468] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:34.611631 21429 ts_tablet_manager.cc:1434] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.064s	user 0.015s	sys 0.006s
I20260709 13:22:34.612624 21281 leader_election.cc:304] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d
I20260709 13:22:34.614815 21431 tablet_bootstrap.cc:654] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.614750 21436 raft_consensus.cc:2804] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.618182 21436 raft_consensus.cc:697] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:34.619452 21436 consensus_queue.cc:237] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.621807 21430 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.622691 21430 ts_tablet_manager.cc:1403] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.042s	user 0.024s	sys 0.000s
I20260709 13:22:34.622928 21429 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.627758 21430 raft_consensus.cc:359] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.629170 21430 raft_consensus.cc:385] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.629602 21430 raft_consensus.cc:740] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.630821 21430 consensus_queue.cc:260] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.633514 21430 ts_tablet_manager.cc:1434] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:22:34.634920 21430 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.635970 21431 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.636772 21431 ts_tablet_manager.cc:1403] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.000s
I20260709 13:22:34.636415 21136 catalog_manager.cc:5697] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:34.637858 21429 tablet_bootstrap.cc:654] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.641016 21431 raft_consensus.cc:359] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.642416 21431 raft_consensus.cc:385] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.642859 21431 raft_consensus.cc:740] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.643410 21430 tablet_bootstrap.cc:654] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.641594 21137 catalog_manager.cc:5697] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:34.644198 21431 consensus_queue.cc:260] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.660240 21431 ts_tablet_manager.cc:1434] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:22:34.660928 21429 tablet_bootstrap.cc:492] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.662417 21429 ts_tablet_manager.cc:1403] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.003s
I20260709 13:22:34.666273 21429 raft_consensus.cc:359] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.669162 21429 raft_consensus.cc:385] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.669845 21429 raft_consensus.cc:740] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.671087 21429 consensus_queue.cc:260] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.674124 21436 raft_consensus.cc:493] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.674803 21429 ts_tablet_manager.cc:1434] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:22:34.674701 21436 raft_consensus.cc:515] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.676198 21429 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.676584 21431 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.678542 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.679512 21239 raft_consensus.cc:2468] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:34.681499 21281 leader_election.cc:304] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.682840 21455 raft_consensus.cc:2804] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.683303 21455 raft_consensus.cc:493] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.683682 21455 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.685457 21429 tablet_bootstrap.cc:654] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.688043 21436 leader_election.cc:290] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.689265 21430 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.690615 21431 tablet_bootstrap.cc:654] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.690361 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:34.691736 21430 ts_tablet_manager.cc:1403] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.032s
W20260709 13:22:34.693327 21281 leader_election.cc:343] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85475c7436484601a066fc66945e140d (127.8.183.67:40029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.695544 21430 raft_consensus.cc:359] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.696498 21430 raft_consensus.cc:385] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.696893 21430 raft_consensus.cc:740] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.697868 21430 consensus_queue.cc:260] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.700941 21430 ts_tablet_manager.cc:1434] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:22:34.702353 21430 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.711148 21430 tablet_bootstrap.cc:654] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.715898 21431 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.717015 21431 ts_tablet_manager.cc:1403] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.041s	user 0.023s	sys 0.009s
I20260709 13:22:34.721886 21431 raft_consensus.cc:359] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.722638 21434 raft_consensus.cc:493] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.723740 21434 raft_consensus.cc:515] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.726496 21430 tablet_bootstrap.cc:492] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.727278 21430 ts_tablet_manager.cc:1403] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 13:22:34.727929 21434 leader_election.cc:290] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.728204 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.728694 21431 raft_consensus.cc:385] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.729025 21431 raft_consensus.cc:740] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.729740 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.730021 21431 consensus_queue.cc:260] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
W20260709 13:22:34.731916 21354 leader_election.cc:343] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:34.732787 21355 leader_election.cc:343] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.733292 21355 leader_election.cc:304] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85475c7436484601a066fc66945e140d; no voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:34.734603 21434 raft_consensus.cc:2749] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:34.731886 21430 raft_consensus.cc:359] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.736346 21431 ts_tablet_manager.cc:1434] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.005s
I20260709 13:22:34.737491 21430 raft_consensus.cc:385] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.737965 21430 raft_consensus.cc:740] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.738921 21431 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.739233 21430 consensus_queue.cc:260] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.742556 21430 ts_tablet_manager.cc:1434] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:22:34.743903 21430 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.751654 21430 tablet_bootstrap.cc:654] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.755661 21431 tablet_bootstrap.cc:654] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.726969 21455 raft_consensus.cc:515] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.766521 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:34.767594 21239 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.773731 21431 tablet_bootstrap.cc:492] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.774847 21431 ts_tablet_manager.cc:1403] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.001s
I20260709 13:22:34.777123 21444 raft_consensus.cc:493] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.777784 21444 raft_consensus.cc:515] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.778873 21431 raft_consensus.cc:359] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.779872 21431 raft_consensus.cc:385] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.780194 21431 raft_consensus.cc:740] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.780731 21444 leader_election.cc:290] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.781471 21431 consensus_queue.cc:260] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.783062 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.783885 21315 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:34.785399 21239 raft_consensus.cc:2468] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:34.786909 21205 leader_election.cc:304] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.787318 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:34.787401 21281 leader_election.cc:304] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.788290 21389 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:34.788678 21436 raft_consensus.cc:2804] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.788218 21444 raft_consensus.cc:2804] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.791077 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:34.791291 21444 raft_consensus.cc:493] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.792110 21389 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.792294 21455 leader_election.cc:290] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.792145 21444 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.798791 21431 ts_tablet_manager.cc:1434] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:22:34.800235 21431 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:34.802315 21430 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.802912 21389 raft_consensus.cc:2468] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:34.803083 21430 ts_tablet_manager.cc:1403] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.059s	user 0.030s	sys 0.023s
I20260709 13:22:34.805966 21444 raft_consensus.cc:515] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.808249 21430 raft_consensus.cc:359] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.810480 21436 raft_consensus.cc:697] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:34.811483 21430 raft_consensus.cc:385] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.812028 21430 raft_consensus.cc:740] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.812338 21436 consensus_queue.cc:237] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.813021 21430 consensus_queue.cc:260] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.809832 21429 tablet_bootstrap.cc:492] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.813516 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:34.814386 21315 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.816027 21430 ts_tablet_manager.cc:1434] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:22:34.817370 21430 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.824183 21431 tablet_bootstrap.cc:654] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.825959 21430 tablet_bootstrap.cc:654] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.823530 21429 ts_tablet_manager.cc:1403] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.148s	user 0.032s	sys 0.060s
I20260709 13:22:34.835896 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:34.836818 21389 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.837332 21434 raft_consensus.cc:493] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.838006 21434 raft_consensus.cc:515] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.838235 21429 raft_consensus.cc:359] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.845786 21429 raft_consensus.cc:385] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.844645 21315 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:34.846261 21389 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:34.846221 21429 raft_consensus.cc:740] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.848598 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.848784 21429 consensus_queue.cc:260] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.849282 21206 leader_election.cc:304] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:34.849838 21239 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:34.833464 21444 leader_election.cc:290] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:34.850965 21444 raft_consensus.cc:2804] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.852527 21444 raft_consensus.cc:697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:34.853806 21444 consensus_queue.cc:237] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.856513 21355 leader_election.cc:304] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:34.863082 21429 ts_tablet_manager.cc:1434] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.004s
I20260709 13:22:34.866576 21431 tablet_bootstrap.cc:492] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:34.866936 21430 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.867409 21431 ts_tablet_manager.cc:1403] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.067s	user 0.020s	sys 0.014s
I20260709 13:22:34.867717 21430 ts_tablet_manager.cc:1403] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.012s
I20260709 13:22:34.869249 21429 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.871286 21474 raft_consensus.cc:2804] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.871347 21431 raft_consensus.cc:359] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.872473 21431 raft_consensus.cc:385] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.872815 21431 raft_consensus.cc:740] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.871950 21137 catalog_manager.cc:5697] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:34.873387 21430 raft_consensus.cc:359] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.874250 21430 raft_consensus.cc:385] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.873739 21431 consensus_queue.cc:260] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.874583 21430 raft_consensus.cc:740] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.875679 21430 consensus_queue.cc:260] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.872025 21474 raft_consensus.cc:493] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.880285 21474 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.884394 21430 ts_tablet_manager.cc:1434] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:22:34.885875 21430 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:34.891868 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.893201 21315 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:34.894434 21434 leader_election.cc:290] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.900002 21430 tablet_bootstrap.cc:654] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.901337 21476 raft_consensus.cc:493] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.902570 21431 ts_tablet_manager.cc:1434] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.035s	user 0.021s	sys 0.012s
I20260709 13:22:34.903874 21429 tablet_bootstrap.cc:654] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.904625 21476 raft_consensus.cc:515] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.913514 21430 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:34.914474 21430 ts_tablet_manager.cc:1403] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.009s
I20260709 13:22:34.919384 21430 raft_consensus.cc:359] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.920511 21430 raft_consensus.cc:385] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.920907 21430 raft_consensus.cc:740] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.922672 21430 consensus_queue.cc:260] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.926538 21430 ts_tablet_manager.cc:1434] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:22:34.908277 21478 raft_consensus.cc:493] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.930881 21478 raft_consensus.cc:515] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.931432 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.932183 21239 raft_consensus.cc:2468] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:34.933755 21355 leader_election.cc:304] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:34.934306 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:34.935057 21239 raft_consensus.cc:2468] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:34.936787 21355 leader_election.cc:304] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:34.938656 21434 raft_consensus.cc:2804] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.939122 21434 raft_consensus.cc:493] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.939505 21434 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.939775 21138 catalog_manager.cc:5697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:34.944373 21476 leader_election.cc:290] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.946394 21478 leader_election.cc:290] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.946942 21314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.947736 21434 raft_consensus.cc:515] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
W20260709 13:22:34.948896 21354 leader_election.cc:343] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.951478 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:34.951767 21474 raft_consensus.cc:515] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.945235 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:34.944677 21477 raft_consensus.cc:2804] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:34.954149 21477 raft_consensus.cc:493] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:34.954252 21354 leader_election.cc:343] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.954648 21477 raft_consensus.cc:3060] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.954813 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:34.952419 21239 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.956980 21240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:34.957304 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:34.957870 21240 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:34.958829 21354 leader_election.cc:343] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:34.957914 21315 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.965991 21429 tablet_bootstrap.cc:492] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:34.966878 21429 ts_tablet_manager.cc:1403] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.098s	user 0.020s	sys 0.028s
I20260709 13:22:34.955592 21434 leader_election.cc:290] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.970602 21315 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:34.954860 21474 leader_election.cc:290] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.970641 21429 raft_consensus.cc:359] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.971839 21429 raft_consensus.cc:385] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:34.972214 21429 raft_consensus.cc:740] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:34.972543 21354 leader_election.cc:304] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:34.973167 21429 consensus_queue.cc:260] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.974064 21474 raft_consensus.cc:2804] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.974730 21474 raft_consensus.cc:697] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:34.975499 21429 ts_tablet_manager.cc:1434] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:22:34.975797 21474 consensus_queue.cc:237] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.976884 21429 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:34.979149 21240 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:34.981467 21239 raft_consensus.cc:2468] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:34.985719 21429 tablet_bootstrap.cc:654] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:34.985165 21477 raft_consensus.cc:515] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.987856 21355 leader_election.cc:304] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:34.989357 21477 leader_election.cc:290] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:34.990036 21434 raft_consensus.cc:2804] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:34.990695 21434 raft_consensus.cc:697] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:34.991851 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:34.991747 21434 consensus_queue.cc:237] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:34.992763 21239 raft_consensus.cc:3060] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:34.993698 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
W20260709 13:22:34.995690 21354 leader_election.cc:343] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:35.003906 21239 raft_consensus.cc:2468] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:35.006337 21355 leader_election.cc:304] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:35.008646 21434 raft_consensus.cc:2804] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.012214 21434 raft_consensus.cc:697] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:35.013674 21434 consensus_queue.cc:237] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.017393 21429 tablet_bootstrap.cc:492] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:35.018038 21429 ts_tablet_manager.cc:1403] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.022s
I20260709 13:22:35.021268 21429 raft_consensus.cc:359] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.021900 21429 raft_consensus.cc:385] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.022181 21429 raft_consensus.cc:740] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.021415 21138 catalog_manager.cc:5697] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.022927 21429 consensus_queue.cc:260] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.023795 21444 raft_consensus.cc:493] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.024391 21444 raft_consensus.cc:515] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.025271 21429 ts_tablet_manager.cc:1434] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:35.026468 21429 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:35.026927 21444 leader_election.cc:290] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.028688 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:35.029547 21389 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:35.030833 21206 leader_election.cc:304] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:35.031711 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:35.031893 21444 raft_consensus.cc:2804] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.032552 21315 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:35.032691 21444 raft_consensus.cc:493] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.033497 21444 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.034706 21429 tablet_bootstrap.cc:654] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:35.042363 21429 tablet_bootstrap.cc:492] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:35.042981 21429 ts_tablet_manager.cc:1403] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 13:22:35.043732 21444 raft_consensus.cc:515] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.046208 21429 raft_consensus.cc:359] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.047219 21429 raft_consensus.cc:385] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:35.047510 21429 raft_consensus.cc:740] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:35.047904 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.048296 21429 consensus_queue.cc:260] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.048758 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:35.048892 21444 leader_election.cc:290] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.049503 21315 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.050917 21429 ts_tablet_manager.cc:1434] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:22:35.048763 21389 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.060529 21136 catalog_manager.cc:5697] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.060459 21315 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:35.063439 21389 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:35.063048 21136 catalog_manager.cc:5697] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:35.065141 21205 leader_election.cc:304] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.065290 21444 consensus_queue.cc:1048] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.066073 21438 raft_consensus.cc:2804] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.067824 21438 raft_consensus.cc:697] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.069404 21438 consensus_queue.cc:237] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.080384 21137 catalog_manager.cc:5697] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.093604 21486 consensus_queue.cc:1048] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.104020 21436 consensus_queue.cc:1048] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.116612 21486 raft_consensus.cc:493] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.117257 21486 raft_consensus.cc:515] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.120285 21389 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.126084 21486 leader_election.cc:290] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.126247 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:35.127246 21315 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:35.128424 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:35.128836 21205 leader_election.cc:304] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.129348 21387 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:35.130018 21486 raft_consensus.cc:2804] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:35.130453 21486 raft_consensus.cc:493] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.130847 21486 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.133062 21470 consensus_queue.cc:1048] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.139057 21486 raft_consensus.cc:515] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.142740 21486 leader_election.cc:290] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.142946 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:35.143879 21315 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.144021 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.144868 21389 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.152367 21315 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:35.153364 21389 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:35.154062 21205 leader_election.cc:304] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.155078 21486 raft_consensus.cc:2804] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:35.155701 21486 raft_consensus.cc:697] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.156489 21486 consensus_queue.cc:237] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.166426 21138 catalog_manager.cc:5697] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.230269 21315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
mode: GRACEFUL
new_leader_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:51302
I20260709 13:22:35.230983 21315 raft_consensus.cc:606] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Received request to transfer leadership to TS 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:35.248467 21389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89a0777da2644a02bda497c1c128979e"
dest_uuid: "85475c7436484601a066fc66945e140d"
mode: GRACEFUL
new_leader_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:22:35.249387 21389 raft_consensus.cc:606] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Received request to transfer leadership to TS 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:35.256359 21315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
mode: GRACEFUL
new_leader_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:51302
I20260709 13:22:35.257017 21315 raft_consensus.cc:606] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Received request to transfer leadership to TS 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:35.264550 21389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165"
dest_uuid: "85475c7436484601a066fc66945e140d"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:22:35.265269 21389 raft_consensus.cc:606] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:35.281441 21239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:36500
I20260709 13:22:35.282078 21239 raft_consensus.cc:606] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:35.288075 21486 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.289463 21239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:36500
I20260709 13:22:35.290266 21239 raft_consensus.cc:606] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:35.297261 21239 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:36500
I20260709 13:22:35.298022 21239 raft_consensus.cc:606] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:35.298734 21444 raft_consensus.cc:993] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.299964 21444 raft_consensus.cc:1081] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.305464 21315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51280
I20260709 13:22:35.305603 21438 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.306257 21315 raft_consensus.cc:493] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.306674 21315 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.319823 21470 consensus_queue.cc:1048] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.319823 21315 raft_consensus.cc:515] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.328542 21315 leader_election.cc:290] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.330628 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.333206 21387 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.335732 21240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:35.336601 21240 raft_consensus.cc:3055] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.336975 21240 raft_consensus.cc:740] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.337941 21240 consensus_queue.cc:260] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.339646 21240 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.342840 21387 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.344487 21281 leader_election.cc:304] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.345573 21470 raft_consensus.cc:2804] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.347302 21436 raft_consensus.cc:993] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: : Instructing follower 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.349514 21240 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.350744 21504 raft_consensus.cc:1081] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Signalling peer 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.358784 21239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:36496
I20260709 13:22:35.359620 21239 raft_consensus.cc:493] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.361275 21239 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.362780 21503 consensus_queue.cc:1048] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.362140 21470 raft_consensus.cc:697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.366468 21470 consensus_queue.cc:237] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.372148 21503 raft_consensus.cc:493] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.372922 21503 raft_consensus.cc:515] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.375737 21239 raft_consensus.cc:515] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.377329 21240 raft_consensus.cc:1240] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Rejecting Update request from peer c1674e5c97eb461c98fb5d863246c0f5 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:35.377625 21238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:35.378750 21238 raft_consensus.cc:2393] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1674e5c97eb461c98fb5d863246c0f5 in current term 1: Already voted for candidate 85475c7436484601a066fc66945e140d in this term.
I20260709 13:22:35.378495 21137 catalog_manager.cc:5697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.381070 21239 leader_election.cc:290] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.382521 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.383548 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f26560550b3b4f1d9cd5dfbcab383c09" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:35.384261 21315 raft_consensus.cc:3055] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.375849 21503 leader_election.cc:290] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.384615 21315 raft_consensus.cc:740] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.385387 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:35.385615 21503 consensus_queue.cc:1059] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.390915 21315 consensus_queue.cc:260] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.395341 21315 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.395341 21281 leader_election.cc:304] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:35.397316 21470 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.406355 21315 raft_consensus.cc:2468] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:35.409734 21205 leader_election.cc:304] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d
I20260709 13:22:35.411913 21438 raft_consensus.cc:2804] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.412513 21434 consensus_queue.cc:1048] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.412578 21438 raft_consensus.cc:697] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.418351 21470 raft_consensus.cc:2749] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:35.418756 21438 consensus_queue.cc:237] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.422904 21505 raft_consensus.cc:493] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.423528 21505 raft_consensus.cc:515] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.425953 21505 leader_election.cc:290] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.427369 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:35.428049 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12bfdc79f28641df8a7d71ab55d0cc59" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:35.429025 21239 raft_consensus.cc:2393] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1674e5c97eb461c98fb5d863246c0f5 in current term 1: Already voted for candidate 85475c7436484601a066fc66945e140d in this term.
I20260709 13:22:35.429637 21478 raft_consensus.cc:993] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.430367 21474 raft_consensus.cc:1081] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.430954 21281 leader_election.cc:304] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:35.433712 21313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:22:35.435395 21470 raft_consensus.cc:3060] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:35.435676 21313 raft_consensus.cc:493] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.436532 21313 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.438853 21434 consensus_queue.cc:1048] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.439404 21137 catalog_manager.cc:5697] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 1 to 2, leader changed from c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66) to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.447073 21313 raft_consensus.cc:515] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.451053 21470 raft_consensus.cc:2749] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:35.454821 21313 leader_election.cc:290] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.456125 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.456156 21238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82394bbf04c1483a9975840ec2e3d165" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:35.457047 21387 raft_consensus.cc:3055] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.457104 21238 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.457770 21387 raft_consensus.cc:740] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:35.458828 21387 consensus_queue.cc:260] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.462430 21387 raft_consensus.cc:3060] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.467031 21238 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.468845 21281 leader_election.cc:304] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.470013 21470 raft_consensus.cc:2804] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.471587 21470 raft_consensus.cc:697] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.471902 21387 raft_consensus.cc:2468] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.472600 21470 consensus_queue.cc:237] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.481828 21138 catalog_manager.cc:5697] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 85475c7436484601a066fc66945e140d (127.8.183.67) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.487984 21434 consensus_queue.cc:1048] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.503257 21478 consensus_queue.cc:1048] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.562343 21478 consensus_queue.cc:1048] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.571004 21474 raft_consensus.cc:993] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: : Instructing follower 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.571734 21434 raft_consensus.cc:1081] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Signalling peer 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.573699 21239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89a0777da2644a02bda497c1c128979e"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:36480
I20260709 13:22:35.576350 21239 raft_consensus.cc:493] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.576943 21239 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.578866 21434 consensus_queue.cc:1048] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.586357 21239 raft_consensus.cc:515] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.589107 21438 consensus_queue.cc:1048] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.590711 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:35.591074 21239 leader_election.cc:290] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.591615 21313 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.594043 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a0777da2644a02bda497c1c128979e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.594869 21388 raft_consensus.cc:3055] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.595284 21388 raft_consensus.cc:740] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:35.596248 21388 consensus_queue.cc:260] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.597877 21388 raft_consensus.cc:3060] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.609093 21313 raft_consensus.cc:2468] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:35.613627 21388 raft_consensus.cc:2468] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:35.615538 21206 leader_election.cc:304] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:35.617173 21486 raft_consensus.cc:2804] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.617027 21444 consensus_queue.cc:1048] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.617928 21486 raft_consensus.cc:697] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.618999 21486 consensus_queue.cc:237] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.629467 21438 raft_consensus.cc:993] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.630167 21486 raft_consensus.cc:1081] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.629351 21136 catalog_manager.cc:5697] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 1 to 2, leader changed from 85475c7436484601a066fc66945e140d (127.8.183.67) to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.632433 21313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51280
I20260709 13:22:35.633335 21313 raft_consensus.cc:493] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.633846 21313 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.641628 21313 raft_consensus.cc:515] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.644070 21313 leader_election.cc:290] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.645416 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:35.646148 21239 raft_consensus.cc:3055] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.646401 21239 raft_consensus.cc:740] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.646234 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53cb077d6586458784a8fa52fa64333a" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.647308 21388 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.647534 21239 consensus_queue.cc:260] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.648872 21239 raft_consensus.cc:3060] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.655431 21388 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.655472 21239 raft_consensus.cc:2468] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.657073 21281 leader_election.cc:304] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.657955 21470 raft_consensus.cc:2804] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.658516 21470 raft_consensus.cc:697] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.659507 21470 consensus_queue.cc:237] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.667094 21136 catalog_manager.cc:5697] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.682502 21486 consensus_queue.cc:1048] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:35.691730 21438 raft_consensus.cc:993] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.692338 21444 raft_consensus.cc:1081] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:35.694710 21314 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51280
I20260709 13:22:35.695678 21314 raft_consensus.cc:493] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.696174 21314 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.701282 21486 consensus_queue.cc:1048] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:35.707675 21314 raft_consensus.cc:515] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.711666 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:35.712512 21239 raft_consensus.cc:3055] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.712893 21239 raft_consensus.cc:740] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.711719 21314 leader_election.cc:290] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.713979 21239 consensus_queue.cc:260] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.714957 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ce7fffd50a45bebc3ba745a2ec9fdc" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.715962 21239 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.716241 21388 raft_consensus.cc:3060] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.723759 21505 raft_consensus.cc:993] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: : Instructing follower 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.724468 21470 raft_consensus.cc:1081] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Signalling peer 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:35.724960 21239 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.725224 21388 raft_consensus.cc:2468] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:35.726394 21281 leader_election.cc:304] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:35.726758 21239 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:36496
I20260709 13:22:35.727363 21470 raft_consensus.cc:2804] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.727566 21239 raft_consensus.cc:493] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:35.727922 21470 raft_consensus.cc:697] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.728048 21239 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.728946 21470 consensus_queue.cc:237] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.737815 21239 raft_consensus.cc:515] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.739677 21136 catalog_manager.cc:5697] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.741693 21314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:35.741899 21239 leader_election.cc:290] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:35.742146 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42e12fa7411c40339b9b70a8ad124b46" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:35.742805 21314 raft_consensus.cc:3055] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:35.742909 21388 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.745196 21314 raft_consensus.cc:740] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:35.746546 21314 consensus_queue.cc:260] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.748198 21314 raft_consensus.cc:3060] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.754072 21388 raft_consensus.cc:2468] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:35.755779 21389 raft_consensus.cc:1275] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:35.755725 21206 leader_election.cc:304] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:35.757514 21486 raft_consensus.cc:2804] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:35.757722 21314 raft_consensus.cc:2468] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:35.758148 21486 raft_consensus.cc:697] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:35.757862 21470 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:35.759342 21486 consensus_queue.cc:237] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:35.771478 21138 catalog_manager.cc:5697] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 1 to 2, leader changed from c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66) to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:35.775323 21239 raft_consensus.cc:1275] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:35.777762 21505 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:35.854912 21389 raft_consensus.cc:3060] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:35.861805 21389 raft_consensus.cc:1275] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:35.863493 21486 consensus_queue.cc:1048] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:35.875207 21314 raft_consensus.cc:1275] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:35.877177 21486 consensus_queue.cc:1048] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:36.024082 21239 raft_consensus.cc:1275] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.026147 21505 consensus_queue.cc:1048] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.032387 21389 raft_consensus.cc:1275] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.034514 21470 consensus_queue.cc:1048] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:36.093891 21389 raft_consensus.cc:1275] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.095886 21486 consensus_queue.cc:1048] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.107438 21314 raft_consensus.cc:1275] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.110347 21438 consensus_queue.cc:1048] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.137995 21239 raft_consensus.cc:1275] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.139674 21470 consensus_queue.cc:1048] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.152268 21389 raft_consensus.cc:1275] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.154745 21503 consensus_queue.cc:1048] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:36.236116 21389 raft_consensus.cc:1275] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.237658 21438 consensus_queue.cc:1048] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.250285 21313 raft_consensus.cc:1275] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.252195 21517 consensus_queue.cc:1048] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.278344 21239 raft_consensus.cc:1275] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.280455 21503 consensus_queue.cc:1048] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:36.290546 21389 raft_consensus.cc:1275] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:36.293339 21455 consensus_queue.cc:1048] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:38.371456 21138 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38878:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:38.374694 21138 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:22:38.426118 21186 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:38.446733 21222 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc231530744a91bbac7f6dd2b389c9 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.448494 21299 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc231530744a91bbac7f6dd2b389c9 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.449344 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc231530744a91bbac7f6dd2b389c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.450141 21299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc231530744a91bbac7f6dd2b389c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.454308 21378 tablet_service.cc:1511] Processing CreateTablet for tablet 74cc231530744a91bbac7f6dd2b389c9 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:38.454389 21297 tablet_service.cc:1511] Processing CreateTablet for tablet dfe505957770401e92725d10d579c64e (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.456276 21378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74cc231530744a91bbac7f6dd2b389c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.456508 21297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfe505957770401e92725d10d579c64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.461048 21303 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7654d07a19420cb4acceea93c8ad79 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.462342 21376 tablet_service.cc:1511] Processing CreateTablet for tablet dfe505957770401e92725d10d579c64e (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.464154 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfe505957770401e92725d10d579c64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.463845 21223 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7654d07a19420cb4acceea93c8ad79 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.465828 21223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7654d07a19420cb4acceea93c8ad79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.468549 21224 tablet_service.cc:1511] Processing CreateTablet for tablet dfe505957770401e92725d10d579c64e (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:38.470551 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfe505957770401e92725d10d579c64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.467521 21301 tablet_service.cc:1511] Processing CreateTablet for tablet 597bcc40ac0247ed855affc1fe0e3a19 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.468354 21303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7654d07a19420cb4acceea93c8ad79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.473908 21298 tablet_service.cc:1511] Processing CreateTablet for tablet 4aba9f5ed69242c586d84b02fa5a247b (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.475669 21298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aba9f5ed69242c586d84b02fa5a247b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.475780 21377 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7654d07a19420cb4acceea93c8ad79 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:38.476764 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597bcc40ac0247ed855affc1fe0e3a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.477520 21377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7654d07a19420cb4acceea93c8ad79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.479439 21374 tablet_service.cc:1511] Processing CreateTablet for tablet 597bcc40ac0247ed855affc1fe0e3a19 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.485374 21225 tablet_service.cc:1511] Processing CreateTablet for tablet 597bcc40ac0247ed855affc1fe0e3a19 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:38.489037 21228 tablet_service.cc:1511] Processing CreateTablet for tablet 4aba9f5ed69242c586d84b02fa5a247b (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.490962 21225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597bcc40ac0247ed855affc1fe0e3a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.492173 21375 tablet_service.cc:1511] Processing CreateTablet for tablet 4aba9f5ed69242c586d84b02fa5a247b (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:38.495357 21302 tablet_service.cc:1511] Processing CreateTablet for tablet f51082bf25994b80bf0923d3951de95c (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.497231 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51082bf25994b80bf0923d3951de95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.498394 21226 tablet_service.cc:1511] Processing CreateTablet for tablet f51082bf25994b80bf0923d3951de95c (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.485518 21374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 597bcc40ac0247ed855affc1fe0e3a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.501430 21375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aba9f5ed69242c586d84b02fa5a247b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.502725 21228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aba9f5ed69242c586d84b02fa5a247b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.530444 21372 tablet_service.cc:1511] Processing CreateTablet for tablet f51082bf25994b80bf0923d3951de95c (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:38.532543 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51082bf25994b80bf0923d3951de95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.532222 21227 tablet_service.cc:1511] Processing CreateTablet for tablet c791a1bc687c456d88cabea4e0bfae27 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.533886 21304 tablet_service.cc:1511] Processing CreateTablet for tablet c791a1bc687c456d88cabea4e0bfae27 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.535656 21304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c791a1bc687c456d88cabea4e0bfae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.525334 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f51082bf25994b80bf0923d3951de95c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.543915 21227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c791a1bc687c456d88cabea4e0bfae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.564292 21371 tablet_service.cc:1511] Processing CreateTablet for tablet c791a1bc687c456d88cabea4e0bfae27 (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:38.566263 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c791a1bc687c456d88cabea4e0bfae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.579429 21534 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:38.589848 21229 tablet_service.cc:1511] Processing CreateTablet for tablet 0257272195d64d82a8f20f125cfe59ad (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:38.592124 21302 tablet_service.cc:1511] Processing CreateTablet for tablet 0257272195d64d82a8f20f125cfe59ad (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:38.593843 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0257272195d64d82a8f20f125cfe59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.593400 21379 tablet_service.cc:1511] Processing CreateTablet for tablet 0257272195d64d82a8f20f125cfe59ad (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:38.595105 21229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0257272195d64d82a8f20f125cfe59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.599489 21230 tablet_service.cc:1511] Processing CreateTablet for tablet fc384aaab9dd45de9c6d07332cd2ccda (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:38.601330 21230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc384aaab9dd45de9c6d07332cd2ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.595105 21379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0257272195d64d82a8f20f125cfe59ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.629860 21535 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.673067 21535 tablet_bootstrap.cc:654] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.678790 21534 tablet_bootstrap.cc:654] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.678217 21304 tablet_service.cc:1511] Processing CreateTablet for tablet fc384aaab9dd45de9c6d07332cd2ccda (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:38.685598 21536 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.691805 21304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc384aaab9dd45de9c6d07332cd2ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.690994 21376 tablet_service.cc:1511] Processing CreateTablet for tablet fc384aaab9dd45de9c6d07332cd2ccda (DEFAULT_TABLE table=soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:38.694809 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc384aaab9dd45de9c6d07332cd2ccda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:38.698766 21536 tablet_bootstrap.cc:654] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.712761 21536 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.714002 21536 ts_tablet_manager.cc:1403] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.009s
I20260709 13:22:38.720644 21536 raft_consensus.cc:359] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.721779 21536 raft_consensus.cc:385] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.722208 21536 raft_consensus.cc:740] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.723389 21536 consensus_queue.cc:260] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.724756 21535 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:38.725576 21535 ts_tablet_manager.cc:1403] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.004s
I20260709 13:22:38.726858 21534 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:38.727660 21534 ts_tablet_manager.cc:1403] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.167s	user 0.015s	sys 0.011s
I20260709 13:22:38.733228 21534 raft_consensus.cc:359] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.734323 21534 raft_consensus.cc:385] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.733214 21535 raft_consensus.cc:359] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.734735 21534 raft_consensus.cc:740] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.734953 21535 raft_consensus.cc:385] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.735780 21535 raft_consensus.cc:740] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.735832 21534 consensus_queue.cc:260] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.739396 21534 ts_tablet_manager.cc:1434] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:22:38.739876 21536 ts_tablet_manager.cc:1434] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 13:22:38.740243 21535 consensus_queue.cc:260] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.744673 21536 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.752451 21534 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:38.753165 21536 tablet_bootstrap.cc:654] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.753017 21535 ts_tablet_manager.cc:1434] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:22:38.760327 21535 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.769263 21535 tablet_bootstrap.cc:654] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.783171 21534 tablet_bootstrap.cc:654] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.783569 21536 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.784430 21536 ts_tablet_manager.cc:1403] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.008s
I20260709 13:22:38.788105 21535 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:38.788969 21535 ts_tablet_manager.cc:1403] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.003s
I20260709 13:22:38.788412 21536 raft_consensus.cc:359] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.789650 21536 raft_consensus.cc:385] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.790032 21536 raft_consensus.cc:740] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.791057 21536 consensus_queue.cc:260] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.793895 21536 ts_tablet_manager.cc:1434] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:22:38.795253 21536 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.793605 21535 raft_consensus.cc:359] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.802708 21535 raft_consensus.cc:385] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.803301 21535 raft_consensus.cc:740] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.803962 21536 tablet_bootstrap.cc:654] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.804766 21535 consensus_queue.cc:260] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.814002 21535 ts_tablet_manager.cc:1434] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:22:38.815780 21535 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.817965 21536 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.818812 21536 ts_tablet_manager.cc:1403] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.024s	user 0.019s	sys 0.000s
I20260709 13:22:38.822839 21536 raft_consensus.cc:359] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.823879 21536 raft_consensus.cc:385] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.824251 21536 raft_consensus.cc:740] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.825239 21536 consensus_queue.cc:260] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.828339 21535 tablet_bootstrap.cc:654] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.828575 21536 ts_tablet_manager.cc:1434] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:22:38.829974 21536 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.838322 21536 tablet_bootstrap.cc:654] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.850481 21534 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:38.850665 21536 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.851052 21535 tablet_bootstrap.cc:492] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:38.851620 21536 ts_tablet_manager.cc:1403] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:22:38.851339 21534 ts_tablet_manager.cc:1403] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.099s	user 0.043s	sys 0.023s
I20260709 13:22:38.855391 21535 ts_tablet_manager.cc:1403] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.009s
I20260709 13:22:38.855752 21536 raft_consensus.cc:359] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.856721 21536 raft_consensus.cc:385] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.857151 21536 raft_consensus.cc:740] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.858255 21536 consensus_queue.cc:260] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.856271 21534 raft_consensus.cc:359] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.861717 21536 ts_tablet_manager.cc:1434] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:22:38.861922 21534 raft_consensus.cc:385] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.862088 21535 raft_consensus.cc:359] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.862689 21534 raft_consensus.cc:740] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.863164 21535 raft_consensus.cc:385] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.863298 21536 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.863556 21535 raft_consensus.cc:740] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.864161 21534 consensus_queue.cc:260] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.864418 21535 consensus_queue.cc:260] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.865803 21523 raft_consensus.cc:493] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.866449 21523 raft_consensus.cc:515] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.868988 21523 leader_election.cc:290] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:38.871587 21535 ts_tablet_manager.cc:1434] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:22:38.872627 21536 tablet_bootstrap.cc:654] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.873291 21535 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.877334 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
W20260709 13:22:38.881896 21281 leader_election.cc:343] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:38.871794 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:38.886812 21522 raft_consensus.cc:493] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.887576 21522 raft_consensus.cc:515] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
W20260709 13:22:38.888550 21281 leader_election.cc:343] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 85475c7436484601a066fc66945e140d (127.8.183.67:40029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:38.889171 21281 leader_election.cc:304] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:38.892015 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:38.892728 21522 leader_election.cc:290] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:38.893198 21534 ts_tablet_manager.cc:1434] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.005s
I20260709 13:22:38.893177 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
W20260709 13:22:38.895998 21354 leader_election.cc:343] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:38.897941 21355 leader_election.cc:343] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:38.898489 21355 leader_election.cc:304] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85475c7436484601a066fc66945e140d; no voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:38.898828 21535 tablet_bootstrap.cc:654] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.901338 21521 raft_consensus.cc:2749] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:38.903546 21522 raft_consensus.cc:2749] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:38.905441 21534 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:38.914506 21535 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:38.915360 21535 ts_tablet_manager.cc:1403] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.042s	user 0.023s	sys 0.004s
I20260709 13:22:38.917289 21536 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.918082 21536 ts_tablet_manager.cc:1403] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.055s	user 0.027s	sys 0.023s
I20260709 13:22:38.920006 21535 raft_consensus.cc:359] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.920980 21535 raft_consensus.cc:385] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.921324 21535 raft_consensus.cc:740] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.921758 21536 raft_consensus.cc:359] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.922729 21536 raft_consensus.cc:385] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.922233 21535 consensus_queue.cc:260] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.923090 21536 raft_consensus.cc:740] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.924088 21536 consensus_queue.cc:260] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.922214 21534 tablet_bootstrap.cc:654] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.927052 21536 ts_tablet_manager.cc:1434] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:22:38.932514 21536 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.928866 21535 ts_tablet_manager.cc:1434] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:38.934731 21535 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.936148 21534 tablet_bootstrap.cc:492] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:38.936945 21534 ts_tablet_manager.cc:1403] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.002s
I20260709 13:22:38.942065 21534 raft_consensus.cc:359] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.942193 21536 tablet_bootstrap.cc:654] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.943176 21534 raft_consensus.cc:385] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.943656 21534 raft_consensus.cc:740] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.944976 21535 tablet_bootstrap.cc:654] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.944813 21534 consensus_queue.cc:260] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.976959 21534 ts_tablet_manager.cc:1434] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:22:38.977349 21535 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:38.978060 21535 ts_tablet_manager.cc:1403] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.015s
I20260709 13:22:38.978328 21534 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:38.978412 21536 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:38.979262 21536 ts_tablet_manager.cc:1403] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.027s
I20260709 13:22:38.982259 21535 raft_consensus.cc:359] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.982924 21536 raft_consensus.cc:359] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.983446 21535 raft_consensus.cc:385] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.983891 21536 raft_consensus.cc:385] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:38.984000 21535 raft_consensus.cc:740] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.984391 21536 raft_consensus.cc:740] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:38.985353 21536 consensus_queue.cc:260] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.985432 21535 consensus_queue.cc:260] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.986442 21522 raft_consensus.cc:493] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.987171 21534 tablet_bootstrap.cc:654] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:38.989240 21535 ts_tablet_manager.cc:1434] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.001s
I20260709 13:22:38.990868 21535 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:38.987222 21522 raft_consensus.cc:515] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:38.996063 21536 ts_tablet_manager.cc:1434] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:22:38.997727 21536 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:38.999500 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.002611 21535 tablet_bootstrap.cc:654] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.003328 21438 raft_consensus.cc:493] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.004017 21438 raft_consensus.cc:515] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.007146 21536 tablet_bootstrap.cc:654] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:39.008319 21355 leader_election.cc:343] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.009537 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfe505957770401e92725d10d579c64e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.010519 21313 raft_consensus.cc:2468] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.010598 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfe505957770401e92725d10d579c64e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.011817 21388 raft_consensus.cc:2468] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.014158 21523 raft_consensus.cc:493] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.015228 21523 raft_consensus.cc:515] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.021492 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.022365 21239 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.014267 21205 leader_election.cc:304] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.026592 21536 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:39.026961 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.027405 21536 ts_tablet_manager.cc:1403] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.007s
W20260709 13:22:39.028829 21354 leader_election.cc:343] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.029423 21354 leader_election.cc:304] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85475c7436484601a066fc66945e140d; no voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.030560 21551 raft_consensus.cc:2749] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:39.030189 21281 leader_election.cc:304] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.030965 21522 leader_election.cc:290] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.031868 21521 raft_consensus.cc:2804] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.032552 21521 raft_consensus.cc:493] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.033187 21521 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.032745 21536 raft_consensus.cc:359] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.033859 21536 raft_consensus.cc:385] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.034231 21536 raft_consensus.cc:740] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.035216 21536 consensus_queue.cc:260] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.036283 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.037633 21388 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.038370 21536 ts_tablet_manager.cc:1434] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:22:39.039953 21536 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:39.041057 21438 leader_election.cc:290] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.042246 21438 raft_consensus.cc:2804] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.042927 21438 raft_consensus.cc:493] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.043588 21438 raft_consensus.cc:3060] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.051890 21523 leader_election.cc:290] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.054260 21536 tablet_bootstrap.cc:654] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.063078 21534 tablet_bootstrap.cc:492] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.063786 21534 ts_tablet_manager.cc:1403] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.086s	user 0.005s	sys 0.022s
W20260709 13:22:39.066327 21205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:46747, user_credentials={real_user=slave}} blocked reactor thread for 52391.2us
I20260709 13:22:39.067577 21534 raft_consensus.cc:359] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.068574 21534 raft_consensus.cc:385] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.068953 21534 raft_consensus.cc:740] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.074049 21536 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:39.074770 21536 ts_tablet_manager.cc:1403] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.003s
I20260709 13:22:39.071740 21534 consensus_queue.cc:260] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.077562 21534 ts_tablet_manager.cc:1434] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.007s
I20260709 13:22:39.078313 21536 raft_consensus.cc:359] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.078963 21534 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:39.079252 21536 raft_consensus.cc:385] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.079684 21536 raft_consensus.cc:740] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.080549 21536 consensus_queue.cc:260] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.083087 21536 ts_tablet_manager.cc:1434] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:22:39.084617 21536 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: Bootstrap starting.
I20260709 13:22:39.091965 21536 tablet_bootstrap.cc:654] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.093250 21534 tablet_bootstrap.cc:654] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.100876 21523 raft_consensus.cc:493] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.102017 21523 raft_consensus.cc:515] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.105400 21536 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: No bootstrap required, opened a new log
I20260709 13:22:39.106319 21536 ts_tablet_manager.cc:1403] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 13:22:39.106825 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.108122 21239 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.110611 21281 leader_election.cc:304] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
W20260709 13:22:39.120184 21205 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.66:46747, user_credentials={real_user=slave}} blocked reactor thread for 45177.1us
I20260709 13:22:39.125770 21438 raft_consensus.cc:515] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.128993 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.135417 21388 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.129385 21535 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:39.127972 21561 raft_consensus.cc:2804] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.110584 21536 raft_consensus.cc:359] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.123250 21521 raft_consensus.cc:515] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.125592 21534 tablet_bootstrap.cc:492] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.141255 21536 raft_consensus.cc:385] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.139849 21438 leader_election.cc:290] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.119071 21523 leader_election.cc:290] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.140558 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfe505957770401e92725d10d579c64e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.141788 21561 raft_consensus.cc:493] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.141649 21536 raft_consensus.cc:740] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.142707 21561 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.141835 21534 ts_tablet_manager.cc:1403] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.021s
I20260709 13:22:39.143404 21536 consensus_queue.cc:260] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.146739 21536 ts_tablet_manager.cc:1434] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.004s
I20260709 13:22:39.141652 21535 ts_tablet_manager.cc:1403] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.151s	user 0.051s	sys 0.069s
I20260709 13:22:39.142874 21388 raft_consensus.cc:3060] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.158067 21535 raft_consensus.cc:359] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.159142 21535 raft_consensus.cc:385] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.159483 21535 raft_consensus.cc:740] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.160000 21534 raft_consensus.cc:359] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.161444 21534 raft_consensus.cc:385] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.160362 21535 consensus_queue.cc:260] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.161713 21561 raft_consensus.cc:515] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.165290 21535 ts_tablet_manager.cc:1434] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.003s
I20260709 13:22:39.166594 21535 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:39.151649 21521 leader_election.cc:290] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.167335 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.168138 21239 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.161945 21534 raft_consensus.cc:740] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.171903 21534 consensus_queue.cc:260] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.173915 21535 tablet_bootstrap.cc:654] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.178411 21239 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.180521 21559 raft_consensus.cc:493] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.180717 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.182826 21534 ts_tablet_manager.cc:1434] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.001s
I20260709 13:22:39.187489 21389 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.188880 21534 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:39.182479 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfe505957770401e92725d10d579c64e" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.184614 21388 raft_consensus.cc:2468] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.191746 21313 raft_consensus.cc:3060] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.193058 21206 leader_election.cc:304] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.194249 21438 raft_consensus.cc:2804] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.194841 21438 raft_consensus.cc:697] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:39.191520 21559 raft_consensus.cc:515] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.195935 21438 consensus_queue.cc:237] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.198138 21559 leader_election.cc:290] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.199177 21534 tablet_bootstrap.cc:654] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.224344 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.225634 21388 raft_consensus.cc:2468] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.232892 21281 leader_election.cc:304] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.234769 21555 raft_consensus.cc:493] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.243640 21555 raft_consensus.cc:515] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.243855 21137 catalog_manager.cc:5697] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.250955 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.252137 21388 raft_consensus.cc:2468] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.254276 21555 leader_election.cc:290] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.255546 21206 leader_election.cc:304] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.259821 21555 raft_consensus.cc:2804] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.260674 21555 raft_consensus.cc:493] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.261554 21555 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.236217 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.266018 21522 raft_consensus.cc:493] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.266628 21239 raft_consensus.cc:2468] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 0.
I20260709 13:22:39.266700 21522 raft_consensus.cc:515] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.268849 21521 raft_consensus.cc:2804] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.269698 21521 raft_consensus.cc:697] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:39.270021 21281 leader_election.cc:304] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.271041 21521 consensus_queue.cc:237] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.272754 21314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.273790 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.274617 21239 raft_consensus.cc:2468] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:39.271075 21523 raft_consensus.cc:2804] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:22:39.275183 21354 leader_election.cc:343] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.275765 21523 raft_consensus.cc:493] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.272774 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.276279 21355 leader_election.cc:304] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.276412 21523 raft_consensus.cc:3060] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.277355 21522 leader_election.cc:290] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.277415 21551 raft_consensus.cc:2804] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.277985 21551 raft_consensus.cc:493] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.278360 21551 raft_consensus.cc:3060] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.291666 21561 leader_election.cc:290] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.293126 21389 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.295536 21313 raft_consensus.cc:2468] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
W20260709 13:22:39.306208 21205 leader_election.cc:343] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.295531 21523 raft_consensus.cc:515] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.311297 21137 catalog_manager.cc:5697] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.322026 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.322875 21239 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.325371 21535 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:39.326088 21535 ts_tablet_manager.cc:1403] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.160s	user 0.038s	sys 0.066s
I20260709 13:22:39.328883 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.329834 21389 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.332074 21534 tablet_bootstrap.cc:492] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.332894 21534 ts_tablet_manager.cc:1403] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.144s	user 0.037s	sys 0.059s
I20260709 13:22:39.330324 21535 raft_consensus.cc:359] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.337102 21535 raft_consensus.cc:385] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.337520 21535 raft_consensus.cc:740] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.338104 21534 raft_consensus.cc:359] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.338421 21535 consensus_queue.cc:260] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.339286 21534 raft_consensus.cc:385] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.339725 21534 raft_consensus.cc:740] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.340785 21534 consensus_queue.cc:260] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.341719 21551 raft_consensus.cc:515] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.341550 21535 ts_tablet_manager.cc:1434] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:22:39.343990 21535 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:39.344666 21551 leader_election.cc:290] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.348289 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.348433 21238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.349588 21238 raft_consensus.cc:3060] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:39.350956 21354 leader_election.cc:343] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.354188 21240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.356737 21535 tablet_bootstrap.cc:654] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.359591 21534 ts_tablet_manager.cc:1434] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.003s
I20260709 13:22:39.360671 21240 raft_consensus.cc:3060] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.360939 21534 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:39.341173 21523 leader_election.cc:290] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.373101 21388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c791a1bc687c456d88cabea4e0bfae27" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.373993 21388 raft_consensus.cc:3060] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.379801 21238 raft_consensus.cc:2468] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:39.380964 21534 tablet_bootstrap.cc:654] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.382289 21355 leader_election.cc:304] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.384187 21522 raft_consensus.cc:493] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.384994 21522 raft_consensus.cc:515] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.391741 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.392701 21312 raft_consensus.cc:2468] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:39.370954 21240 raft_consensus.cc:2468] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.396385 21354 leader_election.cc:304] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.408061 21522 leader_election.cc:290] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.409991 21560 raft_consensus.cc:493] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.395035 21388 raft_consensus.cc:2468] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.410902 21560 raft_consensus.cc:515] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.398523 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.415458 21237 raft_consensus.cc:2468] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:39.396790 21534 tablet_bootstrap.cc:492] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.417287 21534 ts_tablet_manager.cc:1403] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.008s
I20260709 13:22:39.417686 21560 leader_election.cc:290] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.418996 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.399998 21551 raft_consensus.cc:2804] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.420820 21551 raft_consensus.cc:697] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:39.421558 21522 raft_consensus.cc:2804] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.422439 21534 raft_consensus.cc:359] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.423357 21534 raft_consensus.cc:385] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.422674 21387 raft_consensus.cc:2468] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.422786 21551 consensus_queue.cc:237] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.423982 21522 raft_consensus.cc:493] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.405639 21389 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.427479 21534 raft_consensus.cc:740] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.403573 21555 raft_consensus.cc:515] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.427342 21206 leader_election.cc:304] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
W20260709 13:22:39.428406 21186 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:39.428905 21535 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:39.429243 21281 leader_election.cc:304] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.429519 21535 ts_tablet_manager.cc:1403] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.086s	user 0.041s	sys 0.033s
I20260709 13:22:39.424696 21522 raft_consensus.cc:3060] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.429363 21560 raft_consensus.cc:2804] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.431726 21560 raft_consensus.cc:493] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.431845 21555 leader_election.cc:290] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.432241 21560 raft_consensus.cc:3060] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.434636 21239 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 1.
I20260709 13:22:39.434370 21521 raft_consensus.cc:2804] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.439241 21535 raft_consensus.cc:359] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.432953 21534 consensus_queue.cc:260] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
W20260709 13:22:39.440328 21354 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:46747, user_credentials={real_user=slave}} blocked reactor thread for 44164.3us
I20260709 13:22:39.443365 21535 raft_consensus.cc:385] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.443699 21535 raft_consensus.cc:740] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.442579 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.445329 21387 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.450162 21535 consensus_queue.cc:260] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.454933 21535 ts_tablet_manager.cc:1434] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:22:39.456702 21534 ts_tablet_manager.cc:1434] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.001s
I20260709 13:22:39.458581 21534 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:39.459219 21535 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: Bootstrap starting.
I20260709 13:22:39.471972 21551 raft_consensus.cc:493] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.473676 21551 raft_consensus.cc:515] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.451931 21521 raft_consensus.cc:697] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:39.477671 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.478891 21535 tablet_bootstrap.cc:654] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.480235 21312 raft_consensus.cc:2468] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.482880 21521 consensus_queue.cc:237] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.483942 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.488876 21312 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.485308 21551 leader_election.cc:290] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.485070 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
W20260709 13:22:39.491663 21355 leader_election.cc:343] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.496407 21137 catalog_manager.cc:5697] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:39.494567 21281 leader_election.cc:304] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:39.504005 21523 raft_consensus.cc:2804] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.504674 21523 raft_consensus.cc:697] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:39.505663 21523 consensus_queue.cc:237] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.509817 21387 raft_consensus.cc:2468] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.511232 21312 raft_consensus.cc:2468] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.527868 21534 tablet_bootstrap.cc:654] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.523208 21560 raft_consensus.cc:515] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.538496 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
I20260709 13:22:39.539449 21313 raft_consensus.cc:2468] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 0.
I20260709 13:22:39.540158 21560 leader_election.cc:290] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.540500 21206 leader_election.cc:304] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.541570 21560 raft_consensus.cc:2804] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.541074 21137 catalog_manager.cc:5697] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.542164 21560 raft_consensus.cc:697] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:39.542900 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.544353 21560 consensus_queue.cc:237] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.547902 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.548708 21313 raft_consensus.cc:3060] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.547720 21535 tablet_bootstrap.cc:492] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: No bootstrap required, opened a new log
I20260709 13:22:39.550829 21535 ts_tablet_manager.cc:1403] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.017s
I20260709 13:22:39.559252 21354 leader_election.cc:304] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:39.557886 21522 raft_consensus.cc:515] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.574896 21387 raft_consensus.cc:2393] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9e4db2af5b894498a39e7996de5f4420 in current term 1: Already voted for candidate 85475c7436484601a066fc66945e140d in this term.
I20260709 13:22:39.582516 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.583627 21239 raft_consensus.cc:2393] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 85475c7436484601a066fc66945e140d in current term 1: Already voted for candidate 9e4db2af5b894498a39e7996de5f4420 in this term.
I20260709 13:22:39.584883 21522 leader_election.cc:290] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.577270 21575 raft_consensus.cc:2804] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.585929 21575 raft_consensus.cc:493] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.586411 21575 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.587317 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "597bcc40ac0247ed855affc1fe0e3a19" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.587746 21313 raft_consensus.cc:2468] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.589840 21354 leader_election.cc:304] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85475c7436484601a066fc66945e140d; no voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.589901 21205 leader_election.cc:304] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9e4db2af5b894498a39e7996de5f4420, c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d
I20260709 13:22:39.592294 21438 raft_consensus.cc:2804] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.592305 21555 raft_consensus.cc:493] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.594444 21522 raft_consensus.cc:2749] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:39.594091 21555 raft_consensus.cc:515] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.598505 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.599923 21387 raft_consensus.cc:2468] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 0.
I20260709 13:22:39.601527 21206 leader_election.cc:304] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.598789 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5" is_pre_election: true
W20260709 13:22:39.604501 21205 leader_election.cc:343] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:39.599180 21555 leader_election.cc:290] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.605773 21534 tablet_bootstrap.cc:492] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.605892 21555 raft_consensus.cc:2804] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:39.606823 21534 ts_tablet_manager.cc:1403] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.149s	user 0.030s	sys 0.010s
I20260709 13:22:39.606879 21555 raft_consensus.cc:493] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:39.608779 21555 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.593017 21535 raft_consensus.cc:359] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.609997 21535 raft_consensus.cc:385] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.610394 21535 raft_consensus.cc:740] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.597802 21137 catalog_manager.cc:5697] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 1 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.612913 21438 raft_consensus.cc:697] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:39.611521 21535 consensus_queue.cc:260] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.614255 21438 consensus_queue.cc:237] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.618237 21575 raft_consensus.cc:515] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.623978 21535 ts_tablet_manager.cc:1434] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 13:22:39.625052 21555 raft_consensus.cc:515] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.627755 21555 leader_election.cc:290] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.628523 21534 raft_consensus.cc:359] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.629635 21534 raft_consensus.cc:385] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.629869 21534 raft_consensus.cc:740] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.632213 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.633217 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:39.634089 21387 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.635032 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:39.635839 21239 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.634225 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:39.637805 21313 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.642956 21135 catalog_manager.cc:5697] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.644731 21387 raft_consensus.cc:2468] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.632684 21575 leader_election.cc:290] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:39.631214 21534 consensus_queue.cc:260] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.648257 21312 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.650170 21534 ts_tablet_manager.cc:1434] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
I20260709 13:22:39.651928 21206 leader_election.cc:304] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.652572 21239 raft_consensus.cc:2468] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:39.653879 21534 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: Bootstrap starting.
I20260709 13:22:39.653908 21555 raft_consensus.cc:2804] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.653893 21355 leader_election.cc:304] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:39.655078 21555 raft_consensus.cc:697] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:39.655954 21575 raft_consensus.cc:2804] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:39.656215 21555 consensus_queue.cc:237] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.657721 21438 consensus_queue.cc:1048] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.661216 21575 raft_consensus.cc:697] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:39.661923 21523 consensus_queue.cc:1048] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.662737 21534 tablet_bootstrap.cc:654] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:39.664381 21575 consensus_queue.cc:237] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.672838 21312 raft_consensus.cc:2468] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 1.
I20260709 13:22:39.694761 21438 consensus_queue.cc:1048] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.712708 21534 tablet_bootstrap.cc:492] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: No bootstrap required, opened a new log
I20260709 13:22:39.713766 21534 ts_tablet_manager.cc:1403] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.016s
I20260709 13:22:39.721824 21534 raft_consensus.cc:359] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.722685 21534 raft_consensus.cc:385] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:39.723014 21534 raft_consensus.cc:740] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Initialized, Role: FOLLOWER
I20260709 13:22:39.723896 21534 consensus_queue.cc:260] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.726496 21534 ts_tablet_manager.cc:1434] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:22:39.732452 21137 catalog_manager.cc:5697] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 1 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:39.741072 21523 consensus_queue.cc:1048] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:22:39.757300 21521 raft_consensus.cc:493] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.757885 21521 raft_consensus.cc:515] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:39.760339 21521 leader_election.cc:290] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:39.761945 21313 raft_consensus.cc:2468] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 1.
I20260709 13:22:39.762219 21387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85475c7436484601a066fc66945e140d" is_pre_election: true
I20260709 13:22:39.773502 21135 catalog_manager.cc:5697] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.777611 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc384aaab9dd45de9c6d07332cd2ccda" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e4db2af5b894498a39e7996de5f4420" is_pre_election: true
I20260709 13:22:39.778597 21239 raft_consensus.cc:2393] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c1674e5c97eb461c98fb5d863246c0f5 in current term 1: Already voted for candidate 85475c7436484601a066fc66945e140d in this term.
I20260709 13:22:39.780192 21281 leader_election.cc:304] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c1674e5c97eb461c98fb5d863246c0f5; no voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:39.781847 21523 raft_consensus.cc:3060] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:39.776130 21135 catalog_manager.cc:5697] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 1 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.790984 21523 raft_consensus.cc:2749] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:39.864168 21575 consensus_queue.cc:1048] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:39.879796 21522 consensus_queue.cc:1048] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.904560 21315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
mode: GRACEFUL
new_leader_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:51302
I20260709 13:22:39.905279 21315 raft_consensus.cc:606] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Received request to transfer leadership to TS 9e4db2af5b894498a39e7996de5f4420
I20260709 13:22:39.920873 21523 consensus_queue.cc:1048] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.921705 21387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c"
dest_uuid: "85475c7436484601a066fc66945e140d"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:55266
I20260709 13:22:39.922587 21387 raft_consensus.cc:606] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.938977 21521 consensus_queue.cc:1048] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.945755 21521 consensus_queue.cc:1048] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:39.947250 21238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:36500
I20260709 13:22:39.948130 21238 raft_consensus.cc:606] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS c1674e5c97eb461c98fb5d863246c0f5
I20260709 13:22:39.961478 21521 consensus_queue.cc:1048] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.041098 21582 consensus_queue.cc:1048] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.047679 21555 consensus_queue.cc:1048] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.103276 21582 consensus_queue.cc:1048] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.119663 21582 consensus_queue.cc:1048] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.194784 21522 consensus_queue.cc:1048] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.212987 21575 consensus_queue.cc:1048] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.222558 21574 raft_consensus.cc:993] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:40.223222 21575 raft_consensus.cc:1081] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:40.225298 21315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51286
I20260709 13:22:40.226085 21315 raft_consensus.cc:493] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.226512 21315 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.234570 21582 consensus_queue.cc:1048] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:40.236440 21315 raft_consensus.cc:515] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.238881 21315 leader_election.cc:290] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:40.241312 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:40.242193 21389 raft_consensus.cc:3055] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.242569 21389 raft_consensus.cc:740] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:40.244338 21238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f51082bf25994b80bf0923d3951de95c" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:40.245204 21238 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.249459 21389 consensus_queue.cc:260] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.252669 21389 raft_consensus.cc:3060] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.256146 21238 raft_consensus.cc:2468] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:40.261543 21581 consensus_queue.cc:1048] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:40.262820 21389 raft_consensus.cc:2468] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:40.264358 21281 leader_election.cc:304] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:40.265573 21523 raft_consensus.cc:2804] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.266181 21523 raft_consensus.cc:697] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:40.267755 21523 consensus_queue.cc:237] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.284862 21560 raft_consensus.cc:993] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:40.285539 21581 raft_consensus.cc:1081] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer c1674e5c97eb461c98fb5d863246c0f5 to start an election
I20260709 13:22:40.288398 21313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
 from {username='slave'} at 127.0.0.1:51280
I20260709 13:22:40.289100 21313 raft_consensus.cc:493] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.289592 21313 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.291186 21135 catalog_manager.cc:5697] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 85475c7436484601a066fc66945e140d (127.8.183.67) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.300161 21313 raft_consensus.cc:515] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.300834 21561 raft_consensus.cc:993] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: : Instructing follower 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:40.301607 21561 raft_consensus.cc:1081] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Signalling peer 9e4db2af5b894498a39e7996de5f4420 to start an election
I20260709 13:22:40.302966 21313 leader_election.cc:290] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:40.305490 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:40.305804 21238 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
 from {username='slave'} at 127.0.0.1:36496
I20260709 13:22:40.306176 21389 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.306844 21238 raft_consensus.cc:493] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.307487 21238 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.311805 21239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0257272195d64d82a8f20f125cfe59ad" candidate_uuid: "c1674e5c97eb461c98fb5d863246c0f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:40.313097 21239 raft_consensus.cc:3055] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.313483 21239 raft_consensus.cc:740] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:40.313374 21389 raft_consensus.cc:2468] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:40.314652 21239 consensus_queue.cc:260] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.315460 21281 leader_election.cc:304] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:40.316284 21239 raft_consensus.cc:3060] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.316587 21523 raft_consensus.cc:2804] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.317247 21523 raft_consensus.cc:697] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Leader. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:40.318251 21523 consensus_queue.cc:237] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.322907 21238 raft_consensus.cc:515] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.324290 21239 raft_consensus.cc:2468] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1674e5c97eb461c98fb5d863246c0f5 in term 2.
I20260709 13:22:40.325726 21238 leader_election.cc:290] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Requested vote from peers c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747), 85475c7436484601a066fc66945e140d (127.8.183.67:40029)
I20260709 13:22:40.328466 21389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "85475c7436484601a066fc66945e140d"
I20260709 13:22:40.329133 21389 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.330097 21313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7654d07a19420cb4acceea93c8ad79" candidate_uuid: "9e4db2af5b894498a39e7996de5f4420" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:40.330894 21313 raft_consensus.cc:3055] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.331548 21313 raft_consensus.cc:740] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:40.332566 21313 consensus_queue.cc:260] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.334092 21313 raft_consensus.cc:3060] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.333753 21135 catalog_manager.cc:5697] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66). New cstate: current_term: 2 leader_uuid: "c1674e5c97eb461c98fb5d863246c0f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.338129 21389 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:40.339932 21206 leader_election.cc:304] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:40.341666 21581 raft_consensus.cc:2804] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.342281 21581 raft_consensus.cc:697] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Becoming Leader. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:40.343336 21581 consensus_queue.cc:237] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:40.344982 21313 raft_consensus.cc:2468] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e4db2af5b894498a39e7996de5f4420 in term 2.
I20260709 13:22:40.354028 21135 catalog_manager.cc:5697] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 reported cstate change: term changed from 1 to 2, leader changed from c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66) to 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65). New cstate: current_term: 2 leader_uuid: "9e4db2af5b894498a39e7996de5f4420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.680964 21238 raft_consensus.cc:1275] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.682637 21561 consensus_queue.cc:1048] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.693531 21387 raft_consensus.cc:1275] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.698256 21521 consensus_queue.cc:1048] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.735239 21313 raft_consensus.cc:1275] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.736841 21560 consensus_queue.cc:1048] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.748227 21387 raft_consensus.cc:1275] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer 9e4db2af5b894498a39e7996de5f4420: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.751569 21582 consensus_queue.cc:1048] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.777004 21387 raft_consensus.cc:1275] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.778636 21521 consensus_queue.cc:1048] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.792001 21238 raft_consensus.cc:1275] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer c1674e5c97eb461c98fb5d863246c0f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.794667 21561 consensus_queue.cc:1048] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:42.961495 21135 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38878:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:42.985734 21135 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38878:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:43.031932  8925 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:22:43.215312 21315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
mode: GRACEFUL
new_leader_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:52284
I20260709 13:22:43.216432 21315 raft_consensus.cc:606] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Received request to transfer leadership to TS 85475c7436484601a066fc66945e140d
I20260709 13:22:43.302735 21659 tablet.cc:1724] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.338474 21655 log.cc:927] Time spent T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Append to log took a long time: real 0.054s	user 0.004s	sys 0.000s
W20260709 13:22:43.345918 21658 log.cc:927] Time spent T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:22:43.415486 21663 tablet.cc:1724] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.439841 21341 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:f51082bf25994b80bf0923d3951de95c)
W20260709 13:22:43.451795 21341 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:f51082bf25994b80bf0923d3951de95c)
W20260709 13:22:43.466480 21341 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:f51082bf25994b80bf0923d3951de95c)
W20260709 13:22:43.487480 21281 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40029, user_credentials={real_user=slave}} blocked reactor thread for 76742.5us
I20260709 13:22:43.503687 21673 tablet.cc:1724] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.574028 21682 tablet.cc:1724] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.583199 21669 log.cc:927] Time spent T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:22:43.588307 21668 log.cc:927] Time spent T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
I20260709 13:22:43.588822 21647 raft_consensus.cc:993] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: : Instructing follower 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.589598 21647 raft_consensus.cc:1081] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Signalling peer 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.592464 21680 tablet.cc:1724] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.560523 21675 tablet.cc:1724] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.530089 21670 tablet.cc:1724] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.571707 21679 tablet.cc:1724] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.554250 21678 tablet.cc:1724] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.618635 21659 tablet.cc:1724] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.633626 21663 tablet.cc:1724] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.635551 21237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:43284
I20260709 13:22:43.636454 21237 raft_consensus.cc:606] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS 85475c7436484601a066fc66945e140d
I20260709 13:22:43.641911  8925 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:22:43.642870  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:43.648335 21663 tablet.cc:1724] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.670111 21670 tablet.cc:1724] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.706241 21682 tablet.cc:1724] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.706983 21670 tablet.cc:1724] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:43.721613 21281 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:43775, user_credentials={real_user=slave}} blocked reactor thread for 183088us
I20260709 13:22:43.730403 21683 tablet.cc:1724] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.722599 21237 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409"
dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
mode: GRACEFUL
new_leader_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:43284
I20260709 13:22:43.739880 21685 tablet.cc:1724] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.757179 21389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b"
dest_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:22:43.757835 21389 raft_consensus.cc:493] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.758108 21389 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.671047 21675 tablet.cc:1724] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.750839 21237 raft_consensus.cc:606] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Received request to transfer leadership to TS 85475c7436484601a066fc66945e140d
I20260709 13:22:43.777132 21678 tablet.cc:1724] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.780120 21313 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a"
dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
mode: GRACEFUL
new_leader_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:52284
I20260709 13:22:43.781059 21313 raft_consensus.cc:606] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Received request to transfer leadership to TS 85475c7436484601a066fc66945e140d
I20260709 13:22:43.783030  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:43.783987  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:43.783839 21679 tablet.cc:1724] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.786528 21681 tablet.cc:1724] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.800113 21389 raft_consensus.cc:515] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.804363 21659 tablet.cc:1724] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.813046 21682 tablet.cc:1724] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.817744 21389 leader_election.cc:290] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:43.818960 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:43.819772 21237 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.753137 21663 tablet.cc:1724] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.822927 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aba9f5ed69242c586d84b02fa5a247b" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:43.823770 21311 raft_consensus.cc:3055] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:43.824149 21311 raft_consensus.cc:740] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:43.825070 21311 consensus_queue.cc:260] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.826263 21311 raft_consensus.cc:3060] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.827234 21649 raft_consensus.cc:993] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.827876 21649 raft_consensus.cc:1081] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.836701 21389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9"
dest_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:55256
I20260709 13:22:43.840422 21389 raft_consensus.cc:493] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.840958 21389 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.838124 21683 tablet.cc:1724] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.819782 21680 tablet.cc:1724] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.861351 21679 tablet.cc:1724] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:43.857605 21237 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.848850 21311 raft_consensus.cc:2468] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.865355 21355 leader_election.cc:304] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:43.866743 21648 raft_consensus.cc:2804] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:43.867353 21648 raft_consensus.cc:697] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:43.868320 21648 consensus_queue.cc:237] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.881486 21389 raft_consensus.cc:515] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.885746 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:43.886471 21237 raft_consensus.cc:3055] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:43.886812 21237 raft_consensus.cc:740] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:43.887830 21237 consensus_queue.cc:260] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.889232 21237 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.895875 21389 leader_election.cc:290] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:43.907148 21647 raft_consensus.cc:993] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: : Instructing follower 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.907660 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74cc231530744a91bbac7f6dd2b389c9" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:43.907745 21647 raft_consensus.cc:1081] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Signalling peer 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.907707 21237 raft_consensus.cc:2468] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.910632 21355 leader_election.cc:304] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:43.912247 21648 raft_consensus.cc:2804] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:43.910671 21389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a"
dest_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:55258
I20260709 13:22:43.913975 21389 raft_consensus.cc:493] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.914418 21389 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.913070 21648 raft_consensus.cc:697] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:43.916147 21648 consensus_queue.cc:237] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.919181 21136 catalog_manager.cc:5697] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 1 to 2, leader changed from c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66) to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 2 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.925206 21389 raft_consensus.cc:515] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.928505 21389 leader_election.cc:290] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 3 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:43.929927 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:43.930917 21237 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.929988 21315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffeb6d2b8c944c2ba0bd0e7a06fe6a3a" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:43.932062 21315 raft_consensus.cc:3055] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:43.932399 21315 raft_consensus.cc:740] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c1674e5c97eb461c98fb5d863246c0f5, State: Running, Role: LEADER
I20260709 13:22:43.908512 21311 raft_consensus.cc:3060] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.933382 21315 consensus_queue.cc:260] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.935464 21315 raft_consensus.cc:3060] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:43.944222 21311 raft_consensus.cc:2468] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.944222 21237 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 3.
I20260709 13:22:43.945793 21315 raft_consensus.cc:2468] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 3.
I20260709 13:22:43.946393 21355 leader_election.cc:304] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, 9e4db2af5b894498a39e7996de5f4420; no voters: 
I20260709 13:22:43.947516 21648 raft_consensus.cc:2804] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:43.948101 21648 raft_consensus.cc:697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 3 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:43.948989 21648 consensus_queue.cc:237] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.950361 21135 catalog_manager.cc:6242] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: soft_deleted_table [id=b7d5506f49c040e6841c5577dfef13dd] alter complete (version 1)
I20260709 13:22:43.957047 21582 raft_consensus.cc:993] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: : Instructing follower 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.957718 21582 raft_consensus.cc:1081] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Signalling peer 85475c7436484601a066fc66945e140d to start an election
I20260709 13:22:43.961159 21389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409"
dest_uuid: "85475c7436484601a066fc66945e140d"
 from {username='slave'} at 127.0.0.1:55256
I20260709 13:22:43.961944 21389 raft_consensus.cc:493] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:43.962376 21389 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.972996 21389 raft_consensus.cc:515] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.976488 21389 leader_election.cc:290] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Requested vote from peers 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775), c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747)
I20260709 13:22:43.977540 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9e4db2af5b894498a39e7996de5f4420"
I20260709 13:22:43.978703 21237 raft_consensus.cc:3055] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:43.977746 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ecc044526df46e9bcb8a007ec2d0409" candidate_uuid: "85475c7436484601a066fc66945e140d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c1674e5c97eb461c98fb5d863246c0f5"
I20260709 13:22:43.979157 21237 raft_consensus.cc:740] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9e4db2af5b894498a39e7996de5f4420, State: Running, Role: LEADER
I20260709 13:22:43.979394 21311 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.980286 21237 consensus_queue.cc:260] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:43.979540 21135 catalog_manager.cc:5697] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 2 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.981899 21237 raft_consensus.cc:3060] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:43.981865 21135 catalog_manager.cc:5697] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 2 to 3, leader changed from c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66) to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 3 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.989916 21311 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.991928 21354 leader_election.cc:304] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85475c7436484601a066fc66945e140d, c1674e5c97eb461c98fb5d863246c0f5; no voters: 
I20260709 13:22:43.993091 21575 raft_consensus.cc:2804] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:43.993781 21575 raft_consensus.cc:697] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Becoming Leader. State: Replica: 85475c7436484601a066fc66945e140d, State: Running, Role: LEADER
I20260709 13:22:43.993108 21237 raft_consensus.cc:2468] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85475c7436484601a066fc66945e140d in term 2.
I20260709 13:22:43.994702 21575 consensus_queue.cc:237] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } }
I20260709 13:22:44.005362 21135 catalog_manager.cc:5697] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d reported cstate change: term changed from 1 to 2, leader changed from 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65) to 85475c7436484601a066fc66945e140d (127.8.183.67). New cstate: current_term: 2 leader_uuid: "85475c7436484601a066fc66945e140d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85475c7436484601a066fc66945e140d" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40029 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:44.365737 21237 raft_consensus.cc:1275] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 3 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:44.367511 21575 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:44.376914 21237 raft_consensus.cc:1275] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:44.379180 21575 consensus_queue.cc:1048] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:44.384855 21311 raft_consensus.cc:1275] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 3 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:44.387391 21694 consensus_queue.cc:1048] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:44.402485 21315 raft_consensus.cc:1275] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:44.404992 21694 consensus_queue.cc:1048] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:22:44.465008 21237 raft_consensus.cc:1275] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:44.467229 21698 consensus_queue.cc:1048] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:44.472038 21311 raft_consensus.cc:1275] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:44.472537 21315 raft_consensus.cc:1275] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:22:44.473866 21575 consensus_queue.cc:1048] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:44.474609 21694 consensus_queue.cc:1048] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1674e5c97eb461c98fb5d863246c0f5" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 46747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:44.501184 21237 raft_consensus.cc:1275] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Refusing update from remote peer 85475c7436484601a066fc66945e140d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:44.503285 21575 consensus_queue.cc:1048] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e4db2af5b894498a39e7996de5f4420" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:44.870031  8925 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:22:44.875622  8925 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:22:44.876137  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:44.883320  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:22:44.884141  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:45.924942  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
W20260709 13:22:45.926837 21281 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20260709 13:22:45.934808 21281 consensus_peers.cc:597] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:45.968637 21281 consensus_peers.cc:597] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.973618  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:45.974772  8925 tablet_replica.cc:333] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.975808  8925 raft_consensus.cc:2243] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.976938  8925 raft_consensus.cc:2272] T 597bcc40ac0247ed855affc1fe0e3a19 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.979538  8925 tablet_replica.cc:333] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.980321  8925 raft_consensus.cc:2243] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.980914  8925 raft_consensus.cc:2272] T c791a1bc687c456d88cabea4e0bfae27 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.983104  8925 tablet_replica.cc:333] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.983901  8925 raft_consensus.cc:2243] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.984915  8925 raft_consensus.cc:2272] T dfe505957770401e92725d10d579c64e P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.987290  8925 tablet_replica.cc:333] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.988057  8925 raft_consensus.cc:2243] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.988648  8925 raft_consensus.cc:2272] T f51082bf25994b80bf0923d3951de95c P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.990952  8925 tablet_replica.cc:333] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.991796  8925 raft_consensus.cc:2243] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:45.992877  8925 raft_consensus.cc:2272] T 9f7654d07a19420cb4acceea93c8ad79 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.995254  8925 tablet_replica.cc:333] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.995936  8925 raft_consensus.cc:2243] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.996488  8925 raft_consensus.cc:2272] T 74cc231530744a91bbac7f6dd2b389c9 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:45.998579  8925 tablet_replica.cc:333] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:45.999364  8925 raft_consensus.cc:2243] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:45.999917  8925 raft_consensus.cc:2272] T 0257272195d64d82a8f20f125cfe59ad P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.002061  8925 tablet_replica.cc:333] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.002772  8925 raft_consensus.cc:2243] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.003409  8925 raft_consensus.cc:2272] T 82394bbf04c1483a9975840ec2e3d165 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.005764  8925 tablet_replica.cc:333] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.006467  8925 raft_consensus.cc:2243] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.007246  8925 raft_consensus.cc:2272] T 12bfdc79f28641df8a7d71ab55d0cc59 P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.009680  8925 tablet_replica.cc:333] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.010396  8925 raft_consensus.cc:2243] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.010963  8925 raft_consensus.cc:2272] T fc384aaab9dd45de9c6d07332cd2ccda P 9e4db2af5b894498a39e7996de5f4420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.013315  8925 tablet_replica.cc:333] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.014403  8925 raft_consensus.cc:2243] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.015072  8925 raft_consensus.cc:2272] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 9e4db2af5b894498a39e7996de5f4420 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.018584  8925 tablet_replica.cc:333] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.019533  8925 raft_consensus.cc:2243] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.020956  8925 raft_consensus.cc:2272] T 89a0777da2644a02bda497c1c128979e P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.024403  8925 tablet_replica.cc:333] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.025403  8925 raft_consensus.cc:2243] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.026144  8925 raft_consensus.cc:2272] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.029139  8925 tablet_replica.cc:333] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.029882  8925 raft_consensus.cc:2243] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.031198  8925 raft_consensus.cc:2272] T 42e12fa7411c40339b9b70a8ad124b46 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.034337  8925 tablet_replica.cc:333] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.035085  8925 raft_consensus.cc:2243] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.035749  8925 raft_consensus.cc:2272] T 4aba9f5ed69242c586d84b02fa5a247b P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.038144  8925 tablet_replica.cc:333] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.038998  8925 raft_consensus.cc:2243] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.039628  8925 raft_consensus.cc:2272] T 53cb077d6586458784a8fa52fa64333a P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.042120  8925 tablet_replica.cc:333] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.042707  8925 raft_consensus.cc:2243] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.043741  8925 raft_consensus.cc:2272] T f26560550b3b4f1d9cd5dfbcab383c09 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.046082  8925 tablet_replica.cc:333] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420: stopping tablet replica
I20260709 13:22:46.046640  8925 raft_consensus.cc:2243] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.047179  8925 raft_consensus.cc:2272] T 5ecc044526df46e9bcb8a007ec2d0409 P 9e4db2af5b894498a39e7996de5f4420 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:46.092247 21355 consensus_peers.cc:597] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.094146  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
W20260709 13:22:46.111441 21355 consensus_peers.cc:597] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.136289  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
W20260709 13:22:46.168355 21281 consensus_peers.cc:597] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.178905 21281 consensus_peers.cc:597] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.182345  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:46.183396  8925 tablet_replica.cc:333] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.184901  8925 raft_consensus.cc:2243] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.185545  8925 raft_consensus.cc:2272] T 74cc231530744a91bbac7f6dd2b389c9 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.188139  8925 tablet_replica.cc:333] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.188877  8925 raft_consensus.cc:2243] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.189493  8925 raft_consensus.cc:2272] T 597bcc40ac0247ed855affc1fe0e3a19 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.191867  8925 tablet_replica.cc:333] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.192626  8925 raft_consensus.cc:2243] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.193818  8925 raft_consensus.cc:2272] T c791a1bc687c456d88cabea4e0bfae27 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.196703  8925 tablet_replica.cc:333] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.197652  8925 raft_consensus.cc:2243] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.198359  8925 raft_consensus.cc:2272] T 9f7654d07a19420cb4acceea93c8ad79 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:46.198400 21355 consensus_peers.cc:597] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.201126  8925 tablet_replica.cc:333] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.201947  8925 raft_consensus.cc:2243] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:46.202217 21354 consensus_peers.cc:597] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d -> Peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Couldn't send request to peer c1674e5c97eb461c98fb5d863246c0f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.203050  8925 raft_consensus.cc:2272] T f51082bf25994b80bf0923d3951de95c P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.205854  8925 tablet_replica.cc:333] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.206714  8925 raft_consensus.cc:2243] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.207919  8925 raft_consensus.cc:2272] T 0257272195d64d82a8f20f125cfe59ad P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.210469  8925 tablet_replica.cc:333] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.211305  8925 raft_consensus.cc:2243] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.212347  8925 raft_consensus.cc:2272] T 82394bbf04c1483a9975840ec2e3d165 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.214835  8925 tablet_replica.cc:333] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.215587  8925 raft_consensus.cc:2243] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.216156  8925 raft_consensus.cc:2272] T 12bfdc79f28641df8a7d71ab55d0cc59 P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.218606  8925 tablet_replica.cc:333] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.219374  8925 raft_consensus.cc:2243] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.219915  8925 raft_consensus.cc:2272] T dfe505957770401e92725d10d579c64e P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.222291  8925 tablet_replica.cc:333] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.222998  8925 raft_consensus.cc:2243] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.223634  8925 raft_consensus.cc:2272] T fc384aaab9dd45de9c6d07332cd2ccda P c1674e5c97eb461c98fb5d863246c0f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.225981  8925 tablet_replica.cc:333] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.226727  8925 raft_consensus.cc:2243] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.227332  8925 raft_consensus.cc:2272] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P c1674e5c97eb461c98fb5d863246c0f5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.230055  8925 tablet_replica.cc:333] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.230756  8925 raft_consensus.cc:2243] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.231443  8925 raft_consensus.cc:2272] T 89a0777da2644a02bda497c1c128979e P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.234233  8925 tablet_replica.cc:333] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.235029  8925 raft_consensus.cc:2243] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.235602  8925 raft_consensus.cc:2272] T 42e12fa7411c40339b9b70a8ad124b46 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.237988  8925 tablet_replica.cc:333] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.238792  8925 raft_consensus.cc:2243] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.239924  8925 raft_consensus.cc:2272] T 87ce7fffd50a45bebc3ba745a2ec9fdc P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.242434  8925 tablet_replica.cc:333] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.243093  8925 raft_consensus.cc:2243] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.243726  8925 raft_consensus.cc:2272] T 4aba9f5ed69242c586d84b02fa5a247b P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.245940  8925 tablet_replica.cc:333] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.246632  8925 raft_consensus.cc:2243] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.247689  8925 raft_consensus.cc:2272] T 53cb077d6586458784a8fa52fa64333a P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.250051  8925 tablet_replica.cc:333] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.250787  8925 raft_consensus.cc:2243] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.251394  8925 raft_consensus.cc:2272] T f26560550b3b4f1d9cd5dfbcab383c09 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.253746  8925 tablet_replica.cc:333] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5: stopping tablet replica
I20260709 13:22:46.254360  8925 raft_consensus.cc:2243] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.254881  8925 raft_consensus.cc:2272] T 5ecc044526df46e9bcb8a007ec2d0409 P c1674e5c97eb461c98fb5d863246c0f5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:46.270328 21355 consensus_peers.cc:597] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.294540 21354 consensus_peers.cc:597] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d -> Peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Couldn't send request to peer c1674e5c97eb461c98fb5d863246c0f5. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:46747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.305155  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
W20260709 13:22:46.315686 21355 consensus_peers.cc:597] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.346148  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
W20260709 13:22:46.380951 21354 consensus_peers.cc:597] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d -> Peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Couldn't send request to peer c1674e5c97eb461c98fb5d863246c0f5. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:46747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.386698  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:46.387729  8925 tablet_replica.cc:333] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.388495  8925 raft_consensus.cc:2243] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.389539  8925 raft_consensus.cc:2272] T 74cc231530744a91bbac7f6dd2b389c9 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.392241  8925 tablet_replica.cc:333] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.392994  8925 raft_consensus.cc:2243] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.393600  8925 raft_consensus.cc:2272] T f51082bf25994b80bf0923d3951de95c P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.396056  8925 tablet_replica.cc:333] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.396796  8925 raft_consensus.cc:2243] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.397388  8925 raft_consensus.cc:2272] T 9f7654d07a19420cb4acceea93c8ad79 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.399925  8925 tablet_replica.cc:333] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.400686  8925 raft_consensus.cc:2243] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.401252  8925 raft_consensus.cc:2272] T 87ce7fffd50a45bebc3ba745a2ec9fdc P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.403908  8925 tablet_replica.cc:333] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.404656  8925 raft_consensus.cc:2243] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.405258  8925 raft_consensus.cc:2272] T 42e12fa7411c40339b9b70a8ad124b46 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.407946  8925 tablet_replica.cc:333] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.408659  8925 raft_consensus.cc:2243] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.409251  8925 raft_consensus.cc:2272] T 597bcc40ac0247ed855affc1fe0e3a19 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.411564  8925 tablet_replica.cc:333] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.412209  8925 raft_consensus.cc:2243] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.412767  8925 raft_consensus.cc:2272] T c791a1bc687c456d88cabea4e0bfae27 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.414971  8925 tablet_replica.cc:333] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.415696  8925 raft_consensus.cc:2243] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.416249  8925 raft_consensus.cc:2272] T dfe505957770401e92725d10d579c64e P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.418401  8925 tablet_replica.cc:333] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.419092  8925 raft_consensus.cc:2243] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.420087  8925 raft_consensus.cc:2272] T fc384aaab9dd45de9c6d07332cd2ccda P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.422377  8925 tablet_replica.cc:333] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.423103  8925 raft_consensus.cc:2243] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.424146  8925 raft_consensus.cc:2272] T ffeb6d2b8c944c2ba0bd0e7a06fe6a3a P 85475c7436484601a066fc66945e140d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.427594  8925 tablet_replica.cc:333] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.428719  8925 raft_consensus.cc:2243] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.429621  8925 raft_consensus.cc:2272] T 89a0777da2644a02bda497c1c128979e P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:46.431464 21354 consensus_peers.cc:597] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d -> Peer c1674e5c97eb461c98fb5d863246c0f5 (127.8.183.66:46747): Couldn't send request to peer c1674e5c97eb461c98fb5d863246c0f5. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:46747: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.434402  8925 tablet_replica.cc:333] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.435362  8925 raft_consensus.cc:2243] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.436028  8925 raft_consensus.cc:2272] T f26560550b3b4f1d9cd5dfbcab383c09 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:46.436041 21355 consensus_peers.cc:597] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d -> Peer 9e4db2af5b894498a39e7996de5f4420 (127.8.183.65:43775): Couldn't send request to peer 9e4db2af5b894498a39e7996de5f4420. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:43775: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.438740  8925 tablet_replica.cc:333] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.439584  8925 raft_consensus.cc:2243] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.440676  8925 raft_consensus.cc:2272] T 4aba9f5ed69242c586d84b02fa5a247b P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.442901  8925 tablet_replica.cc:333] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.443610  8925 raft_consensus.cc:2243] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.444171  8925 raft_consensus.cc:2272] T 53cb077d6586458784a8fa52fa64333a P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.446489  8925 tablet_replica.cc:333] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.447204  8925 raft_consensus.cc:2243] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.448181  8925 raft_consensus.cc:2272] T 5ecc044526df46e9bcb8a007ec2d0409 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.450781  8925 tablet_replica.cc:333] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.451515  8925 raft_consensus.cc:2243] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.452095  8925 raft_consensus.cc:2272] T 0257272195d64d82a8f20f125cfe59ad P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.454247  8925 tablet_replica.cc:333] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.454886  8925 raft_consensus.cc:2243] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:46.455457  8925 raft_consensus.cc:2272] T 82394bbf04c1483a9975840ec2e3d165 P 85475c7436484601a066fc66945e140d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.457738  8925 tablet_replica.cc:333] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d: stopping tablet replica
I20260709 13:22:46.458297  8925 raft_consensus.cc:2243] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.459362  8925 raft_consensus.cc:2272] T 12bfdc79f28641df8a7d71ab55d0cc59 P 85475c7436484601a066fc66945e140d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.507761  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:22:46.616251  8925 master.cc:562] Master@127.8.183.126:36851 shutting down...
I20260709 13:22:46.664148  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:46.665102  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:46.665485  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P e22de12c734741e8b1f9d7e0cc3cf03e: stopping tablet replica
I20260709 13:22:46.692165  8925 master.cc:584] Master@127.8.183.126:36851 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13571 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:22:46.772133  8925 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.183.126:38967
I20260709 13:22:46.773530  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:46.781188 21709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:46.781718 21708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:46.781893 21711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:46.782357  8925 server_base.cc:1061] running on GCE node
I20260709 13:22:46.783775  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:46.784036  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:46.784165  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603366784153 us; error 0 us; skew 500 ppm
I20260709 13:22:46.784704  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:46.788107  8925 webserver.cc:533] Webserver started at http://127.8.183.126:44859/ using document root <none> and password file <none>
I20260709 13:22:46.788690  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:46.788909  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:46.789162  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:46.790657  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/master-0-root/instance:
uuid: "7472c397f9854384af99c427cb180428"
format_stamp: "Formatted at 2026-07-09 13:22:46 on dist-test-slave-ch3x"
I20260709 13:22:46.796670  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:22:46.801602 21716 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:46.802743  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:46.803239  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/master-0-root
uuid: "7472c397f9854384af99c427cb180428"
format_stamp: "Formatted at 2026-07-09 13:22:46 on dist-test-slave-ch3x"
I20260709 13:22:46.803629  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:46.827600  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:46.829116  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:46.875224  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.126:38967
I20260709 13:22:46.875300 21767 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.126:38967 every 8 connection(s)
I20260709 13:22:46.880105 21768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:46.896632 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: Bootstrap starting.
I20260709 13:22:46.902259 21768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:46.907637 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: No bootstrap required, opened a new log
I20260709 13:22:46.910388 21768 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER }
I20260709 13:22:46.910930 21768 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:46.911250 21768 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7472c397f9854384af99c427cb180428, State: Initialized, Role: FOLLOWER
I20260709 13:22:46.912011 21768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER }
I20260709 13:22:46.912535 21768 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:46.912755 21768 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:46.912984 21768 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:46.919305 21768 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER }
I20260709 13:22:46.919960 21768 leader_election.cc:304] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7472c397f9854384af99c427cb180428; no voters: 
I20260709 13:22:46.921402 21768 leader_election.cc:290] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:46.922051 21771 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:46.924149 21771 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 1 LEADER]: Becoming Leader. State: Replica: 7472c397f9854384af99c427cb180428, State: Running, Role: LEADER
I20260709 13:22:46.925098 21771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER }
I20260709 13:22:46.925415 21768 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:46.933877 21773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7472c397f9854384af99c427cb180428. Latest consensus state: current_term: 1 leader_uuid: "7472c397f9854384af99c427cb180428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER } }
I20260709 13:22:46.934379 21772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7472c397f9854384af99c427cb180428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7472c397f9854384af99c427cb180428" member_type: VOTER } }
I20260709 13:22:46.935321 21773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:46.935912 21772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:46.940102 21780 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:46.949375 21780 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:46.950724  8925 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:46.961675 21780 catalog_manager.cc:1383] Generated new cluster ID: f33cca2fae79488784e71a157f85c8f3
I20260709 13:22:46.962162 21780 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:46.987319 21780 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:46.989321 21780 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:47.012131 21780 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: Generated new TSK 0
I20260709 13:22:47.013150 21780 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:47.018643  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.026551 21789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.028359 21790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.031653  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:47.031949 21792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.033216  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.033545  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.033758  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603367033734 us; error 0 us; skew 500 ppm
I20260709 13:22:47.034441  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.038002  8925 webserver.cc:533] Webserver started at http://127.8.183.65:40115/ using document root <none> and password file <none>
I20260709 13:22:47.038718  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.038915  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.039325  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.040722  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-0-root/instance:
uuid: "c3b3294fda6a4b578c7e149e6990b94e"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.046630  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:22:47.051187 21797 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.052249  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:47.052678  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-0-root
uuid: "c3b3294fda6a4b578c7e149e6990b94e"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.053064  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.077617  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.079070  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.081029  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.084182  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.084481  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.084758  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.084955  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.141927  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.65:34695
I20260709 13:22:47.142084 21859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.65:34695 every 8 connection(s)
I20260709 13:22:47.148823  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.158670 21864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.160745 21865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.163180  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:47.163570 21867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.165484  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.165872  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.166136  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603367166114 us; error 0 us; skew 500 ppm
I20260709 13:22:47.166906  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.166996 21860 heartbeater.cc:344] Connected to a master server at 127.8.183.126:38967
I20260709 13:22:47.167629 21860 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.168780 21860 heartbeater.cc:507] Master 127.8.183.126:38967 requested a full tablet report, sending...
I20260709 13:22:47.170822  8925 webserver.cc:533] Webserver started at http://127.8.183.66:33609/ using document root <none> and password file <none>
I20260709 13:22:47.171787  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.172084  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.172290 21733 ts_manager.cc:194] Registered new tserver with Master: c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695)
I20260709 13:22:47.172513  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.175002  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-1-root/instance:
uuid: "019e784c7f884e0782e9dca6055b7a4b"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.175745 21733 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57458
I20260709 13:22:47.181481  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:47.186676 21872 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.187947  8925 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:22:47.188282  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-1-root
uuid: "019e784c7f884e0782e9dca6055b7a4b"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.188582  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.201153  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.202560  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.204418  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.207751  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.207996  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.208355  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.208531  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.262459  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.66:38507
I20260709 13:22:47.262595 21934 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.66:38507 every 8 connection(s)
I20260709 13:22:47.268776  8925 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.277496 21938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.281738 21939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.284476  8925 server_base.cc:1061] running on GCE node
W20260709 13:22:47.284878 21941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.286446  8925 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.286847  8925 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.287159  8925 hybrid_clock.cc:648] HybridClock initialized: now 1783603367287086 us; error 0 us; skew 500 ppm
I20260709 13:22:47.288060  8925 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.288749 21935 heartbeater.cc:344] Connected to a master server at 127.8.183.126:38967
I20260709 13:22:47.289398 21935 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.290592 21935 heartbeater.cc:507] Master 127.8.183.126:38967 requested a full tablet report, sending...
I20260709 13:22:47.291677  8925 webserver.cc:533] Webserver started at http://127.8.183.67:41669/ using document root <none> and password file <none>
I20260709 13:22:47.292523  8925 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.292809  8925 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.293171  8925 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.293519 21733 ts_manager.cc:194] Registered new tserver with Master: 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:47.295288  8925 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-2-root/instance:
uuid: "434c406d785b40b982cd98e21b7b97c2"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.295956 21733 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57460
I20260709 13:22:47.302162  8925 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:22:47.307518 21946 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.308712  8925 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:22:47.309166  8925 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-2-root
uuid: "434c406d785b40b982cd98e21b7b97c2"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-ch3x"
I20260709 13:22:47.309491  8925 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task5lzfIH/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602929618813-8925-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.344259  8925 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.346093  8925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.348187  8925 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.351025  8925 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.351267  8925 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:22:47.351507  8925 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.351697  8925 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.407694  8925 rpc_server.cc:307] RPC server started. Bound to: 127.8.183.67:40255
I20260709 13:22:47.407840 22008 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.183.67:40255 every 8 connection(s)
I20260709 13:22:47.433687 22009 heartbeater.cc:344] Connected to a master server at 127.8.183.126:38967
I20260709 13:22:47.434386 22009 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.435391 22009 heartbeater.cc:507] Master 127.8.183.126:38967 requested a full tablet report, sending...
I20260709 13:22:47.438115 21733 ts_manager.cc:194] Registered new tserver with Master: 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:47.439280  8925 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.02702716s
I20260709 13:22:47.440457 21733 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57462
I20260709 13:22:47.476830 21733 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57470:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:47.480342 21733 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:47.587980 21899 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb45406e4684da6b8696c4c0f5da2f3 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:47.589098 21969 tablet_service.cc:1511] Processing CreateTablet for tablet a5e0712f6a964772870a3eca408dac96 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:47.589340 21898 tablet_service.cc:1511] Processing CreateTablet for tablet 5466e4df67bb43c599d51a521771792f (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:47.590189 21899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb45406e4684da6b8696c4c0f5da2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.591104 21969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e0712f6a964772870a3eca408dac96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.591425 21898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5466e4df67bb43c599d51a521771792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.592636 21900 tablet_service.cc:1511] Processing CreateTablet for tablet 0693a0ddbfa747b7b5c8ef13feeee767 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:47.589098 21974 tablet_service.cc:1511] Processing CreateTablet for tablet 0693a0ddbfa747b7b5c8ef13feeee767 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:47.594619 21900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0693a0ddbfa747b7b5c8ef13feeee767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.595062 21971 tablet_service.cc:1511] Processing CreateTablet for tablet a4fce7842e6c4feaa1a14e883ce636c5 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:47.596624 21897 tablet_service.cc:1511] Processing CreateTablet for tablet a4fce7842e6c4feaa1a14e883ce636c5 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:47.598317 21970 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5539e7e2024cf38082a16791bfc8c8 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:47.592799 21972 tablet_service.cc:1511] Processing CreateTablet for tablet 5466e4df67bb43c599d51a521771792f (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:47.599874 21896 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5539e7e2024cf38082a16791bfc8c8 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:47.590596 21973 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb45406e4684da6b8696c4c0f5da2f3 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:47.600884 21895 tablet_service.cc:1511] Processing CreateTablet for tablet a5e0712f6a964772870a3eca408dac96 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:47.598323 21897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4fce7842e6c4feaa1a14e883ce636c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.595621 21974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0693a0ddbfa747b7b5c8ef13feeee767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.604107 21971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4fce7842e6c4feaa1a14e883ce636c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.609752 21972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5466e4df67bb43c599d51a521771792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.611231 21968 tablet_service.cc:1511] Processing CreateTablet for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:47.611722 21894 tablet_service.cc:1511] Processing CreateTablet for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:47.613030 21968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.613761 21970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5539e7e2024cf38082a16791bfc8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.619378 21973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb45406e4684da6b8696c4c0f5da2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.602602 21895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e0712f6a964772870a3eca408dac96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.626621 21896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5539e7e2024cf38082a16791bfc8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.629925 21894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.643920 21893 tablet_service.cc:1511] Processing CreateTablet for tablet 4807ded299c54b8a8ec9f561db684065 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:47.646274 21893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4807ded299c54b8a8ec9f561db684065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.648524 21967 tablet_service.cc:1511] Processing CreateTablet for tablet 4807ded299c54b8a8ec9f561db684065 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:47.650413 21967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4807ded299c54b8a8ec9f561db684065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.658471 21823 tablet_service.cc:1511] Processing CreateTablet for tablet 5466e4df67bb43c599d51a521771792f (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:47.659907 21966 tablet_service.cc:1511] Processing CreateTablet for tablet 14a9ebf91f79450f904dbccb7d51ea68 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:47.660444 21822 tablet_service.cc:1511] Processing CreateTablet for tablet a4fce7842e6c4feaa1a14e883ce636c5 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:47.660455 21818 tablet_service.cc:1511] Processing CreateTablet for tablet 4807ded299c54b8a8ec9f561db684065 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:47.661623 21966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a9ebf91f79450f904dbccb7d51ea68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.661928 21821 tablet_service.cc:1511] Processing CreateTablet for tablet 6d5539e7e2024cf38082a16791bfc8c8 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:47.663559 21817 tablet_service.cc:1511] Processing CreateTablet for tablet 14a9ebf91f79450f904dbccb7d51ea68 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:47.663409 21824 tablet_service.cc:1511] Processing CreateTablet for tablet 1fb45406e4684da6b8696c4c0f5da2f3 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:47.664472 21820 tablet_service.cc:1511] Processing CreateTablet for tablet a5e0712f6a964772870a3eca408dac96 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:47.665585 21892 tablet_service.cc:1511] Processing CreateTablet for tablet 14a9ebf91f79450f904dbccb7d51ea68 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:47.668107 21819 tablet_service.cc:1511] Processing CreateTablet for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:47.669509 21892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a9ebf91f79450f904dbccb7d51ea68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.668107 21825 tablet_service.cc:1511] Processing CreateTablet for tablet 0693a0ddbfa747b7b5c8ef13feeee767 (DEFAULT_TABLE table=test-workload [id=29da5dd6c6224899977eb5b383901287]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:47.660493 21823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5466e4df67bb43c599d51a521771792f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.677172 21819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d3d31d96d8f4a38a997ed65bb87e6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.678010 21817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14a9ebf91f79450f904dbccb7d51ea68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.678844 21818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4807ded299c54b8a8ec9f561db684065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.681469 21822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4fce7842e6c4feaa1a14e883ce636c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.683660 21824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fb45406e4684da6b8696c4c0f5da2f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.684616 21820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5e0712f6a964772870a3eca408dac96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.691295 21825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0693a0ddbfa747b7b5c8ef13feeee767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.692238 21821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d5539e7e2024cf38082a16791bfc8c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.756776 22025 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:47.768492 22024 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:47.783526 22024 tablet_bootstrap.cc:654] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.828696 22025 tablet_bootstrap.cc:654] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.855705 22024 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:47.863310 22024 ts_tablet_manager.cc:1403] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.003s
I20260709 13:22:47.866118 22026 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:47.869144 22024 raft_consensus.cc:359] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.869940 22024 raft_consensus.cc:385] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.870247 22024 raft_consensus.cc:740] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.870469 22025 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:47.871377 22025 ts_tablet_manager.cc:1403] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.009s
I20260709 13:22:47.871165 22024 consensus_queue.cc:260] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.874619 22009 heartbeater.cc:499] Master 127.8.183.126:38967 was elected leader, sending a full tablet report...
I20260709 13:22:47.874718 22024 ts_tablet_manager.cc:1434] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:22:47.876121 22026 tablet_bootstrap.cc:654] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.876607 22024 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:47.877648 22025 raft_consensus.cc:359] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.878608 22025 raft_consensus.cc:385] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.878959 22025 raft_consensus.cc:740] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.879927 22025 consensus_queue.cc:260] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.886725 22024 tablet_bootstrap.cc:654] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.889671 22026 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:47.890619 22026 ts_tablet_manager.cc:1403] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.010s
I20260709 13:22:47.894336 22026 raft_consensus.cc:359] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.895897 22026 raft_consensus.cc:385] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.896411 22026 raft_consensus.cc:740] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.897660 22026 consensus_queue.cc:260] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.903932 22024 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:47.905144 22024 ts_tablet_manager.cc:1403] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.029s	user 0.023s	sys 0.004s
I20260709 13:22:47.906594 22026 ts_tablet_manager.cc:1434] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.002s
I20260709 13:22:47.907711 21860 heartbeater.cc:499] Master 127.8.183.126:38967 was elected leader, sending a full tablet report...
I20260709 13:22:47.909636 21935 heartbeater.cc:499] Master 127.8.183.126:38967 was elected leader, sending a full tablet report...
I20260709 13:22:47.910331 22025 ts_tablet_manager.cc:1434] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.033s
I20260709 13:22:47.911927 22025 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
W20260709 13:22:47.913830 22010 tablet.cc:2406] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:47.916288 22024 raft_consensus.cc:359] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.917229 22024 raft_consensus.cc:385] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.917654 22024 raft_consensus.cc:740] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.918545 22026 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:47.921466 22025 tablet_bootstrap.cc:654] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.918780 22024 consensus_queue.cc:260] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.928495 22026 tablet_bootstrap.cc:654] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.929836 22024 ts_tablet_manager.cc:1434] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 13:22:47.932183 22024 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:47.938750 22025 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
W20260709 13:22:47.941943 21781 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:47.939615 22025 ts_tablet_manager.cc:1403] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.014s
I20260709 13:22:47.942951 22024 tablet_bootstrap.cc:654] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.946338 22025 raft_consensus.cc:359] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.947484 22025 raft_consensus.cc:385] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.947825 22025 raft_consensus.cc:740] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.949031 22025 consensus_queue.cc:260] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.951815 22025 ts_tablet_manager.cc:1434] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:22:47.953125 22025 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:47.961884 22025 tablet_bootstrap.cc:654] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.969834 22024 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:47.970980 22024 ts_tablet_manager.cc:1403] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.019s
I20260709 13:22:47.971040 22025 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:47.971983 22025 ts_tablet_manager.cc:1403] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:22:47.976150 22024 raft_consensus.cc:359] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.977206 22024 raft_consensus.cc:385] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.977546 22024 raft_consensus.cc:740] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.978818 22024 consensus_queue.cc:260] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.980589 22025 raft_consensus.cc:359] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.981419 22025 raft_consensus.cc:385] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.981513 22024 ts_tablet_manager.cc:1434] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:22:47.981705 22025 raft_consensus.cc:740] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.983028 22025 consensus_queue.cc:260] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.984319 22026 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:47.985065 22026 ts_tablet_manager.cc:1403] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.032s
I20260709 13:22:47.985613 22025 ts_tablet_manager.cc:1434] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 13:22:47.986968 22025 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:47.987010 22024 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:47.988535 22026 raft_consensus.cc:359] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.989554 22026 raft_consensus.cc:385] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.989883 22026 raft_consensus.cc:740] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.990942 22026 consensus_queue.cc:260] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:47.996101 22025 tablet_bootstrap.cc:654] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.996791 22026 ts_tablet_manager.cc:1434] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.006s
I20260709 13:22:47.998312 22026 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.003638 22031 raft_consensus.cc:493] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.004520 22031 raft_consensus.cc:515] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.006137 22026 tablet_bootstrap.cc:654] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.011533 22031 leader_election.cc:290] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.019255 22025 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.019860 22024 tablet_bootstrap.cc:654] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.020290 22025 ts_tablet_manager.cc:1403] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.003s
I20260709 13:22:48.025087 22025 raft_consensus.cc:359] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.026038 22025 raft_consensus.cc:385] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.026357 22025 raft_consensus.cc:740] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.027704 22025 consensus_queue.cc:260] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.030961 22025 ts_tablet_manager.cc:1434] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:22:48.032445 22025 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:48.041165 22024 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.041209 22025 tablet_bootstrap.cc:654] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.042397 22024 ts_tablet_manager.cc:1403] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.006s
I20260709 13:22:48.055258 22025 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.056277 22025 ts_tablet_manager.cc:1403] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 13:22:48.056595 22024 raft_consensus.cc:359] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.057842 22024 raft_consensus.cc:385] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.058259 22024 raft_consensus.cc:740] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.059900 22024 consensus_queue.cc:260] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.060758 22025 raft_consensus.cc:359] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.063100 22024 ts_tablet_manager.cc:1434] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 13:22:48.064616 22024 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:48.061951 22025 raft_consensus.cc:385] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.065562 22025 raft_consensus.cc:740] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.065693 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.066701 21835 raft_consensus.cc:2468] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.066732 22025 consensus_queue.cc:260] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.070161 22025 ts_tablet_manager.cc:1434] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:22:48.071763 22025 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:48.074352 21873 leader_election.cc:304] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.075811 22031 raft_consensus.cc:2804] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.076336 22031 raft_consensus.cc:493] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.076715 22031 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.079607 22024 tablet_bootstrap.cc:654] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.079986 22025 tablet_bootstrap.cc:654] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.092139 22045 raft_consensus.cc:493] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.092957 22045 raft_consensus.cc:515] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.098951 22045 leader_election.cc:290] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.101148 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.103287 22024 tablet_bootstrap.cc:492] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.104346 22024 ts_tablet_manager.cc:1403] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.009s
I20260709 13:22:48.105674 22029 raft_consensus.cc:493] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:48.104560 21873 leader_election.cc:343] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.106632 22029 raft_consensus.cc:515] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.112660 22024 raft_consensus.cc:359] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.113583 22029 leader_election.cc:290] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.115209 22025 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.114933 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.116137 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.124604 22025 ts_tablet_manager.cc:1403] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.021s
I20260709 13:22:48.125056 21983 raft_consensus.cc:2468] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.118590 22031 raft_consensus.cc:515] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.126971 21874 leader_election.cc:304] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.128135 22045 raft_consensus.cc:2804] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.128758 22045 raft_consensus.cc:493] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.128710 22025 raft_consensus.cc:359] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.129498 22031 leader_election.cc:290] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.129633 22045 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.129863 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.123704 22024 raft_consensus.cc:385] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.129971 22025 raft_consensus.cc:385] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.131309 21835 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.131474 22025 raft_consensus.cc:740] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.131628 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.132707 22025 consensus_queue.cc:260] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.135449 22025 ts_tablet_manager.cc:1434] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:22:48.137082 22025 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:48.131328 22024 raft_consensus.cc:740] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.144527 22025 tablet_bootstrap.cc:654] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.145289 22029 raft_consensus.cc:493] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.145933 22029 raft_consensus.cc:515] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.148911 22029 leader_election.cc:290] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.149891 22024 consensus_queue.cc:260] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.151244 21984 raft_consensus.cc:2468] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.121115 22026 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.153905 22026 ts_tablet_manager.cc:1403] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.156s	user 0.032s	sys 0.061s
I20260709 13:22:48.157188 22024 ts_tablet_manager.cc:1434] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 13:22:48.156776 22045 raft_consensus.cc:515] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.157793 22033 raft_consensus.cc:493] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.158632 22024 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:48.158396 22033 raft_consensus.cc:515] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.160894 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
W20260709 13:22:48.163020 21873 leader_election.cc:343] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.164119 22033 leader_election.cc:290] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.166142 22026 raft_consensus.cc:359] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.167470 22026 raft_consensus.cc:385] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.167876 22026 raft_consensus.cc:740] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.169011 22026 consensus_queue.cc:260] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.144469 21835 raft_consensus.cc:2468] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:48.172752 21873 leader_election.cc:304] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.174022 22031 raft_consensus.cc:2804] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.175321 22025 tablet_bootstrap.cc:492] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.176220 22025 ts_tablet_manager.cc:1403] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.009s
I20260709 13:22:48.179858 22024 tablet_bootstrap.cc:654] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.180655 22045 leader_election.cc:290] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.180781 22025 raft_consensus.cc:359] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.181860 22025 raft_consensus.cc:385] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.182216 22025 raft_consensus.cc:740] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.182617 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.183750 21984 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.183175 22025 consensus_queue.cc:260] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.184738 22026 ts_tablet_manager.cc:1434] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 13:22:48.186244 22026 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.186481 22025 ts_tablet_manager.cc:1434] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:22:48.188733 22025 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:48.188928 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.190596 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.191609 21834 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
W20260709 13:22:48.193312 21947 leader_election.cc:343] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.195295 21947 leader_election.cc:304] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.196476 22029 raft_consensus.cc:2804] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.196928 22029 raft_consensus.cc:493] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.197315 22029 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.204349 22024 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.205171 22024 ts_tablet_manager.cc:1403] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.003s
I20260709 13:22:48.197175 22025 tablet_bootstrap.cc:654] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.209237 22024 raft_consensus.cc:359] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.210232 22024 raft_consensus.cc:385] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.210579 22024 raft_consensus.cc:740] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.213553 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.214344 22024 consensus_queue.cc:260] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.214883 22031 raft_consensus.cc:697] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:48.215317 21910 raft_consensus.cc:2393] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 434c406d785b40b982cd98e21b7b97c2 in current term 1: Already voted for candidate 019e784c7f884e0782e9dca6055b7a4b in this term.
I20260709 13:22:48.216343 22031 consensus_queue.cc:237] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.217342 21948 leader_election.cc:304] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2; no voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.211476 22026 tablet_bootstrap.cc:654] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.213959 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.213581 22053 raft_consensus.cc:493] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.213243 22029 raft_consensus.cc:515] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.229419 21909 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:48.221602 22045 raft_consensus.cc:493] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.230500 22045 raft_consensus.cc:515] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.229444 22053 raft_consensus.cc:515] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.233902 22045 leader_election.cc:290] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.234251 22053 leader_election.cc:290] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.231401 22029 leader_election.cc:290] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.234851 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.235813 21909 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.238039 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.239321 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
W20260709 13:22:48.240113 21874 leader_election.cc:343] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.238086 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.241700 21983 raft_consensus.cc:2468] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.243420 22026 tablet_bootstrap.cc:492] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.243767 21874 leader_election.cc:304] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:48.244143 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.245046 22053 raft_consensus.cc:2804] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.245486 22053 raft_consensus.cc:493] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.245940 22053 raft_consensus.cc:3060] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.247516 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
W20260709 13:22:48.249591 21873 leader_election.cc:343] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.251011 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.251775 21834 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.244118 22026 ts_tablet_manager.cc:1403] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.058s	user 0.015s	sys 0.000s
W20260709 13:22:48.255995 21873 leader_election.cc:343] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.256958 21873 leader_election.cc:304] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b; no voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.257776 22029 raft_consensus.cc:493] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.258422 22029 raft_consensus.cc:515] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.258914 22045 raft_consensus.cc:2749] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.261049 22029 leader_election.cc:290] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.265375 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.266350 21910 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:48.268070 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.268301 21948 leader_election.cc:304] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
W20260709 13:22:48.270082 21947 leader_election.cc:343] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.257165 22053 raft_consensus.cc:515] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.274874 22029 raft_consensus.cc:2804] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.274354 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.276579 22029 raft_consensus.cc:493] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.277163 22029 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.277364 21983 raft_consensus.cc:2393] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 434c406d785b40b982cd98e21b7b97c2 in this term.
I20260709 13:22:48.272583 21909 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.273051 22026 raft_consensus.cc:359] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.280969 22026 raft_consensus.cc:385] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.281718 22026 raft_consensus.cc:740] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.281852 21948 leader_election.cc:304] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:48.282784 22026 consensus_queue.cc:260] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.283824 21984 raft_consensus.cc:2468] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:48.284587 22058 raft_consensus.cc:2749] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.284595 22024 ts_tablet_manager.cc:1434] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.079s	user 0.024s	sys 0.036s
I20260709 13:22:48.286839 22024 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:48.288339 22033 raft_consensus.cc:493] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.288933 22033 raft_consensus.cc:515] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.292141 22033 leader_election.cc:290] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.293313 21834 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.295439 22056 raft_consensus.cc:2804] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.296208 22056 raft_consensus.cc:697] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:48.297199 22025 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.297915 22025 ts_tablet_manager.cc:1403] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.109s	user 0.011s	sys 0.022s
I20260709 13:22:48.297338 22056 consensus_queue.cc:237] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.281114 21799 leader_election.cc:304] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.301468 22025 raft_consensus.cc:359] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.302734 22025 raft_consensus.cc:385] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.303148 22025 raft_consensus.cc:740] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.303347 21874 leader_election.cc:304] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.304185 22025 consensus_queue.cc:260] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.302150 22033 raft_consensus.cc:2749] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.305459 22045 raft_consensus.cc:2804] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.304613 21730 catalog_manager.cc:5697] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.307415 22026 ts_tablet_manager.cc:1434] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.051s	user 0.010s	sys 0.000s
I20260709 13:22:48.308851 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.310168 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.311017 22025 ts_tablet_manager.cc:1434] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.001s
I20260709 13:22:48.301617 22024 tablet_bootstrap.cc:654] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.312525 22025 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
W20260709 13:22:48.311007 21799 leader_election.cc:343] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.311005 21909 raft_consensus.cc:2468] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 0.
I20260709 13:22:48.315434 22033 raft_consensus.cc:493] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.316103 22033 raft_consensus.cc:515] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.319924 22026 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.323258 21799 leader_election.cc:304] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.326978 22025 tablet_bootstrap.cc:654] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.318712 22029 raft_consensus.cc:515] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.334738 22033 leader_election.cc:290] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.336077 22033 raft_consensus.cc:2804] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.336766 22033 raft_consensus.cc:493] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.337354 22033 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.341369 22029 leader_election.cc:290] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.343240 22053 leader_election.cc:290] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.345948 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.346110 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.346935 21984 raft_consensus.cc:3060] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.343009 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.347067 21909 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.347855 22045 raft_consensus.cc:697] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:48.350070 22045 consensus_queue.cc:237] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.348815 21834 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.348291 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
W20260709 13:22:48.358487 21873 leader_election.cc:343] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.358898 22026 tablet_bootstrap.cc:654] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.360037 22024 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.368220 22024 ts_tablet_manager.cc:1403] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.004s
I20260709 13:22:48.373739 21834 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.375734 21984 raft_consensus.cc:2468] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:48.376214 21947 leader_election.cc:304] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.377417 21874 leader_election.cc:304] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.377852 22056 raft_consensus.cc:2804] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.381893 22056 raft_consensus.cc:697] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:48.383539 22062 raft_consensus.cc:2804] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.384405 22062 raft_consensus.cc:697] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:48.384866 22056 consensus_queue.cc:237] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.385432 22062 consensus_queue.cc:237] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.386343 21909 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.392565 22024 raft_consensus.cc:359] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.393483 22024 raft_consensus.cc:385] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.393831 22024 raft_consensus.cc:740] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
W20260709 13:22:48.391413 21799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.183.66:38507, user_credentials={real_user=slave}} blocked reactor thread for 70928us
I20260709 13:22:48.394703 22025 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:48.389689 22033 raft_consensus.cc:515] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.395393 22025 ts_tablet_manager.cc:1403] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.083s	user 0.017s	sys 0.003s
I20260709 13:22:48.396256 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.400630 21909 raft_consensus.cc:2393] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 434c406d785b40b982cd98e21b7b97c2 in this term.
I20260709 13:22:48.404599 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.405471 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.406461 21909 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.406225 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.395092 22024 consensus_queue.cc:260] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.409227 21799 leader_election.cc:304] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.411556 22024 ts_tablet_manager.cc:1434] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.043s	user 0.011s	sys 0.000s
I20260709 13:22:48.411918 22033 leader_election.cc:290] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.402576 22025 raft_consensus.cc:359] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.413615 22025 raft_consensus.cc:385] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.415091 22025 raft_consensus.cc:740] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.412994 22024 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:48.412747 22033 raft_consensus.cc:2749] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:22:48.416428 21799 leader_election.cc:343] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.418613 22025 consensus_queue.cc:260] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.422024 22025 ts_tablet_manager.cc:1434] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:22:48.426600 21909 raft_consensus.cc:2468] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 1.
I20260709 13:22:48.431102 22024 tablet_bootstrap.cc:654] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.431269 21799 leader_election.cc:304] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.432360 22026 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.433107 22026 ts_tablet_manager.cc:1403] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.114s	user 0.018s	sys 0.006s
I20260709 13:22:48.433012 21733 catalog_manager.cc:5697] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.435395 22033 raft_consensus.cc:2804] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.436098 22033 raft_consensus.cc:697] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:48.437294 22056 raft_consensus.cc:493] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.437101 22033 consensus_queue.cc:237] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.437734 22026 raft_consensus.cc:359] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.437949 22056 raft_consensus.cc:515] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.438644 22026 raft_consensus.cc:385] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.439057 22026 raft_consensus.cc:740] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.441102 22056 leader_election.cc:290] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.442234 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.443274 22024 tablet_bootstrap.cc:492] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.443928 22024 ts_tablet_manager.cc:1403] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.001s
I20260709 13:22:48.444398 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
W20260709 13:22:48.444787 21947 leader_election.cc:343] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.445243 21909 raft_consensus.cc:2468] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:48.446871 21948 leader_election.cc:304] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.448715 22056 raft_consensus.cc:2804] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.449365 22056 raft_consensus.cc:493] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.449882 22056 raft_consensus.cc:3060] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.449708 22024 raft_consensus.cc:359] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.450734 22024 raft_consensus.cc:385] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.451145 22024 raft_consensus.cc:740] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.445865 22026 consensus_queue.cc:260] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.452387 22024 consensus_queue.cc:260] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.454277 21730 catalog_manager.cc:5697] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.455478 22026 ts_tablet_manager.cc:1434] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.003s
I20260709 13:22:48.456933 22026 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.456789 21730 catalog_manager.cc:5697] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.454943 22024 ts_tablet_manager.cc:1434] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:22:48.462177 22024 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:48.467973 22056 raft_consensus.cc:515] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.469538 22026 tablet_bootstrap.cc:654] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.473106 22024 tablet_bootstrap.cc:654] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.473596 22029 raft_consensus.cc:493] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.478385 22029 raft_consensus.cc:515] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.480051 21733 catalog_manager.cc:5697] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 0 to 1, leader changed from <none> to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.483219 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.485723 22024 tablet_bootstrap.cc:492] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:48.486469 22024 ts_tablet_manager.cc:1403] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.002s
I20260709 13:22:48.487810 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.488814 21909 raft_consensus.cc:2393] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 434c406d785b40b982cd98e21b7b97c2 in current term 1: Already voted for candidate c3b3294fda6a4b578c7e149e6990b94e in this term.
I20260709 13:22:48.490386 21948 leader_election.cc:304] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2; no voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.490196 22024 raft_consensus.cc:359] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.491066 22024 raft_consensus.cc:385] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.491544 22058 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.491840 22029 leader_election.cc:290] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.494431 22056 leader_election.cc:290] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.498054 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.498620 22026 tablet_bootstrap.cc:492] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.498907 21909 raft_consensus.cc:3060] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.499841 22026 ts_tablet_manager.cc:1403] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.019s
I20260709 13:22:48.502525 21729 catalog_manager.cc:5697] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.503973 22058 raft_consensus.cc:2749] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.503696 22026 raft_consensus.cc:359] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.504958 22026 raft_consensus.cc:385] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.491588 22024 raft_consensus.cc:740] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.505458 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.505746 22026 raft_consensus.cc:740] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.506307 22024 consensus_queue.cc:260] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.507025 22026 consensus_queue.cc:260] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
W20260709 13:22:48.507587 21947 leader_election.cc:343] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.508924 22024 ts_tablet_manager.cc:1434] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.003s
I20260709 13:22:48.510782 22026 ts_tablet_manager.cc:1434] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.006s
I20260709 13:22:48.512389 22026 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.514698 21909 raft_consensus.cc:2468] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.516479 21948 leader_election.cc:304] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.518061 22058 raft_consensus.cc:2804] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.518884 22058 raft_consensus.cc:697] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:48.519912 22058 consensus_queue.cc:237] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.521253 22026 tablet_bootstrap.cc:654] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.528275 22058 raft_consensus.cc:493] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.529331 22058 raft_consensus.cc:515] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.530406 22026 tablet_bootstrap.cc:492] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.531476 22026 ts_tablet_manager.cc:1403] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:22:48.534219 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.535363 21909 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:48.534482 21730 catalog_manager.cc:5697] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.535983 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.535753 22026 raft_consensus.cc:359] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.537051 21834 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:48.537137 22026 raft_consensus.cc:385] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.537511 22058 leader_election.cc:290] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.537952 22026 raft_consensus.cc:740] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.537196 21948 leader_election.cc:304] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:48.539050 22058 raft_consensus.cc:2804] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.539594 22058 raft_consensus.cc:493] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.539227 22026 consensus_queue.cc:260] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.540011 22058 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.543375 22026 ts_tablet_manager.cc:1434] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 13:22:48.544965 22026 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.551399 22058 raft_consensus.cc:515] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.552807 22026 tablet_bootstrap.cc:654] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.554204 22058 leader_election.cc:290] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:48.555790 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.556600 21834 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.556811 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.557962 21909 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.565165 22026 tablet_bootstrap.cc:492] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.565963 22026 ts_tablet_manager.cc:1403] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:22:48.567618 21834 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.569188 21947 leader_election.cc:304] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.569099 22026 raft_consensus.cc:359] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.570232 21909 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:48.570559 22026 raft_consensus.cc:385] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.570643 22058 raft_consensus.cc:2804] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.571120 22026 raft_consensus.cc:740] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.571470 22058 raft_consensus.cc:697] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:48.572113 22026 consensus_queue.cc:260] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.572487 22058 consensus_queue.cc:237] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.574609 22026 ts_tablet_manager.cc:1434] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:48.575995 22026 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:48.583191 22026 tablet_bootstrap.cc:654] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.582753 21729 catalog_manager.cc:5697] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.589262 22026 tablet_bootstrap.cc:492] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:48.589895 22026 ts_tablet_manager.cc:1403] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 13:22:48.592969 22026 raft_consensus.cc:359] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.593596 22026 raft_consensus.cc:385] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.593930 22026 raft_consensus.cc:740] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.594625 22026 consensus_queue.cc:260] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.596499 22026 ts_tablet_manager.cc:1434] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:22:48.611842 22062 raft_consensus.cc:493] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.612576 22062 raft_consensus.cc:515] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.614893 22062 leader_election.cc:290] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.616057 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d3d31d96d8f4a38a997ed65bb87e6d0" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:48.616166 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d3d31d96d8f4a38a997ed65bb87e6d0" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.616969 21834 raft_consensus.cc:2468] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.617074 21983 raft_consensus.cc:2468] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:48.618376 21873 leader_election.cc:304] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.619313 22062 raft_consensus.cc:2804] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.619740 22062 raft_consensus.cc:493] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.620018 22062 raft_consensus.cc:3060] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.625353 22033 raft_consensus.cc:493] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.626191 22033 raft_consensus.cc:515] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.627485 22062 raft_consensus.cc:515] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.629230 22033 leader_election.cc:290] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.630278 22062 leader_election.cc:290] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.630344 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.630995 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5e0712f6a964772870a3eca408dac96" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.631937 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d3d31d96d8f4a38a997ed65bb87e6d0" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:48.631961 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d3d31d96d8f4a38a997ed65bb87e6d0" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.632794 21983 raft_consensus.cc:2393] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 019e784c7f884e0782e9dca6055b7a4b in this term.
I20260709 13:22:48.633249 21834 raft_consensus.cc:3060] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.634287 21799 leader_election.cc:304] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.633276 21984 raft_consensus.cc:3060] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.635336 22033 raft_consensus.cc:3060] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.642208 21984 raft_consensus.cc:2468] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:48.643343 21834 raft_consensus.cc:2468] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:48.643975 21874 leader_election.cc:304] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:48.644694 22033 raft_consensus.cc:2749] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.645291 22062 raft_consensus.cc:2804] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.645900 22062 raft_consensus.cc:697] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:48.646711 22062 consensus_queue.cc:237] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.657740 21729 catalog_manager.cc:5697] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.685761 21984 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.689491 22058 consensus_queue.cc:1048] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.697018 22062 consensus_queue.cc:1048] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.716770 22058 consensus_queue.cc:1048] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.720824 22045 consensus_queue.cc:1048] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.775422 21984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:48.776638 21984 raft_consensus.cc:606] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.796907 21909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:48892
I20260709 13:22:48.797612 21909 raft_consensus.cc:606] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.805389 21909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5466e4df67bb43c599d51a521771792f"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:48892
I20260709 13:22:48.806342 21909 raft_consensus.cc:606] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.813654 21984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:48.814499 21984 raft_consensus.cc:606] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:48.821305 22045 consensus_queue.cc:1048] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.834146 22033 raft_consensus.cc:493] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.835187 22033 raft_consensus.cc:515] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.836556 21835 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:33938
I20260709 13:22:48.837433 21835 raft_consensus.cc:606] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:48.839190 22062 consensus_queue.cc:1048] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.839375 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.841760 22033 leader_election.cc:290] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.844034 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.845242 21983 raft_consensus.cc:2393] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 019e784c7f884e0782e9dca6055b7a4b in this term.
I20260709 13:22:48.846850 21799 leader_election.cc:304] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.848495 22067 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.862557 22033 consensus_queue.cc:1048] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.864637 21984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:48.865624 21984 raft_consensus.cc:606] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:48.866387 22067 raft_consensus.cc:2749] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.873479 22062 consensus_queue.cc:1048] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.881541 22053 consensus_queue.cc:1048] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.889397 22053 raft_consensus.cc:993] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:48.889976 22053 raft_consensus.cc:1081] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:48.893632 21835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5466e4df67bb43c599d51a521771792f"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33908
I20260709 13:22:48.894781 21835 raft_consensus.cc:493] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:48.895455 21835 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:48.901696 22098 consensus_queue.cc:1048] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.911268 21835 raft_consensus.cc:515] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.915088 21834 raft_consensus.cc:1240] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Rejecting Update request from peer 019e784c7f884e0782e9dca6055b7a4b for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:48.917163 22029 consensus_queue.cc:1048] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.921427 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:48.919906 21835 leader_election.cc:290] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.922791 21984 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:48.923174 21908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5466e4df67bb43c599d51a521771792f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:48.928202 21908 raft_consensus.cc:3055] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:48.928653 21908 raft_consensus.cc:740] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:48.926975 22058 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:22:48.930795 21908 consensus_queue.cc:260] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.935259 22062 consensus_queue.cc:1059] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.954335 21908 raft_consensus.cc:3060] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:48.963485 22033 raft_consensus.cc:493] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.964574 22033 raft_consensus.cc:515] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.969412 22033 leader_election.cc:290] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:48.972338 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:48.972748 21908 raft_consensus.cc:2468] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:48.975620 22098 raft_consensus.cc:993] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:48.976145 21799 leader_election.cc:304] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:48.976297 22098 raft_consensus.cc:1081] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:48.977663 22098 raft_consensus.cc:2804] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:48.977857 21834 raft_consensus.cc:3060] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.978241 22098 raft_consensus.cc:697] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:48.979321 22098 consensus_queue.cc:237] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:48.982002 21909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:22:48.975662 21984 raft_consensus.cc:2468] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:48.983084 21909 raft_consensus.cc:493] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:48.983605 21909 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:48.987910 22058 raft_consensus.cc:993] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:48.982254 21908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14a9ebf91f79450f904dbccb7d51ea68" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:48.989888 21799 leader_election.cc:304] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:48.993377 22058 raft_consensus.cc:1081] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:48.994144 22098 raft_consensus.cc:2749] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.998409 22056 consensus_queue.cc:1048] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.999603 21835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33922
I20260709 13:22:49.000320 21835 raft_consensus.cc:493] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:49.000753 21835 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.004580 22056 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.012496 21835 raft_consensus.cc:515] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.016204 21909 raft_consensus.cc:515] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.021152 21909 leader_election.cc:290] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:49.022106 21835 leader_election.cc:290] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:49.022004 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:49.022986 21984 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.023538 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:49.024284 21983 raft_consensus.cc:3055] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:49.024585 21983 raft_consensus.cc:740] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:49.025744 21983 consensus_queue.cc:260] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.026100 21730 catalog_manager.cc:5697] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.024549 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:49.027415 21983 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.028743 21909 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.032027 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d5539e7e2024cf38082a16791bfc8c8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:49.032855 21835 raft_consensus.cc:3055] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:49.032986 21984 raft_consensus.cc:2468] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:49.033281 21835 raft_consensus.cc:740] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:49.034768 21874 leader_election.cc:304] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:49.035920 22062 raft_consensus.cc:2804] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:49.036430 22062 raft_consensus.cc:697] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:49.037370 22062 consensus_queue.cc:237] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.034281 21835 consensus_queue.cc:260] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.038312 21983 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.039526 21835 raft_consensus.cc:3060] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.044287 21799 leader_election.cc:304] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:49.045569 22098 raft_consensus.cc:2804] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:49.046103 22098 raft_consensus.cc:697] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:49.047084 22098 consensus_queue.cc:237] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.052594 21835 raft_consensus.cc:2468] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:49.055148 21909 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.062654 21730 catalog_manager.cc:5697] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.072857 21729 catalog_manager.cc:5697] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.095384 22029 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:49.112924 22029 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.124650 22058 raft_consensus.cc:993] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:49.125244 22106 raft_consensus.cc:1081] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:49.127568 21909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:22:49.128389 21909 raft_consensus.cc:493] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:49.128842 21909 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.136459 21909 raft_consensus.cc:515] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.139647 21909 leader_election.cc:290] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:49.140304 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:49.140301 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:49.141515 21984 raft_consensus.cc:3055] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:49.141906 21984 raft_consensus.cc:740] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:49.142004 21835 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.143029 21984 consensus_queue.cc:260] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.144752 21984 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.152304 22106 raft_consensus.cc:993] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:49.152304 21835 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:49.153311 22106 raft_consensus.cc:1081] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:49.154592 21873 leader_election.cc:304] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:49.155997 22062 raft_consensus.cc:2804] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:49.156188 21835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33922
I20260709 13:22:49.156700 22062 raft_consensus.cc:697] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:49.157226 21835 raft_consensus.cc:493] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:49.156668 21984 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:49.157790 21835 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.158021 22062 consensus_queue.cc:237] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.168155 21835 raft_consensus.cc:515] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.168277 21730 catalog_manager.cc:5697] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.170394 21835 leader_election.cc:290] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:49.171710 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:49.172256 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0693a0ddbfa747b7b5c8ef13feeee767" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:49.172602 21909 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.172952 21984 raft_consensus.cc:3055] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:49.173305 21984 raft_consensus.cc:740] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:49.174285 21984 consensus_queue.cc:260] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.175881 21984 raft_consensus.cc:3060] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.181874 22062 consensus_queue.cc:1048] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.185230 21984 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.185230 21909 raft_consensus.cc:2468] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.187278 21799 leader_election.cc:304] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:49.189095 22098 raft_consensus.cc:2804] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:49.189708 22098 raft_consensus.cc:697] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:49.190712 22098 consensus_queue.cc:237] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.200846 22031 consensus_queue.cc:1048] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.204917 21729 catalog_manager.cc:5697] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.295586 22031 raft_consensus.cc:993] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:49.296131 22045 raft_consensus.cc:1081] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:49.298177 21835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33908
I20260709 13:22:49.298892 21835 raft_consensus.cc:493] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:49.299396 21835 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.307713 21835 raft_consensus.cc:515] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.309680 21835 leader_election.cc:290] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:49.311717 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:49.311872 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fb45406e4684da6b8696c4c0f5da2f3" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:49.312767 21909 raft_consensus.cc:3055] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:49.312880 21984 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.313309 21909 raft_consensus.cc:740] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:49.314551 21909 consensus_queue.cc:260] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.316037 21909 raft_consensus.cc:3060] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:49.322611 21984 raft_consensus.cc:2468] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.324345 21799 leader_election.cc:304] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:49.324419 21909 raft_consensus.cc:2468] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:49.325536 22098 raft_consensus.cc:2804] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:49.326164 22098 raft_consensus.cc:697] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:49.327045 22098 consensus_queue.cc:237] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:49.335006 21730 catalog_manager.cc:5697] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:49.431268 21909 raft_consensus.cc:1275] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.433071 22098 consensus_queue.cc:1048] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:49.444849 21984 raft_consensus.cc:1275] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.448930 22110 consensus_queue.cc:1048] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:49.466907 21984 raft_consensus.cc:1275] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.468791 22045 consensus_queue.cc:1048] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.479081 21835 raft_consensus.cc:1275] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.480786 22053 consensus_queue.cc:1048] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:49.542459 21984 raft_consensus.cc:1275] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.544294 22053 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.554991 21835 raft_consensus.cc:1275] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.557379 22053 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.564895 21984 raft_consensus.cc:1275] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.566787 22110 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.577168 21910 raft_consensus.cc:1275] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.579766 22033 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.660406 21910 raft_consensus.cc:1275] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.662369 22098 consensus_queue.cc:1048] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:49.672814 21984 raft_consensus.cc:1275] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.674777 22110 consensus_queue.cc:1048] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.871747 21983 raft_consensus.cc:1275] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.874258 22110 consensus_queue.cc:1048] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:49.887081 21910 raft_consensus.cc:1275] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:49.889760 22098 consensus_queue.cc:1048] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:51.908953 21733 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52912:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:51.911643 21733 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:51.972559 21819 tablet_service.cc:1511] Processing CreateTablet for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:51.973343 21894 tablet_service.cc:1511] Processing CreateTablet for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:51.974189 21819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.974941 21894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.975404 21967 tablet_service.cc:1511] Processing CreateTablet for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:22:51.976985 21967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ceb7deddda64d9a9a92cbdd45ca1cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.983639 21820 tablet_service.cc:1511] Processing CreateTablet for tablet ba04bc19cddf4b8a9b20658ba3dac173 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:51.986217 21820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba04bc19cddf4b8a9b20658ba3dac173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.991766 21893 tablet_service.cc:1511] Processing CreateTablet for tablet ba04bc19cddf4b8a9b20658ba3dac173 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:51.993549 21893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba04bc19cddf4b8a9b20658ba3dac173. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:51.994522 21781 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:51.994753 21968 tablet_service.cc:1511] Processing CreateTablet for tablet ba04bc19cddf4b8a9b20658ba3dac173 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:22:51.996609 21968 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba04bc19cddf4b8a9b20658ba3dac173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.001708 21817 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:52.003507 21817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.004688 21892 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:52.006367 21892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.007902 21970 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:22:52.009588 21970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.014650 21823 tablet_service.cc:1511] Processing CreateTablet for tablet 24371feb96a24bf68b9db97aa7dbc13f (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:52.016525 21823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24371feb96a24bf68b9db97aa7dbc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.017262 21900 tablet_service.cc:1511] Processing CreateTablet for tablet 24371feb96a24bf68b9db97aa7dbc13f (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:52.020326 21966 tablet_service.cc:1511] Processing CreateTablet for tablet 24371feb96a24bf68b9db97aa7dbc13f (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:22:52.021978 21966 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24371feb96a24bf68b9db97aa7dbc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.018930 21900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24371feb96a24bf68b9db97aa7dbc13f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.028798 21818 tablet_service.cc:1511] Processing CreateTablet for tablet d3dcb75d93364aa5bbd14334506279a5 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:52.029453 21895 tablet_service.cc:1511] Processing CreateTablet for tablet d3dcb75d93364aa5bbd14334506279a5 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:52.030709 21818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dcb75d93364aa5bbd14334506279a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.030915 21895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dcb75d93364aa5bbd14334506279a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.032827 21971 tablet_service.cc:1511] Processing CreateTablet for tablet d3dcb75d93364aa5bbd14334506279a5 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:22:52.034578 21971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3dcb75d93364aa5bbd14334506279a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.042822 21899 tablet_service.cc:1511] Processing CreateTablet for tablet a165d14408d1482e9e70ba2bef7f040b (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:52.044895 21899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165d14408d1482e9e70ba2bef7f040b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.047298 21969 tablet_service.cc:1511] Processing CreateTablet for tablet a165d14408d1482e9e70ba2bef7f040b (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:52.049153 21969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165d14408d1482e9e70ba2bef7f040b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.053566 21897 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad026d6c08b443c95483cf1dae10334 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:52.042822 21825 tablet_service.cc:1511] Processing CreateTablet for tablet a165d14408d1482e9e70ba2bef7f040b (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:22:52.056650 21825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a165d14408d1482e9e70ba2bef7f040b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.061662 21972 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad026d6c08b443c95483cf1dae10334 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:52.063371 21897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad026d6c08b443c95483cf1dae10334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.067811 21896 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe76cb5c2434e588405c55b2f5ceeaf (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:52.069644 21896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe76cb5c2434e588405c55b2f5ceeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.063424 21972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad026d6c08b443c95483cf1dae10334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.084806 21973 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe76cb5c2434e588405c55b2f5ceeaf (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:52.087863 21822 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad026d6c08b443c95483cf1dae10334 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:22:52.089965 21822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad026d6c08b443c95483cf1dae10334. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.092162 21973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe76cb5c2434e588405c55b2f5ceeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.100744 21821 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe76cb5c2434e588405c55b2f5ceeaf (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:22:52.102710 21821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe76cb5c2434e588405c55b2f5ceeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.132040 21892 tablet_service.cc:1511] Processing CreateTablet for tablet df75b588722b4192b84352600b2161a8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:52.134580 21892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df75b588722b4192b84352600b2161a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.131660 21824 tablet_service.cc:1511] Processing CreateTablet for tablet df75b588722b4192b84352600b2161a8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:52.140678 21974 tablet_service.cc:1511] Processing CreateTablet for tablet df75b588722b4192b84352600b2161a8 (DEFAULT_TABLE table=soft_deleted_table [id=526e38cc2e844c19852625857da9d279]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:22:52.142602 21974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df75b588722b4192b84352600b2161a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.140774 21824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df75b588722b4192b84352600b2161a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:52.219992 22124 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.220044 22125 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.220849 22127 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.229470 22127 tablet_bootstrap.cc:654] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.232669 22125 tablet_bootstrap.cc:654] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.274821 22124 tablet_bootstrap.cc:654] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.284149 22127 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.284847 22127 ts_tablet_manager.cc:1403] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.000s
I20260709 13:22:52.284933 22124 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.285060 22125 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.286020 22125 ts_tablet_manager.cc:1403] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.031s
I20260709 13:22:52.285887 22124 ts_tablet_manager.cc:1403] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.005s
I20260709 13:22:52.288631 22127 raft_consensus.cc:359] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.289227 22125 raft_consensus.cc:359] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.289764 22127 raft_consensus.cc:385] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.290237 22125 raft_consensus.cc:385] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.290336 22127 raft_consensus.cc:740] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.290714 22125 raft_consensus.cc:740] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.291102 22124 raft_consensus.cc:359] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.291613 22127 consensus_queue.cc:260] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.292210 22124 raft_consensus.cc:385] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.291983 22125 consensus_queue.cc:260] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.292639 22124 raft_consensus.cc:740] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.293614 22124 consensus_queue.cc:260] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.295564 22125 ts_tablet_manager.cc:1434] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:22:52.296717 22124 ts_tablet_manager.cc:1434] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:22:52.297091 22125 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.298224 22124 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.302811 22127 ts_tablet_manager.cc:1434] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:22:52.304308 22127 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.305985 22125 tablet_bootstrap.cc:654] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.307509 22126 raft_consensus.cc:493] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.308159 22126 raft_consensus.cc:515] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.311756 22124 tablet_bootstrap.cc:654] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.314409 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.312196 22127 tablet_bootstrap.cc:654] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:52.316362 21799 leader_election.cc:343] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.318780 22126 leader_election.cc:290] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.311909 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
W20260709 13:22:52.324023 21799 leader_election.cc:343] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.324609 21799 leader_election.cc:304] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:52.325934 22126 raft_consensus.cc:2749] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:52.332077 22124 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.332904 22124 ts_tablet_manager.cc:1403] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.000s
I20260709 13:22:52.338923 22125 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.339864 22125 ts_tablet_manager.cc:1403] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.015s
I20260709 13:22:52.342239 22124 raft_consensus.cc:359] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.343407 22124 raft_consensus.cc:385] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.343773 22124 raft_consensus.cc:740] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.344712 22124 consensus_queue.cc:260] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.348057 22124 ts_tablet_manager.cc:1434] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:22:52.349453 22124 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.350845 22125 raft_consensus.cc:359] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.352049 22125 raft_consensus.cc:385] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.352456 22125 raft_consensus.cc:740] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.353578 22125 consensus_queue.cc:260] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.359081 22125 ts_tablet_manager.cc:1434] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:22:52.359284 22124 tablet_bootstrap.cc:654] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.363951 22125 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.364041 22127 tablet_bootstrap.cc:492] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.373508 22125 tablet_bootstrap.cc:654] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.376636 22127 ts_tablet_manager.cc:1403] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.005s
I20260709 13:22:52.379333 22124 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.380343 22124 ts_tablet_manager.cc:1403] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.009s
I20260709 13:22:52.380623 22127 raft_consensus.cc:359] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.381598 22127 raft_consensus.cc:385] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.382409 22127 raft_consensus.cc:740] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.383301 22127 consensus_queue.cc:260] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.385058 22124 raft_consensus.cc:359] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.385797 22127 ts_tablet_manager.cc:1434] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:52.388512 22125 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.389535 22125 ts_tablet_manager.cc:1403] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.009s
I20260709 13:22:52.389726 22127 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.385964 22124 raft_consensus.cc:385] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.391899 22124 raft_consensus.cc:740] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.393103 22124 consensus_queue.cc:260] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.395234 22125 raft_consensus.cc:359] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.396541 22125 raft_consensus.cc:385] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.396915 22125 raft_consensus.cc:740] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.402508 22127 tablet_bootstrap.cc:654] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.407284 22124 ts_tablet_manager.cc:1434] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.001s
I20260709 13:22:52.409592 22124 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.408018 22125 consensus_queue.cc:260] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.419915 22127 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.420282 22125 ts_tablet_manager.cc:1434] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 13:22:52.420835 22127 ts_tablet_manager.cc:1403] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 13:22:52.421671 22125 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.425676 22127 raft_consensus.cc:359] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.426678 22127 raft_consensus.cc:385] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.427047 22127 raft_consensus.cc:740] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.428133 22127 consensus_queue.cc:260] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.429934 22125 tablet_bootstrap.cc:654] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.438634 22127 ts_tablet_manager.cc:1434] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.005s
I20260709 13:22:52.440018 22053 raft_consensus.cc:493] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.440251 22127 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.440759 22053 raft_consensus.cc:515] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.444773 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.445747 21835 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:52.434736 22124 tablet_bootstrap.cc:654] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.448035 21873 leader_election.cc:304] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:52.449247 22113 raft_consensus.cc:2804] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.449519 22127 tablet_bootstrap.cc:654] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.449719 22113 raft_consensus.cc:493] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.450103 22113 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.459086 22124 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.459635 22125 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.460073 22124 ts_tablet_manager.cc:1403] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.005s
I20260709 13:22:52.459667 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.461239 21984 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:52.460212 22125 ts_tablet_manager.cc:1403] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.019s
I20260709 13:22:52.463824 22127 tablet_bootstrap.cc:492] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.464673 22127 ts_tablet_manager.cc:1403] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.012s
I20260709 13:22:52.464998 22124 raft_consensus.cc:359] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.465265 22125 raft_consensus.cc:359] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.466023 22124 raft_consensus.cc:385] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.466317 22125 raft_consensus.cc:385] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.466444 22124 raft_consensus.cc:740] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.466814 22125 raft_consensus.cc:740] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.467646 22124 consensus_queue.cc:260] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.467686 22125 consensus_queue.cc:260] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.469494 22127 raft_consensus.cc:359] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.470718 22127 raft_consensus.cc:385] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.470798 22053 leader_election.cc:290] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.471201 22127 raft_consensus.cc:740] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.476348 22127 consensus_queue.cc:260] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.472200 22125 ts_tablet_manager.cc:1434] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:52.479374 22125 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.480101 22127 ts_tablet_manager.cc:1434] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:22:52.481827 22127 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.490319 22125 tablet_bootstrap.cc:654] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.490522 22127 tablet_bootstrap.cc:654] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.515053 22113 raft_consensus.cc:515] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.516692 22125 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.517608 22125 ts_tablet_manager.cc:1403] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.039s	user 0.023s	sys 0.003s
I20260709 13:22:52.521994 22127 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.521981 22125 raft_consensus.cc:359] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.524447 22125 raft_consensus.cc:385] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.524832 22125 raft_consensus.cc:740] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.527143 22127 ts_tablet_manager.cc:1403] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.046s	user 0.024s	sys 0.004s
I20260709 13:22:52.531008 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.532233 21835 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.531209 22113 leader_election.cc:290] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.531008 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:52.533581 21984 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.534348 22127 raft_consensus.cc:359] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.535392 22127 raft_consensus.cc:385] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.535709 22127 raft_consensus.cc:740] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.526033 22125 consensus_queue.cc:260] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.536466 22124 ts_tablet_manager.cc:1434] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.076s	user 0.033s	sys 0.025s
I20260709 13:22:52.538195 22124 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.541917 22125 ts_tablet_manager.cc:1434] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:22:52.543778 22125 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.546921 22124 tablet_bootstrap.cc:654] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.550294 21984 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:52.551636 22127 consensus_queue.cc:260] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.552547 21874 leader_election.cc:304] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.554615 22113 raft_consensus.cc:2804] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:52.555419 22113 raft_consensus.cc:697] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:52.556387 22113 consensus_queue.cc:237] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.559854 22127 ts_tablet_manager.cc:1434] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 13:22:52.561343 22124 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.562014 22125 tablet_bootstrap.cc:654] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.562048 22124 ts_tablet_manager.cc:1403] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:22:52.567049 22127 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.567364 22124 raft_consensus.cc:359] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.565392 22140 raft_consensus.cc:493] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.570093 22140 raft_consensus.cc:515] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.573189 22140 leader_election.cc:290] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.574781 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.576979 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.577890 21984 raft_consensus.cc:2468] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:52.578933 22127 tablet_bootstrap.cc:654] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.579205 21874 leader_election.cc:304] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.580506 22053 raft_consensus.cc:493] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.581045 22053 raft_consensus.cc:515] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.568920 22124 raft_consensus.cc:385] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.583041 22124 raft_consensus.cc:740] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.584180 22053 leader_election.cc:290] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.584350 22124 consensus_queue.cc:260] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.587059 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
W20260709 13:22:52.590489 21874 leader_election.cc:343] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.594096 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
W20260709 13:22:52.595983 21873 leader_election.cc:343] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.596429 22124 ts_tablet_manager.cc:1434] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.003s
I20260709 13:22:52.597611 22124 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
W20260709 13:22:52.599172 21873 leader_election.cc:343] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.585222 22053 raft_consensus.cc:2804] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.600294 21873 leader_election.cc:304] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b; no voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:52.600634 22053 raft_consensus.cc:493] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.602550 22053 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.605310 22124 tablet_bootstrap.cc:654] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.606233 22140 raft_consensus.cc:2749] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:52.624759 22053 raft_consensus.cc:515] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.610036 21835 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:52.617065 21729 catalog_manager.cc:5697] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:52.632287 22127 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.633080 22127 ts_tablet_manager.cc:1403] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.002s
I20260709 13:22:52.635076 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.636740 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:52.638275 21984 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.637446 22125 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.639210 22125 ts_tablet_manager.cc:1403] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.096s	user 0.021s	sys 0.032s
I20260709 13:22:52.636822 22127 raft_consensus.cc:359] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.640655 22127 raft_consensus.cc:385] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.640983 22127 raft_consensus.cc:740] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.641813 22124 tablet_bootstrap.cc:492] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.642156 22127 consensus_queue.cc:260] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.642890 22124 ts_tablet_manager.cc:1403] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.014s
I20260709 13:22:52.643052 22125 raft_consensus.cc:359] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.644066 22125 raft_consensus.cc:385] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.644382 22125 raft_consensus.cc:740] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.645398 22125 consensus_queue.cc:260] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.646325 22053 leader_election.cc:290] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
W20260709 13:22:52.641283 21873 leader_election.cc:343] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.654102 21984 raft_consensus.cc:2468] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 1.
I20260709 13:22:52.648547 22125 ts_tablet_manager.cc:1434] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:22:52.646675 22124 raft_consensus.cc:359] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.655242 22124 raft_consensus.cc:385] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.655608 22124 raft_consensus.cc:740] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.655972 21874 leader_election.cc:304] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:52.656664 22124 consensus_queue.cc:260] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.658124 22053 raft_consensus.cc:2804] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:52.659384 22053 raft_consensus.cc:697] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:52.660336 22124 ts_tablet_manager.cc:1434] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:22:52.661800 22125 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.662333 22124 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.648492 22127 ts_tablet_manager.cc:1434] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:22:52.660485 22053 consensus_queue.cc:237] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.676915 22127 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.677786 22029 raft_consensus.cc:493] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.678853 22125 tablet_bootstrap.cc:654] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.678553 22029 raft_consensus.cc:515] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.681509 22029 leader_election.cc:290] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.683501 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.683933 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.684497 21910 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.686578 21948 leader_election.cc:304] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.684578 21835 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.688153 22029 raft_consensus.cc:2804] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.688746 22029 raft_consensus.cc:493] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.689096 22029 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.692121 22127 tablet_bootstrap.cc:654] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.694200 22125 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.694968 22125 ts_tablet_manager.cc:1403] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.033s	user 0.017s	sys 0.003s
I20260709 13:22:52.698649 22124 tablet_bootstrap.cc:654] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.698400 22125 raft_consensus.cc:359] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.699323 22125 raft_consensus.cc:385] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.699659 22125 raft_consensus.cc:740] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.700414 22125 consensus_queue.cc:260] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.702970 22125 ts_tablet_manager.cc:1434] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.004s
I20260709 13:22:52.704452 22125 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.707093 22127 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.707861 22127 ts_tablet_manager.cc:1403] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.004s
I20260709 13:22:52.713022 21729 catalog_manager.cc:5697] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 1 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:52.717677 22125 tablet_bootstrap.cc:654] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.728593 22029 raft_consensus.cc:515] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.720155 22127 raft_consensus.cc:359] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.732539 22127 raft_consensus.cc:385] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.732874 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.733035 22127 raft_consensus.cc:740] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.733637 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:52.733824 21835 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.734347 21910 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.734351 22127 consensus_queue.cc:260] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.735898 22125 tablet_bootstrap.cc:492] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
I20260709 13:22:52.736505 22125 ts_tablet_manager.cc:1403] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.007s
I20260709 13:22:52.737826 22127 ts_tablet_manager.cc:1434] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:22:52.741047 22125 raft_consensus.cc:359] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.742009 22125 raft_consensus.cc:385] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.742321 22125 raft_consensus.cc:740] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.742539 22127 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.743214 22125 consensus_queue.cc:260] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.745779 22125 ts_tablet_manager.cc:1434] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:22:52.747305 22029 leader_election.cc:290] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.753228 21910 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:52.755491 21948 leader_election.cc:304] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.759960 22029 raft_consensus.cc:2804] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:52.760565 22127 tablet_bootstrap.cc:654] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.748945 22125 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: Bootstrap starting.
I20260709 13:22:52.761204 22029 raft_consensus.cc:697] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:52.762697 22029 consensus_queue.cc:237] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.770257 22125 tablet_bootstrap.cc:654] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.777102 22127 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.777827 22127 ts_tablet_manager.cc:1403] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 13:22:52.785369 22127 raft_consensus.cc:359] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.786387 22127 raft_consensus.cc:385] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.786751 22127 raft_consensus.cc:740] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.788069 22029 raft_consensus.cc:493] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.789263 22029 raft_consensus.cc:515] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.791600 21733 catalog_manager.cc:5697] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:52.793835 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.794566 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.793875 22127 consensus_queue.cc:260] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.795336 21910 raft_consensus.cc:2468] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.797092 22125 tablet_bootstrap.cc:492] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: No bootstrap required, opened a new log
W20260709 13:22:52.796907 21947 leader_election.cc:343] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.797788 22125 ts_tablet_manager.cc:1403] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.009s
I20260709 13:22:52.798645 21948 leader_election.cc:304] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:52.797022 22127 ts_tablet_manager.cc:1434] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:22:52.799840 22158 raft_consensus.cc:2804] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.800318 22158 raft_consensus.cc:493] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.800674 22158 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.800937 22127 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: Bootstrap starting.
I20260709 13:22:52.801491 22125 raft_consensus.cc:359] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.802325 22125 raft_consensus.cc:385] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.802634 22125 raft_consensus.cc:740] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.807061 22125 consensus_queue.cc:260] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.809804 22125 ts_tablet_manager.cc:1434] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:22:52.810329 22029 leader_election.cc:290] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.816226 22158 raft_consensus.cc:515] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.818934 22127 tablet_bootstrap.cc:654] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.823248 22126 raft_consensus.cc:493] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.824203 22126 raft_consensus.cc:515] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.829833 22053 raft_consensus.cc:493] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.830485 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.830649 22053 raft_consensus.cc:515] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.831789 22127 tablet_bootstrap.cc:492] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: No bootstrap required, opened a new log
I20260709 13:22:52.832423 22127 ts_tablet_manager.cc:1403] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.000s
I20260709 13:22:52.834980 22029 raft_consensus.cc:493] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.835043 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.836164 22029 raft_consensus.cc:515] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.837182 21984 raft_consensus.cc:2393] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 019e784c7f884e0782e9dca6055b7a4b in this term.
I20260709 13:22:52.840155 21799 leader_election.cc:304] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:52.841361 22067 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.841482 22127 raft_consensus.cc:359] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.842363 22127 raft_consensus.cc:385] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.842394 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.843781 21910 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.842733 22127 raft_consensus.cc:740] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.845801 21948 leader_election.cc:304] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.845861 22127 consensus_queue.cc:260] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.849058 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.849992 22029 leader_election.cc:290] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.851063 22126 leader_election.cc:290] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.853755 22158 leader_election.cc:290] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.835186 22053 leader_election.cc:290] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.850342 21984 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 0.
I20260709 13:22:52.862032 21874 leader_election.cc:304] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.862708 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:52.863715 22053 raft_consensus.cc:2804] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.864068 21910 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.864501 22053 raft_consensus.cc:493] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.865166 22053 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.867429 22126 raft_consensus.cc:493] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.867493 22029 raft_consensus.cc:2804] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.868330 22126 raft_consensus.cc:515] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.870127 22029 raft_consensus.cc:493] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.870656 22029 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.875725 22164 raft_consensus.cc:493] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.876601 22164 raft_consensus.cc:515] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.880081 22067 raft_consensus.cc:2749] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:52.883755 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.888270 21835 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:52.891947 21910 raft_consensus.cc:2468] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:52.870954 22126 leader_election.cc:290] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.883752 22164 leader_election.cc:290] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.895408 22029 raft_consensus.cc:515] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
W20260709 13:22:52.900120 21948 leader_election.cc:343] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.902449 21948 leader_election.cc:304] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:52.904474 22165 raft_consensus.cc:2804] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:52.905273 22165 raft_consensus.cc:697] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:52.906021 22124 tablet_bootstrap.cc:492] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.906350 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.906783 22124 ts_tablet_manager.cc:1403] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.245s	user 0.048s	sys 0.058s
I20260709 13:22:52.907145 21834 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.908336 21833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.909351 21833 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.910705 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.910974 22124 raft_consensus.cc:359] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.911880 21984 raft_consensus.cc:2468] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 0.
I20260709 13:22:52.907060 22165 consensus_queue.cc:237] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.912477 22127 ts_tablet_manager.cc:1434] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: Time spent starting tablet: real 0.080s	user 0.021s	sys 0.014s
I20260709 13:22:52.911837 22124 raft_consensus.cc:385] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.913697 22124 raft_consensus.cc:740] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.914188 21799 leader_election.cc:304] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:52.914358 21833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.914588 22124 consensus_queue.cc:260] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.915342 22126 raft_consensus.cc:2804] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.916003 21833 raft_consensus.cc:2468] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:52.916638 22126 raft_consensus.cc:493] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.917070 22126 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.918386 22124 ts_tablet_manager.cc:1434] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:22:52.918776 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.920022 22124 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:52.922775 22029 leader_election.cc:290] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:52.927802 21833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.940241 21833 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.940794 22053 raft_consensus.cc:515] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.945303 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:52.946267 21984 raft_consensus.cc:2393] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 019e784c7f884e0782e9dca6055b7a4b in current term 1: Already voted for candidate 434c406d785b40b982cd98e21b7b97c2 in this term.
W20260709 13:22:52.932791 21799 leader_election.cc:343] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.934552 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:52.927954 22124 tablet_bootstrap.cc:654] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:52.947399 21730 catalog_manager.cc:5697] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:52.950980 21835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.935824 22166 raft_consensus.cc:493] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.936421 21947 leader_election.cc:304] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:52.953328 22166 raft_consensus.cc:515] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.927065 21834 raft_consensus.cc:2468] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:52.954583 22165 raft_consensus.cc:2804] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.955164 22165 raft_consensus.cc:493] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.955559 22165 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.956271 22147 consensus_queue.cc:1048] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:52.959316 21833 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:52.960769 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:52.961870 21909 raft_consensus.cc:2468] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 0.
I20260709 13:22:52.948048 22053 leader_election.cc:290] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.935181 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:52.967689 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.968750 21910 raft_consensus.cc:2393] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 434c406d785b40b982cd98e21b7b97c2 in current term 1: Already voted for candidate 019e784c7f884e0782e9dca6055b7a4b in this term.
I20260709 13:22:52.968755 21984 raft_consensus.cc:2468] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 0.
I20260709 13:22:52.969882 22167 raft_consensus.cc:493] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.970639 22167 raft_consensus.cc:515] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.973345 22167 leader_election.cc:290] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.973634 21799 leader_election.cc:304] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:52.974058 22166 leader_election.cc:290] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:52.974911 22166 raft_consensus.cc:2804] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:52.975378 22166 raft_consensus.cc:493] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.975718 22166 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:52.977306 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
W20260709 13:22:52.981082 21799 leader_election.cc:343] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:52.978914 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2" is_pre_election: true
I20260709 13:22:52.970008 22124 tablet_bootstrap.cc:492] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:52.982225 22124 ts_tablet_manager.cc:1403] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.009s
I20260709 13:22:52.985464 21982 raft_consensus.cc:2468] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 0.
I20260709 13:22:52.982717 22165 raft_consensus.cc:515] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.986284 22124 raft_consensus.cc:359] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.997370 22124 raft_consensus.cc:385] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:52.997772 22124 raft_consensus.cc:740] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:52.998517 22124 consensus_queue.cc:260] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
W20260709 13:22:53.000353 21781 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:52.934099 22126 raft_consensus.cc:515] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:52.988783 21873 leader_election.cc:304] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b; no voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:52.989696 21947 leader_election.cc:304] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.001680 22165 leader_election.cc:290] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:53.002426 22165 raft_consensus.cc:2804] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:53.002954 22165 raft_consensus.cc:697] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.003213 22149 raft_consensus.cc:2749] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:53.004026 22165 consensus_queue.cc:237] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.005897 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:52.991735 21799 leader_election.cc:304] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:52.994719 22166 raft_consensus.cc:515] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.000901 22124 ts_tablet_manager.cc:1434] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 13:22:53.007339 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.008136 21910 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.008821 22124 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: Bootstrap starting.
I20260709 13:22:53.009516 22166 leader_election.cc:290] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.009263 22149 consensus_queue.cc:1048] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.016705 22124 tablet_bootstrap.cc:654] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:53.020092 22166 raft_consensus.cc:2804] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:53.020910 22166 raft_consensus.cc:493] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:53.021412 22166 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.022069 21908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.023200 21908 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.028636 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.032385 22124 tablet_bootstrap.cc:492] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: No bootstrap required, opened a new log
I20260709 13:22:53.035470 21834 raft_consensus.cc:2393] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 434c406d785b40b982cd98e21b7b97c2 in current term 1: Already voted for candidate c3b3294fda6a4b578c7e149e6990b94e in this term.
I20260709 13:22:53.039566 22126 leader_election.cc:290] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.040637 21984 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.041851 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.043546 21910 raft_consensus.cc:2468] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:53.044138 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.040864 22124 ts_tablet_manager.cc:1403] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.003s
I20260709 13:22:53.050873 21982 raft_consensus.cc:2393] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c3b3294fda6a4b578c7e149e6990b94e in current term 1: Already voted for candidate 434c406d785b40b982cd98e21b7b97c2 in this term.
I20260709 13:22:53.055052 21948 leader_election.cc:304] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:53.062016 22147 consensus_queue.cc:1048] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.062847 22165 raft_consensus.cc:2804] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:53.063620 22165 raft_consensus.cc:697] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.063279 22124 raft_consensus.cc:359] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.065001 22165 consensus_queue.cc:237] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.056092 21732 catalog_manager.cc:5697] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:53.066160 21984 raft_consensus.cc:2468] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 1.
I20260709 13:22:53.068110 22166 raft_consensus.cc:515] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.065485 21799 leader_election.cc:304] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:53.071163 22166 leader_election.cc:290] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.071873 22166 raft_consensus.cc:2749] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:53.073051 21799 leader_election.cc:304] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.073827 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.076076 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.056867 21908 raft_consensus.cc:2468] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 1.
I20260709 13:22:53.076918 21984 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.077152 22166 raft_consensus.cc:2804] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:53.080175 22166 raft_consensus.cc:697] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.065001 22124 raft_consensus.cc:385] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:53.081521 22124 raft_consensus.cc:740] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Initialized, Role: FOLLOWER
I20260709 13:22:53.081629 22166 consensus_queue.cc:237] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.082688 22124 consensus_queue.cc:260] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.088928 21910 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.091010 22124 ts_tablet_manager.cc:1434] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.004s
I20260709 13:22:53.092983 21984 raft_consensus.cc:2468] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 1.
I20260709 13:22:53.095172 21799 leader_election.cc:304] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.096318 22126 raft_consensus.cc:2804] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:53.096879 22126 raft_consensus.cc:697] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.097824 22126 consensus_queue.cc:237] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.112001 22147 consensus_queue.cc:1048] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.117884 21910 raft_consensus.cc:2468] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 1.
I20260709 13:22:53.141413 21730 catalog_manager.cc:5697] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:53.145875 22165 raft_consensus.cc:493] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:53.146588 22165 raft_consensus.cc:515] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.149070 22165 leader_election.cc:290] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:53.150581 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b" is_pre_election: true
I20260709 13:22:53.151540 21910 raft_consensus.cc:2468] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:53.152575 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e" is_pre_election: true
I20260709 13:22:53.153507 21834 raft_consensus.cc:2468] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 0.
I20260709 13:22:53.155591 21947 leader_election.cc:304] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.160588 22029 raft_consensus.cc:2804] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:53.161108 22029 raft_consensus.cc:493] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:53.161525 22029 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.161561 22165 consensus_queue.cc:1048] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.162993 21729 catalog_manager.cc:5697] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 0 to 1, leader changed from <none> to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.165246 21729 catalog_manager.cc:5697] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 0 to 1, leader changed from <none> to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 1 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.170910 22029 raft_consensus.cc:515] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.174775 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.175571 21834 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.184906 21834 raft_consensus.cc:2468] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:53.186399 21947 leader_election.cc:304] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.187902 22165 raft_consensus.cc:2804] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:53.187687 21908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.188724 21908 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:53.189186 22165 raft_consensus.cc:697] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.190483 22165 consensus_queue.cc:237] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.199677 21908 raft_consensus.cc:2468] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 1.
I20260709 13:22:53.201930 22029 leader_election.cc:290] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 1 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:53.218276 22165 consensus_queue.cc:1048] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.221778 21732 catalog_manager.cc:5697] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 1 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:53.360504 22165 consensus_queue.cc:1048] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:53.373799 22158 consensus_queue.cc:1048] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.427834 21910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:48892
I20260709 13:22:53.428670 21910 raft_consensus.cc:606] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:53.436362 21982 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:53.437146 21982 raft_consensus.cc:606] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:53.443955 21910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:48892
I20260709 13:22:53.444865 21910 raft_consensus.cc:606] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:53.451790 21984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:53.452401 21984 raft_consensus.cc:606] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS c3b3294fda6a4b578c7e149e6990b94e
I20260709 13:22:53.458873 21833 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:33938
I20260709 13:22:53.459715 21833 raft_consensus.cc:606] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.467485 21984 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:53.468281 21984 raft_consensus.cc:606] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.475523 21982 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:53.476207 21982 raft_consensus.cc:606] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.482880 21982 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
mode: GRACEFUL
new_leader_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:49388
I20260709 13:22:53.483578 21982 raft_consensus.cc:606] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Received request to transfer leadership to TS 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.491436 21833 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df75b588722b4192b84352600b2161a8"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
mode: GRACEFUL
new_leader_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:33938
I20260709 13:22:53.492228 21833 raft_consensus.cc:606] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Received request to transfer leadership to TS 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:53.526597 22158 consensus_queue.cc:1048] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.529601 22147 raft_consensus.cc:993] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.530385 22053 raft_consensus.cc:1081] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.533759 21834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33908
I20260709 13:22:53.534622 21834 raft_consensus.cc:493] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.535087 21834 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.546288 21834 raft_consensus.cc:515] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.547516 22029 consensus_queue.cc:1048] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.550007 21910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.550973 21910 raft_consensus.cc:3055] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.551442 21910 raft_consensus.cc:740] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.552407 21910 consensus_queue.cc:260] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.552999 21834 leader_election.cc:290] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.553864 21910 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.556648 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69df12953b4f2a9f02e8e48ebb3fdb" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.557431 21982 raft_consensus.cc:3060] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.561266 22165 raft_consensus.cc:993] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.561954 22165 raft_consensus.cc:1081] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.564159 21908 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:22:53.564916 21908 raft_consensus.cc:493] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.565352 21908 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.566047 21910 raft_consensus.cc:2468] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.567737 21799 leader_election.cc:304] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.568769 22126 raft_consensus.cc:2804] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.569317 22126 raft_consensus.cc:697] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.570312 22126 consensus_queue.cc:237] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.574069 21982 raft_consensus.cc:2468] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.577912 21908 raft_consensus.cc:515] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.580049 22126 consensus_queue.cc:1048] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.581038 21908 leader_election.cc:290] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.582288 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.583087 21834 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.583745 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a165d14408d1482e9e70ba2bef7f040b" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.584532 21982 raft_consensus.cc:3055] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.584932 21982 raft_consensus.cc:740] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.586064 21982 consensus_queue.cc:260] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.591954 21982 raft_consensus.cc:3060] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.594955 21729 catalog_manager.cc:5697] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.598657 21834 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.599772 22149 raft_consensus.cc:993] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.600333 22149 raft_consensus.cc:1081] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.600467 21873 leader_election.cc:304] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.601464 22053 raft_consensus.cc:2804] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.602000 22053 raft_consensus.cc:697] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.602648 21833 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33908
I20260709 13:22:53.603451 21833 raft_consensus.cc:493] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.602967 22053 consensus_queue.cc:237] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.603860 21833 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.616534 21833 raft_consensus.cc:515] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.618772 21833 leader_election.cc:290] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.620467 22165 consensus_queue.cc:1048] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:53.621104 21909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.622014 21909 raft_consensus.cc:3055] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.622416 21909 raft_consensus.cc:740] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.623373 21909 consensus_queue.cc:260] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.619812 21982 raft_consensus.cc:2468] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.624940 21909 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.623387 21984 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ceb7deddda64d9a9a92cbdd45ca1cf8" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.628360 21984 raft_consensus.cc:3060] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.635525 21909 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.638034 22126 consensus_queue.cc:1048] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.638962 21984 raft_consensus.cc:2468] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.648346 22029 raft_consensus.cc:993] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.648914 22029 raft_consensus.cc:1081] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.651530 21799 leader_election.cc:304] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.652558 22126 raft_consensus.cc:2804] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.653123 22126 raft_consensus.cc:697] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.653093 21729 catalog_manager.cc:5697] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.654439 22126 consensus_queue.cc:237] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.657475 21907 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:22:53.658375 21907 raft_consensus.cc:493] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.658771 21907 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.669369 22191 consensus_queue.cc:1048] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.670303 22164 consensus_queue.cc:1048] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.685547 21907 raft_consensus.cc:515] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.687065 21729 catalog_manager.cc:5697] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.687893 22164 consensus_queue.cc:1048] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.692126 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.692932 21834 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.709120 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad026d6c08b443c95483cf1dae10334" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.710213 21982 raft_consensus.cc:3055] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.711278 21982 raft_consensus.cc:740] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.711865 22192 raft_consensus.cc:993] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.711994 22165 raft_consensus.cc:993] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.712491 22192 raft_consensus.cc:1081] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.712565 22165 raft_consensus.cc:1081] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.714104 21834 raft_consensus.cc:2468] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.714771 22067 consensus_queue.cc:1048] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.716952 21982 consensus_queue.cc:260] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.716526 22192 raft_consensus.cc:993] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: : Instructing follower 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:53.719043 22192 raft_consensus.cc:1081] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Signalling peer 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:53.721098 21835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33922
I20260709 13:22:53.721957 21835 raft_consensus.cc:493] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.722390 21835 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.718637 22029 consensus_queue.cc:1048] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:22:53.727608 21910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48878
I20260709 13:22:53.728931 21910 raft_consensus.cc:493] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.729408 21910 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.730723 21873 leader_election.cc:304] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.732067 22053 raft_consensus.cc:2804] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.732636 22053 raft_consensus.cc:697] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.733603 22053 consensus_queue.cc:237] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.733976 21982 raft_consensus.cc:3060] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.734112 21983 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df75b588722b4192b84352600b2161a8"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:49384
I20260709 13:22:53.734936 21983 raft_consensus.cc:493] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.735451 21983 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.747262 21907 leader_election.cc:290] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.748724 21983 raft_consensus.cc:515] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.751510 21910 raft_consensus.cc:515] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.753928 21983 leader_election.cc:290] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:53.754899 21834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.755337 21907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df75b588722b4192b84352600b2161a8" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.756502 21907 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.756899 21835 raft_consensus.cc:515] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.758680 21834 raft_consensus.cc:3055] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.759238 21834 raft_consensus.cc:740] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.760673 21908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.760514 21834 consensus_queue.cc:260] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.762048 21834 raft_consensus.cc:3060] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.758225 21833 raft_consensus.cc:1240] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Rejecting Update request from peer 434c406d785b40b982cd98e21b7b97c2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:53.754596 21982 raft_consensus.cc:2468] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.767524 21832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.768296 21832 raft_consensus.cc:3055] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.768704 21832 raft_consensus.cc:740] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.769610 22165 consensus_queue.cc:1059] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:53.770551 22029 raft_consensus.cc:3055] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.770934 22029 raft_consensus.cc:740] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.771907 22029 consensus_queue.cc:260] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.772936 22029 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.773778 21907 raft_consensus.cc:2468] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 2.
I20260709 13:22:53.769693 21832 consensus_queue.cc:260] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.777259 21832 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.787524 21835 leader_election.cc:290] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.787197 21730 catalog_manager.cc:5697] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.787705 21910 leader_election.cc:290] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.788497 21982 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3dcb75d93364aa5bbd14334506279a5" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.790597 21948 leader_election.cc:304] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:53.791718 22029 raft_consensus.cc:2804] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.789906 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24371feb96a24bf68b9db97aa7dbc13f" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.792341 22029 raft_consensus.cc:697] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.797070 21834 raft_consensus.cc:2468] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 2.
I20260709 13:22:53.797199 22029 consensus_queue.cc:237] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.805109 21983 raft_consensus.cc:2468] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.808219 21799 leader_election.cc:304] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 019e784c7f884e0782e9dca6055b7a4b
I20260709 13:22:53.809777 22192 raft_consensus.cc:2804] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.810299 22192 raft_consensus.cc:697] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.812232 21832 raft_consensus.cc:2468] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.812448 22192 consensus_queue.cc:237] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.814292 21873 leader_election.cc:304] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:53.815363 22053 raft_consensus.cc:2804] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.815956 22053 raft_consensus.cc:697] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.816938 22053 consensus_queue.cc:237] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.821447 21730 catalog_manager.cc:5697] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 1 to 2, leader changed from c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65) to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 2 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:53.823921 21729 catalog_manager.cc:5697] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.832794 21732 catalog_manager.cc:5697] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.846540 22164 raft_consensus.cc:993] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.847229 22164 raft_consensus.cc:1081] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer c3b3294fda6a4b578c7e149e6990b94e to start an election
I20260709 13:22:53.849630 21832 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
 from {username='slave'} at 127.0.0.1:33922
I20260709 13:22:53.850972 21832 raft_consensus.cc:493] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.851464 21832 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.859367 21832 raft_consensus.cc:515] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.861375 21832 leader_election.cc:290] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Requested vote from peers 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.862557 21907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:53.863448 21907 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.863593 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba04bc19cddf4b8a9b20658ba3dac173" candidate_uuid: "c3b3294fda6a4b578c7e149e6990b94e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.864411 21983 raft_consensus.cc:3055] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.864746 21983 raft_consensus.cc:740] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.865661 21983 consensus_queue.cc:260] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.866880 21983 raft_consensus.cc:3060] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.870414 21907 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.872394 21799 leader_election.cc:304] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.873603 22067 raft_consensus.cc:2804] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.874151 22067 raft_consensus.cc:697] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Leader. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:53.875079 22067 consensus_queue.cc:237] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.875734 21983 raft_consensus.cc:2468] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3b3294fda6a4b578c7e149e6990b94e in term 2.
I20260709 13:22:53.877684 22158 raft_consensus.cc:993] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: : Instructing follower 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.878266 22029 raft_consensus.cc:1081] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Signalling peer 019e784c7f884e0782e9dca6055b7a4b to start an election
I20260709 13:22:53.880434 21907 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
 from {username='slave'} at 127.0.0.1:48862
I20260709 13:22:53.881135 21907 raft_consensus.cc:493] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:53.881498 21907 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.884986 21732 catalog_manager.cc:5697] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65). New cstate: current_term: 2 leader_uuid: "c3b3294fda6a4b578c7e149e6990b94e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.889745 21907 raft_consensus.cc:515] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.893428 21832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:53.893985 21983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe76cb5c2434e588405c55b2f5ceeaf" candidate_uuid: "019e784c7f884e0782e9dca6055b7a4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
I20260709 13:22:53.894202 21832 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.894707 21907 leader_election.cc:290] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67:40255)
I20260709 13:22:53.894870 21983 raft_consensus.cc:3055] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:53.895534 21983 raft_consensus.cc:740] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:53.896661 21983 consensus_queue.cc:260] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.898519 21983 raft_consensus.cc:3060] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:53.904997 21832 raft_consensus.cc:2468] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.906520 21873 leader_election.cc:304] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:53.907452 22053 raft_consensus.cc:2804] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:53.907999 22053 raft_consensus.cc:697] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Leader. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:53.908876 22053 consensus_queue.cc:237] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:53.910830 21983 raft_consensus.cc:2468] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019e784c7f884e0782e9dca6055b7a4b in term 2.
I20260709 13:22:53.918812 21732 catalog_manager.cc:5697] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b reported cstate change: term changed from 1 to 2, leader changed from 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67) to 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66). New cstate: current_term: 2 leader_uuid: "019e784c7f884e0782e9dca6055b7a4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:53.964303 21907 raft_consensus.cc:1275] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:53.966058 22067 consensus_queue.cc:1048] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:53.978041 21983 raft_consensus.cc:1275] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:53.980701 22067 consensus_queue.cc:1048] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.078550 21983 raft_consensus.cc:1275] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.080168 22053 consensus_queue.cc:1048] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:54.091099 21832 raft_consensus.cc:1275] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.093283 22149 consensus_queue.cc:1048] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.125468 21983 raft_consensus.cc:1275] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.127534 22067 consensus_queue.cc:1048] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.139884 21907 raft_consensus.cc:1275] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.142335 22067 consensus_queue.cc:1048] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.213816 21983 raft_consensus.cc:1275] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.215020 21984 raft_consensus.cc:3060] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.215586 22067 consensus_queue.cc:1048] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.224462 21984 raft_consensus.cc:1275] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.226711 21907 raft_consensus.cc:3060] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.227039 22149 consensus_queue.cc:1048] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:22:54.233713 21910 raft_consensus.cc:1275] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.236467 22029 consensus_queue.cc:1048] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.238633 21907 raft_consensus.cc:1275] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.244580 22067 consensus_queue.cc:1048] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.248118 21832 raft_consensus.cc:1275] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.250584 22053 consensus_queue.cc:1048] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:54.262293 21834 raft_consensus.cc:1275] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.264719 22029 consensus_queue.cc:1048] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.275396 21984 raft_consensus.cc:1275] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.282965 22149 consensus_queue.cc:1048] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.290575 21983 raft_consensus.cc:1275] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.297737 22149 consensus_queue.cc:1048] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:22:54.305852 21832 raft_consensus.cc:1275] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.308818 22146 consensus_queue.cc:1048] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.324515 21832 raft_consensus.cc:1275] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Refusing update from remote peer 019e784c7f884e0782e9dca6055b7a4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.326923 22053 consensus_queue.cc:1048] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:54.347257 21983 raft_consensus.cc:1275] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.349043 22192 consensus_queue.cc:1048] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:54.359920 21907 raft_consensus.cc:1275] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Refusing update from remote peer c3b3294fda6a4b578c7e149e6990b94e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:54.361747 22126 consensus_queue.cc:1048] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:56.496821 21733 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52912:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:22:56.520797 21733 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52912:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:22:56.553831  8925 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:56.636488 22226 log.cc:927] Time spent T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:22:56.759398 22229 log.cc:927] Time spent T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:22:56.826489 22241 tablet.cc:1724] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:56.929920 22245 log.cc:927] Time spent T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:22:56.836932 22248 tablet.cc:1724] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:56.934571 21910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065"
dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
mode: GRACEFUL
new_leader_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:52864
I20260709 13:22:56.990657 22259 tablet.cc:1724] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:57.014282 22252 log.cc:927] Time spent T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Append to log took a long time: real 0.087s	user 0.002s	sys 0.000s
I20260709 13:22:56.990670 21910 raft_consensus.cc:606] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Received request to transfer leadership to TS 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:57.023749 22241 tablet.cc:1724] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:57.049082 22255 log.cc:927] Time spent T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:22:57.063783 22256 tablet.cc:1724] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:57.053406 21947 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:34695, user_credentials={real_user=slave}} blocked reactor thread for 52030.2us
I20260709 13:22:57.139104 22263 tablet.cc:1724] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.183892 22264 tablet.cc:1724] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:22:57.186913 21873 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.65:34695, user_credentials={real_user=slave}} blocked reactor thread for 130663us
W20260709 13:22:57.196892 21874 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.183.67:40255, user_credentials={real_user=slave}} blocked reactor thread for 167998us
I20260709 13:22:57.197862 22248 tablet.cc:1724] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.193758 22241 tablet.cc:1724] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.227700 22270 tablet.cc:1724] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.238772 22272 tablet.cc:1724] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.250085 22271 tablet.cc:1724] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.269155 22053 raft_consensus.cc:993] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: : Instructing follower 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:57.269953 22053 raft_consensus.cc:1081] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Signalling peer 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:57.272085 22273 tablet.cc:1724] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.273059 21981 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:49368
I20260709 13:22:57.274255 21981 raft_consensus.cc:493] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:57.274941 21981 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.290812 22276 tablet.cc:1724] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.331262 22277 tablet.cc:1724] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.339587 22259 tablet.cc:1724] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.371789 22264 tablet.cc:1724] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.374594 22248 tablet.cc:1724] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.396427 22270 tablet.cc:1724] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.385432 21981 raft_consensus.cc:515] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.403644 21833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:57.404749 21833 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.401202 22248 tablet.cc:1724] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.421646 22256 tablet.cc:1724] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.423884 22271 tablet.cc:1724] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.424153 22241 tablet.cc:1724] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.444252 21981 leader_election.cc:290] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 3 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:57.445930 21905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4807ded299c54b8a8ec9f561db684065" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:57.446767 21905 raft_consensus.cc:3055] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:57.447168 21905 raft_consensus.cc:740] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 019e784c7f884e0782e9dca6055b7a4b, State: Running, Role: LEADER
I20260709 13:22:57.448263 21905 consensus_queue.cc:260] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.445791 22263 tablet.cc:1724] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.449929 21905 raft_consensus.cc:3060] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.457795 22272 tablet.cc:1724] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.462196 21833 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 3.
I20260709 13:22:57.449820 21730 catalog_manager.cc:5789] TS c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695) does not have the latest schema for tablet 3d69df12953b4f2a9f02e8e48ebb3fdb (table soft_deleted_table [id=526e38cc2e844c19852625857da9d279]). Expected version 1 got 0
I20260709 13:22:57.464928 21730 catalog_manager.cc:5789] TS c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695) does not have the latest schema for tablet 24371feb96a24bf68b9db97aa7dbc13f (table soft_deleted_table [id=526e38cc2e844c19852625857da9d279]). Expected version 1 got 0
I20260709 13:22:57.466827 22277 tablet.cc:1724] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.492864 21947 leader_election.cc:304] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 434c406d785b40b982cd98e21b7b97c2, c3b3294fda6a4b578c7e149e6990b94e; no voters: 
I20260709 13:22:57.493201 21832 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5"
dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
mode: GRACEFUL
new_leader_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:50756
I20260709 13:22:57.494019 21832 raft_consensus.cc:606] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Received request to transfer leadership to TS 434c406d785b40b982cd98e21b7b97c2
I20260709 13:22:57.494292 22231 raft_consensus.cc:2804] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:57.494939 22231 raft_consensus.cc:697] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 3 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:57.498481  8925 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:22:57.500039  8925 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:57.501557 22231 consensus_queue.cc:237] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.506989 22264 tablet.cc:1724] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:22:57.526546 21905 raft_consensus.cc:2468] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 3.
I20260709 13:22:57.547497 22264 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:33902 (request call id 21) took 1010 ms. Trace:
I20260709 13:22:57.554493 21733 catalog_manager.cc:5697] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 2 to 3, leader changed from 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66) to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:57.548243 22264 rpcz_store.cc:276] 0709 13:22:56.537454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:56.538490 (+  1036us) service_pool.cc:224] Handling call
0709 13:22:57.547374 (+1008884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:56.614730 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 13:22:56.615449 (+   719us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 13:22:56.615488 (+    39us) alter_schema_op.cc:66] Acquired schema lock
0709 13:22:56.615582 (+    94us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 13:22:56.615756 (+   174us) alter_schema_op.cc:128] START. Timestamp: 1783603376615671
0709 13:22:56.615780 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:22:56.616598 (+   818us) log.cc:844] Serialized 204 byte log entry
0709 13:22:56.785213 (+168615us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa274d9fdb1 00007fa274e3dc85 00007fa274e3e6a2 00007fa277bb01db 00007fa277babaaf 00007fa274efa0ac 00007fa27dd54ab9 00007fa274f1669b 00007fa274f1493f 00007fa274f16c02 00007fa277b9e20b
0709 13:22:57.329829 (+544616us) op_driver.cc:464] REPLICATION: finished
0709 13:22:57.506690 (+176861us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 13:22:57.518629 (+ 11939us) log.cc:844] Serialized 5 byte log entry
0709 13:22:57.530218 (+ 11589us) tablet_metadata.cc:665] Metadata flushed
0709 13:22:57.531440 (+  1222us) log.cc:844] Serialized 14 byte log entry
0709 13:22:57.533020 (+  1580us) alter_schema_op.cc:73] Released schema lock
0709 13:22:57.534037 (+  1017us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":176650,"prepare.queue_time_us":75017,"prepare.run_cpu_time_us":11456,"prepare.run_wall_time_us":232403,"raft.queue_time_us":13579,"raft.run_cpu_time_us":4908,"raft.run_wall_time_us":75432,"replication_time_us":713840,"spinlock_wait_cycles":4425088,"thread_start_us":182152,"threads_started":2,"wal-append.queue_time_us":215412}]]}
I20260709 13:22:57.561115 22272 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:33902 (request call id 20) took 1025 ms. Trace:
I20260709 13:22:57.561724 22272 rpcz_store.cc:276] 0709 13:22:56.536092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:56.536249 (+   157us) service_pool.cc:224] Handling call
0709 13:22:57.561009 (+1024760us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:56.593349 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 13:22:56.597047 (+  3698us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 13:22:56.597104 (+    57us) alter_schema_op.cc:66] Acquired schema lock
0709 13:22:56.597228 (+   124us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 13:22:56.597447 (+   219us) alter_schema_op.cc:128] START. Timestamp: 1783603376597334
0709 13:22:56.597473 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:22:56.607916 (+ 10443us) log.cc:844] Serialized 204 byte log entry
0709 13:22:57.045869 (+437953us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b58003a7fd0. stack: 00007fa274ee5343 00007fa274ee5240 00007fa2748741d4 00007fa27e5b2304 00007fa27e5b2299 00007fa277f6e090 00007fa277f5e43c 00007fa277f64d53 00007fa277f64cd9 00007fa277f64c69 00007fa277f64c31 00007fa277f63dfd 000000000038f354 000000000038f199 00007fa274f17475 00007fa274f197c1
0709 13:22:57.262933 (+217064us) op_driver.cc:464] REPLICATION: finished
0709 13:22:57.457573 (+194640us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 13:22:57.486951 (+ 29378us) log.cc:844] Serialized 5 byte log entry
0709 13:22:57.558698 (+ 71747us) tablet_metadata.cc:665] Metadata flushed
0709 13:22:57.559841 (+  1143us) log.cc:844] Serialized 14 byte log entry
0709 13:22:57.560795 (+   954us) alter_schema_op.cc:73] Released schema lock
0709 13:22:57.560846 (+    51us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":194355,"mutex_wait_us":9581,"prepare.queue_time_us":55759,"prepare.run_cpu_time_us":31212,"prepare.run_wall_time_us":186604,"raft.queue_time_us":17609,"raft.run_cpu_time_us":2839,"raft.run_wall_time_us":267256,"replication_time_us":665135,"spinlock_wait_cycles":104208640,"thread_start_us":205964,"threads_started":2,"wal-append.queue_time_us":78351}]]}
I20260709 13:22:57.579499 21732 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: soft_deleted_table [id=526e38cc2e844c19852625857da9d279] alter complete (version 1)
I20260709 13:22:57.813310 22230 raft_consensus.cc:993] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: : Instructing follower 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:57.813947 22251 raft_consensus.cc:1081] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Signalling peer 434c406d785b40b982cd98e21b7b97c2 to start an election
I20260709 13:22:57.815740 21981 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5"
dest_uuid: "434c406d785b40b982cd98e21b7b97c2"
 from {username='slave'} at 127.0.0.1:49384
I20260709 13:22:57.816404 21981 raft_consensus.cc:493] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:57.816789 21981 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.825462 21981 raft_consensus.cc:515] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.829766 21981 leader_election.cc:290] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 3 election: Requested vote from peers c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695), 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507)
I20260709 13:22:57.830825 21833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3b3294fda6a4b578c7e149e6990b94e"
I20260709 13:22:57.832108 21833 raft_consensus.cc:3055] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:57.831894 21905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4fce7842e6c4feaa1a14e883ce636c5" candidate_uuid: "434c406d785b40b982cd98e21b7b97c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019e784c7f884e0782e9dca6055b7a4b"
I20260709 13:22:57.832723 21833 raft_consensus.cc:740] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c3b3294fda6a4b578c7e149e6990b94e, State: Running, Role: LEADER
I20260709 13:22:57.832962 21905 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.833909 21833 consensus_queue.cc:260] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.835626 21833 raft_consensus.cc:3060] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.842448 21905 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 3.
I20260709 13:22:57.844329 21833 raft_consensus.cc:2468] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 434c406d785b40b982cd98e21b7b97c2 in term 3.
I20260709 13:22:57.844457 21948 leader_election.cc:304] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019e784c7f884e0782e9dca6055b7a4b, 434c406d785b40b982cd98e21b7b97c2; no voters: 
I20260709 13:22:57.845920 22231 raft_consensus.cc:2804] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:57.846572 22231 raft_consensus.cc:697] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 3 LEADER]: Becoming Leader. State: Replica: 434c406d785b40b982cd98e21b7b97c2, State: Running, Role: LEADER
I20260709 13:22:57.847659 22231 consensus_queue.cc:237] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } }
I20260709 13:22:57.858346 21730 catalog_manager.cc:5697] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 reported cstate change: term changed from 2 to 3, leader changed from c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65) to 434c406d785b40b982cd98e21b7b97c2 (127.8.183.67). New cstate: current_term: 3 leader_uuid: "434c406d785b40b982cd98e21b7b97c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "434c406d785b40b982cd98e21b7b97c2" member_type: VOTER last_known_addr { host: "127.8.183.67" port: 40255 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:57.935453 21908 raft_consensus.cc:1275] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:57.937379 22231 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:57.952500 21833 raft_consensus.cc:1275] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:57.954664 22029 consensus_queue.cc:1048] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:58.400629 21833 raft_consensus.cc:1275] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.400629 21908 raft_consensus.cc:1275] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Refusing update from remote peer 434c406d785b40b982cd98e21b7b97c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.402869 22287 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019e784c7f884e0782e9dca6055b7a4b" member_type: VOTER last_known_addr { host: "127.8.183.66" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:58.403836 22029 consensus_queue.cc:1048] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3b3294fda6a4b578c7e149e6990b94e" member_type: VOTER last_known_addr { host: "127.8.183.65" port: 34695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:58.586215  8925 tablet_server.cc:179] TabletServer@127.8.183.65:0 shutting down...
I20260709 13:22:58.628630  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:58.629937  8925 tablet_replica.cc:333] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.631708  8925 raft_consensus.cc:2243] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.632445  8925 raft_consensus.cc:2272] T 3fe76cb5c2434e588405c55b2f5ceeaf P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.635403  8925 tablet_replica.cc:333] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.636370  8925 raft_consensus.cc:2243] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.637048  8925 raft_consensus.cc:2272] T 8ad026d6c08b443c95483cf1dae10334 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.638762 21873 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 13 similar messages]
I20260709 13:22:58.640688  8925 tablet_replica.cc:333] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.642102  8925 raft_consensus.cc:2243] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.642843  8925 raft_consensus.cc:2272] T d3dcb75d93364aa5bbd14334506279a5 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.643829 21873 consensus_peers.cc:597] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.646272  8925 tablet_replica.cc:333] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.647294  8925 raft_consensus.cc:2243] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.647900  8925 raft_consensus.cc:2272] T df75b588722b4192b84352600b2161a8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.650903  8925 tablet_replica.cc:333] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.651839  8925 raft_consensus.cc:2243] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.653111  8925 raft_consensus.cc:2272] T 24371feb96a24bf68b9db97aa7dbc13f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.655419 21873 consensus_peers.cc:597] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.656625  8925 tablet_replica.cc:333] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.657434  8925 raft_consensus.cc:2243] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.658419  8925 raft_consensus.cc:2272] T 5466e4df67bb43c599d51a521771792f P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.661007  8925 tablet_replica.cc:333] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.661676  8925 raft_consensus.cc:2243] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.662601  8925 raft_consensus.cc:2272] T 3d69df12953b4f2a9f02e8e48ebb3fdb P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.664978  8925 tablet_replica.cc:333] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.665637  8925 raft_consensus.cc:2243] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.666141  8925 raft_consensus.cc:2272] T 14a9ebf91f79450f904dbccb7d51ea68 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.668761  8925 tablet_replica.cc:333] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.669446  8925 raft_consensus.cc:2243] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.669956  8925 raft_consensus.cc:2272] T a165d14408d1482e9e70ba2bef7f040b P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.672309  8925 tablet_replica.cc:333] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.672942  8925 raft_consensus.cc:2243] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.673868  8925 raft_consensus.cc:2272] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.676568  8925 tablet_replica.cc:333] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.677165  8925 raft_consensus.cc:2243] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.677610  8925 raft_consensus.cc:2272] T a4fce7842e6c4feaa1a14e883ce636c5 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.681474  8925 tablet_replica.cc:333] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.682325  8925 raft_consensus.cc:2243] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.683027  8925 raft_consensus.cc:2272] T a5e0712f6a964772870a3eca408dac96 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.686002  8925 tablet_replica.cc:333] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.686820  8925 raft_consensus.cc:2243] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.687487  8925 raft_consensus.cc:2272] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P c3b3294fda6a4b578c7e149e6990b94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.691423  8925 tablet_replica.cc:333] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.692229  8925 raft_consensus.cc:2243] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.693040  8925 raft_consensus.cc:2272] T 6d5539e7e2024cf38082a16791bfc8c8 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.696609 21947 consensus_peers.cc:597] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.697005  8925 tablet_replica.cc:333] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.697926  8925 raft_consensus.cc:2243] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.699085  8925 raft_consensus.cc:2272] T ba04bc19cddf4b8a9b20658ba3dac173 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.702399  8925 tablet_replica.cc:333] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.703225  8925 raft_consensus.cc:2243] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.703804  8925 raft_consensus.cc:2272] T 4807ded299c54b8a8ec9f561db684065 P c3b3294fda6a4b578c7e149e6990b94e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.706115 21873 consensus_peers.cc:597] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.707696  8925 tablet_replica.cc:333] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.708523  8925 raft_consensus.cc:2243] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.709555  8925 raft_consensus.cc:2272] T 1fb45406e4684da6b8696c4c0f5da2f3 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.712630  8925 tablet_replica.cc:333] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e: stopping tablet replica
I20260709 13:22:58.713481  8925 raft_consensus.cc:2243] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.714844  8925 raft_consensus.cc:2272] T 0693a0ddbfa747b7b5c8ef13feeee767 P c3b3294fda6a4b578c7e149e6990b94e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.757755  8925 tablet_server.cc:196] TabletServer@127.8.183.65:0 shutdown complete.
I20260709 13:22:58.813549  8925 tablet_server.cc:179] TabletServer@127.8.183.66:0 shutting down...
W20260709 13:22:58.842864 21948 consensus_peers.cc:597] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 -> Peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Couldn't send request to peer 019e784c7f884e0782e9dca6055b7a4b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.857259  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:58.858199  8925 tablet_replica.cc:333] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.859284  8925 raft_consensus.cc:2243] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.860311  8925 raft_consensus.cc:2272] T 8ad026d6c08b443c95483cf1dae10334 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.862851  8925 tablet_replica.cc:333] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.863754  8925 raft_consensus.cc:2243] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.864367  8925 raft_consensus.cc:2272] T 24371feb96a24bf68b9db97aa7dbc13f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.866683  8925 tablet_replica.cc:333] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.867556  8925 raft_consensus.cc:2243] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.868592  8925 raft_consensus.cc:2272] T 3fe76cb5c2434e588405c55b2f5ceeaf P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.871384  8925 tablet_replica.cc:333] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.872313  8925 raft_consensus.cc:2243] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.873054  8925 raft_consensus.cc:2272] T df75b588722b4192b84352600b2161a8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.876995  8925 tablet_replica.cc:333] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.878156  8925 raft_consensus.cc:2243] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:58.878495 21948 consensus_peers.cc:597] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 -> Peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Couldn't send request to peer 019e784c7f884e0782e9dca6055b7a4b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.879511  8925 raft_consensus.cc:2272] T d3dcb75d93364aa5bbd14334506279a5 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.879963 21873 consensus_peers.cc:597] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:34695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.882647  8925 tablet_replica.cc:333] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.883596  8925 raft_consensus.cc:2243] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.884616  8925 raft_consensus.cc:2272] T a165d14408d1482e9e70ba2bef7f040b P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.887235  8925 tablet_replica.cc:333] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.887889  8925 raft_consensus.cc:2243] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.888410  8925 raft_consensus.cc:2272] T a4fce7842e6c4feaa1a14e883ce636c5 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.892273  8925 tablet_replica.cc:333] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.893023  8925 raft_consensus.cc:2243] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.894037  8925 raft_consensus.cc:2272] T a5e0712f6a964772870a3eca408dac96 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.896816  8925 tablet_replica.cc:333] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.897501  8925 raft_consensus.cc:2243] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.898065  8925 raft_consensus.cc:2272] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.900836  8925 tablet_replica.cc:333] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.901576  8925 raft_consensus.cc:2243] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.902153  8925 raft_consensus.cc:2272] T 0693a0ddbfa747b7b5c8ef13feeee767 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.905400  8925 tablet_replica.cc:333] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.906385  8925 raft_consensus.cc:2243] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.907225  8925 raft_consensus.cc:2272] T ba04bc19cddf4b8a9b20658ba3dac173 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.910274  8925 tablet_replica.cc:333] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.911281  8925 raft_consensus.cc:2243] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.912043  8925 raft_consensus.cc:2272] T 4807ded299c54b8a8ec9f561db684065 P 019e784c7f884e0782e9dca6055b7a4b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.916945  8925 tablet_replica.cc:333] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.917881  8925 raft_consensus.cc:2243] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:58.918430 21947 consensus_peers.cc:597] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:34695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.918648  8925 raft_consensus.cc:2272] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.921723  8925 tablet_replica.cc:333] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.922366  8925 raft_consensus.cc:2243] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.922850  8925 raft_consensus.cc:2272] T 14a9ebf91f79450f904dbccb7d51ea68 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.925479  8925 tablet_replica.cc:333] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.926131  8925 raft_consensus.cc:2243] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.927381  8925 raft_consensus.cc:2272] T 6d5539e7e2024cf38082a16791bfc8c8 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.930452  8925 tablet_replica.cc:333] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.931075  8925 raft_consensus.cc:2243] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:58.932309  8925 raft_consensus.cc:2272] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 019e784c7f884e0782e9dca6055b7a4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.935072  8925 tablet_replica.cc:333] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.935807  8925 raft_consensus.cc:2243] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.936278  8925 raft_consensus.cc:2272] T 1fb45406e4684da6b8696c4c0f5da2f3 P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:58.938841  8925 tablet_replica.cc:333] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b: stopping tablet replica
I20260709 13:22:58.939461  8925 raft_consensus.cc:2243] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:58.939939  8925 raft_consensus.cc:2272] T 5466e4df67bb43c599d51a521771792f P 019e784c7f884e0782e9dca6055b7a4b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:58.981746 21947 consensus_peers.cc:597] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:34695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:58.983872  8925 tablet_server.cc:196] TabletServer@127.8.183.66:0 shutdown complete.
W20260709 13:22:58.992978 21948 consensus_peers.cc:597] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 -> Peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Couldn't send request to peer 019e784c7f884e0782e9dca6055b7a4b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:38507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:59.014613 21948 consensus_peers.cc:597] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 -> Peer 019e784c7f884e0782e9dca6055b7a4b (127.8.183.66:38507): Couldn't send request to peer 019e784c7f884e0782e9dca6055b7a4b. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.66:38507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.032526  8925 tablet_server.cc:179] TabletServer@127.8.183.67:0 shutting down...
I20260709 13:22:59.077195  8925 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:59.078611  8925 tablet_replica.cc:333] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
W20260709 13:22:59.079525 21947 consensus_peers.cc:597] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 -> Peer c3b3294fda6a4b578c7e149e6990b94e (127.8.183.65:34695): Couldn't send request to peer c3b3294fda6a4b578c7e149e6990b94e. Status: Network error: Client connection negotiation failed: client connection to 127.8.183.65:34695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.079954  8925 raft_consensus.cc:2243] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.080691  8925 raft_consensus.cc:2272] T d3dcb75d93364aa5bbd14334506279a5 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.083876  8925 tablet_replica.cc:333] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.084986  8925 raft_consensus.cc:2243] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.086091  8925 raft_consensus.cc:2272] T df75b588722b4192b84352600b2161a8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.089697  8925 tablet_replica.cc:333] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.090565  8925 raft_consensus.cc:2243] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.091255  8925 raft_consensus.cc:2272] T 24371feb96a24bf68b9db97aa7dbc13f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.093547  8925 tablet_replica.cc:333] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.094264  8925 raft_consensus.cc:2243] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.094816  8925 raft_consensus.cc:2272] T 8ad026d6c08b443c95483cf1dae10334 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.097061  8925 tablet_replica.cc:333] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.097819  8925 raft_consensus.cc:2243] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.098374  8925 raft_consensus.cc:2272] T 3fe76cb5c2434e588405c55b2f5ceeaf P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.100601  8925 tablet_replica.cc:333] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.101300  8925 raft_consensus.cc:2243] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.101871  8925 raft_consensus.cc:2272] T ba04bc19cddf4b8a9b20658ba3dac173 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.104192  8925 tablet_replica.cc:333] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.105130  8925 raft_consensus.cc:2243] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.106616  8925 raft_consensus.cc:2272] T 4807ded299c54b8a8ec9f561db684065 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.109671  8925 tablet_replica.cc:333] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.110426  8925 raft_consensus.cc:2243] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.110996  8925 raft_consensus.cc:2272] T 3d3d31d96d8f4a38a997ed65bb87e6d0 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.113834  8925 tablet_replica.cc:333] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.114549  8925 raft_consensus.cc:2243] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.115097  8925 raft_consensus.cc:2272] T 6d5539e7e2024cf38082a16791bfc8c8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.117802  8925 tablet_replica.cc:333] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.118546  8925 raft_consensus.cc:2243] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.119167  8925 raft_consensus.cc:2272] T 1fb45406e4684da6b8696c4c0f5da2f3 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.122067  8925 tablet_replica.cc:333] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.122833  8925 raft_consensus.cc:2243] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.123466  8925 raft_consensus.cc:2272] T a165d14408d1482e9e70ba2bef7f040b P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.125965  8925 tablet_replica.cc:333] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.126688  8925 raft_consensus.cc:2243] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.127812  8925 raft_consensus.cc:2272] T a4fce7842e6c4feaa1a14e883ce636c5 P 434c406d785b40b982cd98e21b7b97c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.131424  8925 tablet_replica.cc:333] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.132133  8925 raft_consensus.cc:2243] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.132688  8925 raft_consensus.cc:2272] T a5e0712f6a964772870a3eca408dac96 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.135569  8925 tablet_replica.cc:333] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.136322  8925 raft_consensus.cc:2243] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.136900  8925 raft_consensus.cc:2272] T 5ceb7deddda64d9a9a92cbdd45ca1cf8 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.139467  8925 tablet_replica.cc:333] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.140157  8925 raft_consensus.cc:2243] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.140751  8925 raft_consensus.cc:2272] T 0693a0ddbfa747b7b5c8ef13feeee767 P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.143589  8925 tablet_replica.cc:333] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.144228  8925 raft_consensus.cc:2243] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.144796  8925 raft_consensus.cc:2272] T 3d69df12953b4f2a9f02e8e48ebb3fdb P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.147367  8925 tablet_replica.cc:333] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.147980  8925 raft_consensus.cc:2243] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.149099  8925 raft_consensus.cc:2272] T 14a9ebf91f79450f904dbccb7d51ea68 P 434c406d785b40b982cd98e21b7b97c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.151773  8925 tablet_replica.cc:333] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2: stopping tablet replica
I20260709 13:22:59.152369  8925 raft_consensus.cc:2243] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.152956  8925 raft_consensus.cc:2272] T 5466e4df67bb43c599d51a521771792f P 434c406d785b40b982cd98e21b7b97c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.202056  8925 tablet_server.cc:196] TabletServer@127.8.183.67:0 shutdown complete.
I20260709 13:22:59.247012  8925 master.cc:562] Master@127.8.183.126:38967 shutting down...
I20260709 13:22:59.281667  8925 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.282440  8925 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.282863  8925 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7472c397f9854384af99c427cb180428: stopping tablet replica
I20260709 13:22:59.306509  8925 master.cc:584] Master@127.8.183.126:38967 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12605 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (26176 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (449569 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:22:59.379956  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 432 messages since previous log ~15 seconds ago
I20260709 13:22:59.380251  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 432 messages since previous log ~15 seconds ago
I20260709 13:22:59.380479  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 428 messages since previous log ~15 seconds ago
I20260709 13:22:59.380667  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:22:59.380842  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:22:59.380996  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 3 messages since previous log ~280 seconds ago
I20260709 13:22:59.381147  8925 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 16 messages since previous log ~287 seconds ago
I20260709 13:22:59.381310  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 3 messages since previous log ~285 seconds ago
I20260709 13:22:59.381485  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~207 seconds ago
I20260709 13:22:59.381706  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 12 messages since previous log ~0 seconds ago
I20260709 13:22:59.382002  8925 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
